Over the past year Scott Leslie from BCcampus has been investigating various potential collaborative authoring environments for open textbooks. The key elements of such an environment are that it

• be web-based and allow for public viewing of the content
• facilitate collaboration by multiple authors
• provide multiple output formats, including basic ones like HTML, PDF and ideally ebook formats
• ideally that it allow for new versions and remixes of the textbook to be easily created

There are likely others, these are just some high level ones used to try to constrain the scope.
Many of these have been installed on the BCcampus Sandbox server at http://sandbox.bccampus.ca/open-textbook-authoring/.
In addition, a key component of open projects is how well they attract additional collaborators and reusers. A major strategy in this regards needs to be using existing open textbook platforms for development. In this regard both Connexions and Wikieducator are major contenders for consideration, as they both have sizeable user populations already focused on creating open educational resources. – SWL