This section presents the results of our portals evaluation. As we expected, a core group of portals fit the generic model described in "What's (In) a Portal?" and share a basic feature set (see "Common Characters" below) but, at the same time, each tool has its own distinguishing characteristics, that basically reflect differences in their target markets.
Therefore, as well as making sure it has the right feature set, every developer intending to use one of these portals must ponder the following questions: Is the portal biased toward a particular development process or can it (easily) be used to support many different ones? Is it open source, closed source, or some mix of the two? Is it free to use or not? Is the portal deployed as a hosted service? Does the portal manage user identities itself? Can third-party tools be plugged in by end users using a well-defined API or is some combination of reprogramming and screen scraping needed to create mashups?, to ensure the portal suits his/her needs. It is also worth to note that even if two tools share the same set of features, the way they enable or emphasize them may result in a completely different usage (and with a different purpose).
All portals satisfy the two main conditions stated in "What's (In) a Portal?": they all offer some kind of repository and ticketing system for managing the project tasks. Support for communication tools (third condition on the list) is also high: all portals offer email alerts and RSS feeds to keep team members informed about project progress. Search capabilities turn out not to be so widespread, basic (predefined) searchs within the portal data are common but only a few portals offer advanced search features (see the next section).
Besides this core features, other aspects were found to be shared by many portals. All portals offer a role-based access control system with portal-managed identities, external authentication (e.g. supporting LDAP as Mingle and IBM Jazz do) or both. All also allow users to group tasks into milestones (though sometimes milestones are defined just as a special kind of task). Finally, most support multiple projects to be hosted in one portal (the exception is Trac, which only supports one) and let managers to roll up information across them.
Portals' Distinguishing Features
Beyond the pricing and licensing, the major differences between them reflect differences in their target markets. For example, Rally and VersionOne have been built by and for developers. They are used by some organizations' marketing and customer support groups, but their typical users are more likely to use CruiseControl than PhotoShop or Excel. BaseCamp, on the other hand, primarily targets small organizations like print shops that are staffed by non-programmers working on short- or medium-length projects; software developers are a minority of its users (though there are still many of them). It therefore offers an easy-to-use file upload and synchronization service rather than a full-blown version control system. In what follows, we briefly describe each portal focusing on their main distinguishing features.
The two leading commercial portals, Rally and VersionOne, match each other almost feature by feature. Both target medium-to-large agile teams, and both companies offer extensive training and consulting as part of the sales and deployment process to, in their own words, help groups using more "traditional" development lifecycles "go agile". They support template-based project creation with customization so that teams can quickly start using local adaptations of different agile processes, and project and task hierarchies with roll-ups for reporting. Rally decomposes user stories (i.e., a very high-level specification of a system requirement) into tasks, tests, and defects, while VersionOne allows project managers to redefine the schema of work items (i.e., the fields to be recorded for each task) at will.
These core features are complemented by predefined connectors to legacy ticketing systems and build tools. In particular, both accept Subversion and Microsoft Team Foundation Server as source code management (SCM) tools and JIRA and Bugzilla (among others) as defect management systems. Both also offer web services-based APIs to create additional connections and mash-ups.
Mingle is younger than these two systems, but similar in most respects: it uses Subversion and Perforce for SCM, and programmers can manually link cards (i.e., work items) and source code by including the card ID number in the commit comments of the code. One distinguishing feature is that requirements, tasks, and bugs are represented as hierarchical cards whose types and attributes can be adapted to the needs of each project. Another is that report generation is based on an SQL-like query language, which makes it easy for users to set up filters so that they can track what they are most interested in.
Acunote and Assembla are aimed at small-to-medium teams. Acunote offers a simple task management system for Scrum-based projects. Unlike some systems, tasks can span several Scrum sprints, which its creators feel reflects how projects unfold in real life. It emphasizes the use of time-tracking and burndown charts (type of chart that shows the remaining work in the current sprint) for monitoring progress. Tasks can be hierarchical but their structure is fixed. Assembla is a hosted version of Trac and Subversion supplemented with custom permission, ticketing, and team management modules, though it also offers built-in milestone and ticketing systems. It also includes a job/recruitment section and a time tracking system, which is important to the independent contractors and job-specific teams it is meant to support. In particular, the executive at Assembla commented that Assembla is designed to support the whole development process, including the billing and payment processes and that, as a result, time tracking is one of their most popular tools since it is how managers decide how to pay people.
The granddaddy (and inspiration, as many interviewees acknowledged) of all portals is SourceForge, which offers a free hosting service for open source projects (though credit should also go to Richard Hipp's CVSTrac, the first widely-used self-hosted web-based portal). Unlike the agile-oriented tools above, it is neutral with respect to development process -- in fact, the interviewees reported that most of the projects it hosts don't really have a "process" per se. Despite including a bug tracker and other project management tools, it is primarily used for project releases, hosting version control repositories, and managing its mailing lists.
According to the interviewee from Google, "Google Code was created in part to provide an insurance policy for the open source community in case SourceForge ever shut down". Its design is loosely based on Google's internal development tools and practices: for example, it is the only portal we examined that natively supports code review. Authentication is managed through GMail; like messages in that system, tasks have a predefined set of fields, but users can attach arbitrary labels to classify each task and use Google's free-text search technology to search this metadata. Projects can be connected to Subversion repositories but no other predefined integrations are available. A single issue tracking system manages software defects, change requests, technical-support requests, and development tasks.
Trac is a self-hosting portal that was originally designed as a replacement for CVSTrac. Each Trac installation can host a single project, though an extension allows a single project to span multiple version control repositories. Work is defined by tickets and milestones; the attributes of tickets can be redefined, but unlike other tools, this requires some programming effort. While it is very popular, its development has slowed in the past few years: each new release has largely been the effort of one dedicated volunteer, who usually then moves on to other interests (according to the interviewee from Trac: "We cannot really say that we plan ahead for some big features and they get done"). Like SourceForge, it is neutral with respect to the development process.
BaseCamp is an example of how a system aimed at a broader audience can be used in software development. It offers a file upload system instead of a version control system, to-do lists instead of tasks and tickets, and makes it easy to add external stakeholders with limited permissions to projects. On the other hand, it deliberately does not offer more sophisticated tools such as burndown charts, and instead of providing connectors to defect management and control quality tools, it focuses on integration with third-party invoicing, billing and accounting applications. Its developer explicitly aimed to support certain collaboration and project management practices, and to keep it so simple that, in his own words, "...learning to use the application was not a project in itself."
The self-hosting portal DotProject lies somewhere between BaseCamp and Trac: it has a fine-grained Access Control List (ACL) based permissioning system and hierarchical task management, but only limited support for releases or milestones (milestones are just a special kind of tasks), and does not attempt to manage code.
Finally, IBM's new Jazz project is more an extensible technology platform than a portal (e.g. Rational Team Concert, the IBM collaborative software delivery environment, is built upon it). Its design draws heavily on IBM's experience with Eclipse, which is also a platform that happens to be very good for building IDEs. Unlike the other systems we studied, Jazz has a custom two-level version control system: each developer can commit changes to a local repository, which can then be dumped into the shared public one. It is also the only portal that allows multiple branches (called "development lines") for the same project. As fully-distributed version control systems such as Git and Bazaar become more popular, we expect to see other portals move to a similar model.