Last week's editorial (Just Let Me Code!) went viral and received literally hundreds of comments on the main developer aggregators and on this site. By and large, readers were very familiar with the frustration I expressed of not being able to code as much as I like because I'm spending time taming the tools that support my work: the IDE, the VCS, the defect tracker, test tools, and especially the build system. (I could just as easily have added the database, the deployment stack, the cloud, and so on.) My conclusion was that this tool-level complexity (as opposed to the complexity of the code itself) had turned programming into an operational activity rather than a creative one.
The pressing question is what can be done to restore a more favorable balance? The simple and obvious answer would be better tools, especially friendlier tools. It's also the least likely path to resolution. Tool vendors have several misperceptions that stand in the way. The first is a long-standing issue, which is "featuritis": the tendency to create the perception of greater value in upgrades by adding rarely needed features. I'll return to this in a moment. The second misperception is that many tool vendors view the user experience they offer as already pretty darn good. Compared with tools we had 10 years ago or more, UIs have indeed improved significantly. But they have not improved as fast as complexity has increased. And in that gap lies the problem.
That problem is most acutely felt by two groups: solo developers and those working either in SMBs or for-hire at a client site. Those least affected are programmers at enterprises, which can afford dedicated staff to manage the toolchain, including especially the build, the testing, and the production pipeline.
When considering the most affected developers, it's valuable to focus on what consultants do. They're often brought in and paid to get the job done fast and well. They incarnate rapid app development (RAD), and increasingly, they use omnibus solutions. Typically, these come from a single vendor and provide a top-to-bottom set of integrated tools. Progress Software's Pacific platform is an example of this, as is OutSystems. In both cases, there is some early learning of how the walled garden works, but once learned, rapid development is greatly facilitated.
Interestingly, both solutions depend in part on Platform as a Service (PaaS) clouds and I expect that long-term, this trend will be the principal solution. Toolchains will be assembled and run in the cloud by vendors who specialize in tool integration effectively filling the role of the enterprise IT staff who do this work at enterprise sites today. Several companies are already heading in this direction. The first one to provide a complete, integrated development toolchain with swappable components will likely enjoy a significant success.
For individual developers, the closest you can come now is using host platforms such as BitBucket or GitHub. Both are widely respected open-source hosts that also support private (that is, not publicly accessible) projects. They offer turnkey, integrated solutions for SCM, defect tracking, and code reviews. GitHub adds features more frequently than BitBucket, but the latter offers free private directories and offers integration with scads of existing tools.
For small development organizations, either the PaaS or the hosted options have considerable appeal and enough history to suggest that they provide plenty of lift.
But let's drill into the nub of the problem. The reason that so many tools have become unwieldy is that they chase after certain values that add complexity but not usefulness to SMB-sized projects. For example, in the conversation I linked to last time, XML co-developer Tim Bray was struggling with Gradle, a build tool we covered recently in Dr. Dobb's. Bray was complaining of having to read through a 60-chapter document to get what he needed. Hans Dockter, the primary developer of Gradle, responded that the product was designed principally for scalability. This is an important detail. If you're considering Gradle, is scalability the problem you're trying to solve? If not, you might well be giving up simplicity for a capability that will not be used.
If this observation sounds familiar, it comes from coding and is articulated as YAGNI (You ain't gonna need it.) The thrust of YAGNI is that you should not code features that are not immediately needed in the current sprint. Adding code and hence complexity for features you will possibly use in the future is an anti-pattern. While YAGNI is rarely discussed outside of coding, it should be extended to tool selection. Choose tools that do the least amount more than you need. If Ant is good enough for your builds, use Ant. If you need some scripting, use Gant. If you need hugely more scripting and greater scalability, use Gradle.
If Hudson or TeamCity work well enough for your continuous integration needs, you don't need anything more complicated. If Subversion is good enough and you know Subversion, use it. Don't be pressured by the perceived popularity of Git to adopt Git. (Obviously, if you want to explore Git for your own edification, I'm not recommending against that. But understand that that choice will add complexity to your other work.)
The principle of YAGNI is remarkably freeing when it comes to tool selection, even for enterprise shops. Not only does it do away with unneeded labor, but it often reveals that many of today's technologies du jour are hot because they're popular in small, but vocal segments. Chasing after the latest fad will only occasionally bring happiness, but will always add complexity. This is where a good architect can be particularly valuable to a team, choosing the technologies that fit the project needs without adding unnecessary capabilities. To do this well, architects need to be courageous, as they will always be opposed by vendors enamored of new product features and geeks enamored of new technologies.