Endless Flexibility, The Enemy of Agile
I've come to the conclusion that the biggest enemy of true agility at the code level is a desire to make the code agile. Let me explain.
I needed to write a simple web service. Instead of just writing a quick servlet or equivalent, I decided to save time by using a REST framework. So, I picked up Jersey. It's been a nightmare.
First of all, Jersey has so many dependencies that it requires Maven to install it. That requirement should have set off loud warning bells, but I proceeded anyway.
Jersey is deeply layered, serving as a facade for several highly complex subsystems. In theory, you don't have to know about those, but in practice, they all have bugs and idiosyncrasies that you must work around, so you do have to know about them in some depth. The learning curve is nontrivial.
Hiding complexity does not get rid of complexity. When you have a problem (and you will), you'll be knee deep in complexity that you don't understand. If the outer layer makes the inner stuff inaccessible, then you're really stuck. The option changes from weeks-finding-the-problem to throw-it-away. Not a great choice. I'm never going to leverage the vast majority of the features that those complex subsystems support. Nonetheless, I will have to deal with their bugs and idiosyncrasies and the deadweight they add to my application.
More to the point, hardly any of this complexity is necessary because it implements a flexibility that I don't want or need. I'm writing a simple JSON-based REST service. My guess is that the majority of Jersey users don't need to do any more than that. I really don't need or want support for a dozen different data representations, for example.
Between Jersey, Java 8, Tomcat, and Eclipse, and all the
WSJunk packages, the system is incredibly delicate. Change one thing (a single private field declaration, for example), and suddenly nothing works. No error messages, no trace diagnostics (or any way to provide them in anything like enough depth), no compiler errors; just a 500 response from the server when you run a simple test. You have to code this stuff one word at a time. Change one word. Test. Change the next word. Test. You can't identify the culprit when the system fails. It just doesn't work.
I could, of course, start messing with other combinations of things to see if they work any better. Throw out Tomcat and use Grizzly. Throw out Jersey and use Wink. Throw out Eclipse and Use IntelliJ IDEA. Each of these changes are another few days. I'm half way through my two-week iteration, and haven't gotten any closer to code that's useful to anybody.
There's no way that I can develop in an agile way when I have these kinds of problems. Every minute I spend either learning or wrestling with an off-the-shelf package is waste. Having to learn about a feature that I don't want or need to work around a bug just adds insult to injury.
So, here's my analysis:
Jersey is too flexible. All the layering and complexity I mentioned is included in Jersey on the off chance that someone might need it, and that same problem extends down into the sublayers, all of which are much more flexible (and much larger, and much too complex) than necessary. The system solves problems that 90% of its users don't have, all the way down the stack. This flexibility adds complexity, and the complexity creates innumerable problems.
The desire to solve every possible problem that might come up in some context leads to failure. Solve the problem at hand, nothing more. Start with the application at hand. Everything that's not needed to implement that application, down to the function-argument level, has to go. Sure, write the code so that you can extend it later, but don't put those extensions into it right off the bat.
Don't layer on top of a generalized subsystem. If you don't leverage 100% of the underlying subsystem, don't use it. This rule is the Interface-Segregation Principle (one of the SOLID principles) applied to subsystems. Monolithic subsystems that can do anything are a really bad idea.
If you're writing a generic subsystem, use an architecture that's easily extended but simple at its core. If I really need XML or ATOM support, I should be able to plug in an XML- or ATOM-support module. It shouldn't be built in on the off chance that I might need it someday. Someday never comes. That plugin architecture will also let me add support for protocols that the original programmer didn't imagine.
The basic Agile principle: "Simplicity—the art of maximizing the amount of work not done—is essential" is indeed essential.