But What Difference Will the Theory Make?
This isn't just something that affects methodologists, process geeks, and academics; it is something that will benefit everybody involved in software development. But what difference will it really make?
What's in it for the industry?
Most large companies have their own home-grown method or process, one which is a soup of standard ideas complemented by some of their own more business specific ideas. Usually these processes are described in a thick book or a website, and lots of money has been invested to document them. Sometimes people are trained on the process, sometimes they are just pointed toward it.
In reality the process is often ignored; the only parts that are actually used are the parts that form the "oral tradition" of the organization. This is explained by the re-discovered law of nature: people don't read process books. New ideas are introduced into the organization, and the old processes fall out of fashion and their books become shelf-ware.
In these large companies there may even be more than one process. For example the big system integrators may have 10 or 20 different processes. Sometimes they are quite similar, but the similarities are hidden behind all the cosmetic differences.
If your company was to adopt the practice idea, you wouldn't need to throw out your entire way of working because something new and sexy is becoming popular. Instead you would improve your existing way of working one practice at a time. You could even adopt practices used by other companies without throwing out those existing practices that seem to be working well.
To get started you would look at your existing way of working as a set of practices. Then you would look for your pain points and complement your existing way of working by removing any practices that are not working and replacing them with practices that address the weaknesses. This is easy to do once you have understood the kernel and its usage.
In a large organization with many different ways of working, you can use this approach to successively improve each way of working without forcing everyone to use the same method or process.
This approach will make it easier to adopt new practices without having to change the other practices you have. Imagine that you have already introduced the kernel and described your practices a couple of years ago. Then you would be able to introduce Scrum easily by replacing your existing practice for project management by Scrum without having to make any major modifications to your other practices. Looking into the future, Scrum will most likely be replaced by a new practice, and you will be able to do that easily, too.
What's in it for the academic and educational community?
It would be excellent if our technical institutes or universities would educate students in the basics of software engineering, followed up by training the students in a set of good practices using that base. Education will become more logical since it focuses on individual ideas instead of the particular soup of ideas that forms every method, process, or methodology. I believe students will love it. There is also space for a lot of relevant research here.
Remember Kurt Lewin's words: "There is nothing as practical as a good theory." A good theory makes it easy to learn and develop your knowledge without going overly religious. That would be smart.
Most university professors have made an academic career and have never really had a chance to practice software development on a large scale. Still they have to teach software engineering which of course is not easy or stimulating. They just do it because it is in the curriculum and not because they have something to contribute. They have no theory to teach, just a set of ideas or one specific approach. When challenged, one professor, who is successful in computer science and who teaches software engineering, said: "Amazingly, the students like to bathe in the mess we ship to them". I realize this was not serious, but for sure this teacher was not proud of what he was doing.
A theory will fundamentally change this situation. Students will learn the basics of software. They will get a language to communicate about software process, practices, patterns, etc. I can imagine that they will get a language with a grammar that represents the kernel and language constructs to describe the practices being constituents of the process. The language needs to be executable so that the practices can come alive. By this I mean that practices are not just specifications but also executables. When running a project, the practices will start to run and instances of activities, work products, roles with competencies will be created and populated by real things. Aspects seem to fit well to model practices. There are very interesting semantic rules that need to be identified and defined. There is whole new world that opens up to the students to understand the fundamentals of software engineering. Not to mention the whole new world for practically and theoretically interested researchers.
What's in it for the methodologists?
Thinking back on my own career from 1987, I was asked by many people to write a book on the methodology I created. At that time Objectory had a number of new ideas such as use cases, use case driven development which is a kind of test-driven design, collaborations, sequence diagrams, components and component based development. Most of the rest was not special. Implementation, unit testing, system test, performance test, configurations, planning was quite conventional. Of course, I had experience from the whole lifecycle, but I was not a world-class expert on everything. However, to write a book I had to cover the whole lifecycle, even if much of it was not my expertise.
With the new theory we are looking for, there will be no need to describe anything else than what is new. You won't need to write a book to publish your new ideas and put them in the context of all the other things a software development team needs to do. You will just describe your new practice or your new pattern and it will be available to the whole world the next day. Anybody in the whole world with great ideas can contribute and become successful.
What's in it for the teams developing the software?
Finally they will be able to escape from the endless churn caused by slavishly following fashions and become proper software engineering teams. Teams that build and extend their knowledge by practicing good software development based on a solid foundation, one that is not constantly changing and forcing you to learn the same things over and over again. One that lets you demonstrates your expertise by the results you've achieved rather than the courses you've attended. One that lets you bring in new ideas and new team mates easily and seamlessly without performance dips or wasted effort.
Finally teams will be able to continuously improve and adapt the way that they work to meet the challenges that they face on a day-to-day basis. They will be able to develop their knowledge and skills in a way that will enable them to work successfully with other people from different backgrounds, teams and organizations without having to continually relearn the same things over and over again.
Our understanding of software engineering lacks a basic theory. As a result, we keep reinventing old approaches with slightly different words, obscuring the real innovations and making it harder to use the good parts of the new while discarding the bad parts of the old. The theory will help us to substantially improve our education in software engineering. It will help us to be less nave in how we react to new ideas popping up around us. Finally, it will also help us to adopt new ideas faster than we can today.
The real beneficiary of this theory will be the software industry, as has already been proven in many companies. We will be able to easily educate our people, get them up to speed, improve the way we work with our products, reengineer (a stronger word than refactoring) our products systematically, and continually improve the way we work.
The result will be better software, significantly faster and at dramatically reduced costs. As mentioned above this is something we need to work on together. As seen from Scott Ambler's article The Theory Needs a Strategy, momentum is starting to build but there is still a lot to be done.
We have demonstrated that it works, but we still have much to do to establish an accepted standard. This has to be done by consensus building among a group of experts and authorities. We look forward to working with such experts.
To find out more about how you can get involved visit www.ivarblog.com and give your feedback to the blog "In need of a theory for software engineering" or e-mail us at email@example.com.
For More Information
- The Missing Theory of Refactoring, by Chuck Connell