The sea of information — make that the oceans of data — that we sail today are no longer the neutral repositories of data they once were. The data is perhaps the same, save for its volume, but it is no longer presented neutrally. Seemingly innocuous facts, even well-established data items, have gone from being stepping-stones to larger insights to controversial statements that thrust us into having to take a position in an unexpected political debate. For example, a belief in evolution can be as incendiary a position as arguing either side of the abortion debate. As a result, you, I, and most rational people do what is necessary to be able to function in our lives: We pare down the mountain of data and limit the inflow of extremist views by intentionally walling off certain data inlets. This is normal and, in many ways, an effective way to preserve sanity.
Alas, it's not possible to shut out all the wing-nuts. And as a result, we're forced to close the spigots on larger data pipes. As we do, we retreat further into a world that reflects the points of view to which we already subscribe. An unfortunate consequence is that we begin to look at opinionated sources of information as neutral and trustworthy if they agree with our beliefs. For this reason, in the last 10 years, the liberal news outlet MSNBC and the conservative Fox News both have obtained larger reach and greater revenue than the neutral CNN — ironically, the station that popularized 24-hour news.
And it's not just our intentional acts that produce this homogeneity of opinion around us. There are another set of forces at play that impede access to neutral information and intelligent opposing perspectives. Most advanced websites that we visit regularly track what we look at and, in response, they serve up content based on individual past access and reading patterns. So, if you're used to looking at conservative blogs, and if you try to find opposing points of view, you'll have to overcome a system that's tuned to feeding you conservative points of view. In other words, stepping out of the echo chamber you've created becomes increasingly difficult to do. The term for this state is "epistemic closure," where epistemic refers to how we know what we know.
Epistemic closure plays a role in programming as well. While most Dr. Dobb's readers are motivated to explore and so are less likely to be hunkered down in a world closed upon itself, there are pockets of our industry that are unto themselves and prefer to stay that way. Perhaps the most insular, sealed world is mainframe programming. There the machines, the OS, and the tools are provided primarily by a single vendor, IBM. Things are done as IBM suggests and the programming challenges, such as they are, exist within a narrowly circumscribed world in which the rest of the programming universe has little interest. Just this year, I was speaking with a mainframe programmer of 30 years' experience about his tool choices. I asked him what editor he used for programming. His first question was what do I mean by "editor"? When I explained, the unexpected query was replaced by a sudden glint of recognition! "Ah, you want to know what TSO version I use." And the proceeded to tell me. Such distance from basic terminology of the trade was profoundly disturbing and I could feel myself start shutting down. How would I ever ask him about how continuous delivery played out in a mainframe environment? There would be no way to ask.
Mainframe programmers inhabit a small universe that survives fine the way it's set up. But the model it uses — a single vendor dictating all the OS, the tools, and the approaches to programming — is a prime recipe for enforced epistemic closure. For this reason, many all-Microsoft shops today find themselves not using techniques and technologies of value. Consider for example, how long it took for Microsoft sites to adopt unit testing — long after it had been widely embraced elsewhere. The next barrier to fall might be the use of open source, which while widely accepted elsewhere is still rare at all-MS shops.
I choose mainframes and all-Microsoft sites as examples because they have the ideal profile for epistemic closure — the whole stack is dictated by one vendor. But all communities risk an inability to absorb valuable information from competing technologies and other communities. And, of course, most profoundly within ourselves there is ever the likelihood that because we subscribe to one approach to development that has worked for us, we become closed to other ways of doing things. From that point, decline is inevitable, not only in universal terms, but because technology is such a fast-moving area that stagnation relegates us to the sidelines or confines us to niches where we can indulge epistemic closure.
