I've liked figuring out how things worked and designing them from a very early age (I loved the Tom Swift books). In an earlier age, I could see myself designing locomotives or Schneider Trophy racers. My degree was in mechanical engineering. But mechanical engineering was frustrating because of the large expense involved with building anything, and my rather poor fabrication skills.
With programming, on the other hand, I could build the most intricate machines imaginable at no expense, needing only access to a computer. Like many programmers, I started with writing games. The appeal of writing games is sort of a God complex you get to create a whole world that slavishly follows your rules. With the game Empire, not only did I create such a world, but I spent endless hours figuring out how to make the computer best operate the enemy armies. Even now, I still think of ways to improve it.
The trouble, though, was that the computer strategy's thirst for compute power far exceeded the abilities of the machines available, and so I became interested in how the compiler optimized the code. Compilers seemed like utterly magical devices that transformed source into machine code. How did this sorcery actually work? This was one of the Great Mysteries Of The Universe until BYTE magazine published the source code to Tiny Pascal. I devoured every line of that program; and once mastered, I felt I'd been given the password to the inner sanctum.
Some years went by. In the early '80s, I found myself part of a programming team developing software for MS-DOS. We were using C because it was the only high-level language we could find that actually worked on the PC. The other language implementations were unbelievably awful. Even the C compilers were execrable, but at least they were usable. The code they generated was terrible, the optimization nonexistent. I had the idea I could write a better C compiler myself.
I confided this to a colleague, and he suggested a lunch with him and the local C guru, who'd give me some advice on how to proceed. We went and my friend explained my ambition to the guru. His contemptuous response is still etched on my brain, "Who the hell do you think you are thinking you can write a C compiler?"
I have to retroactively thank him for that because I found the desire to show him up to be powerfully motivating. I went on to implement the C compiler, known as Datalight C. True to my interest in optimization, it was the first on the PC to have a data-flow optimizing compiler. Such a concept was new enough that the compiler got into trouble in the computer magazine benchmarks because the optimizer figured out that the benchmarks did nothing and so deleted all that dead code the journalist assumed my compiler was broken or cheating and Datalight C got a bad review. (This, of course, made me pretty mad, but there was no ubiquitous Internet in those days where I could post a riposte.)
It was later reincarnated as Zortech C. By then, there were many other C compilers on the PC (I think I counted 30 at one point). I was looking around for a competitive edge, and found Bjarne Stroustrup's book on C++ in the bookstore. I thought, "Heh, add a couple of new keywords, and in a couple of months, and I'll have a C++ compiler!" Probably the understatement of the programming century. If I'd known what I was getting into, perhaps I would have believed the people who told me I couldn't do it.
Anyhow, the late 1980s was a time when lots of people were working on successors to C.
You can even find references to one such project called "D" on usenet of the time. For various reasons, including the fact that Zortech had an inexpensive C++ compiler ready to go on the most popular platform of the day, C++ buried those other languages and dominated the 1990s of programming; and I was carried along with that boom.
For example, I was out jogging one day with a programmer friend who said, "You know, what the world is desperate for is a Java compiler that generates native code. You'll make a mint off of that! I use Java and this is really needed." I told him that, coincidentally, I had written one and he could start using it right away. Of course, he never did.
Nobody could possibly work on compilers for 15+ years and not come up with ideas for language improvements. I tried out many language improvements in the C and C++ compilers, and they all fell flat. Nobody was interested in language extensions they wanted standard compliant languages, and they were right. Unfortunately, getting ideas adopted by standards committees is an arduous, multiyear bureaucratic process a process I am not very patient with.
In 1999, I decided I was retired and spent about six weeks watching TV until I thought I'd go mad.
It was time to get back to living. Whining about perceived problems with existing languages had gone on long enough; I decided to power up the machine shop. When tackling a problem like this, I am always reminded of Gimli the dwarf: "Certainty of death. Small chance of success. What are we waiting for?" Why not? At least I'll go down sword in hand fighting the glorious fight.
So D started out, stumbling along as a solo project. I've never been comfortable programming in a language I hadn't written a compiler for (I know, another oddity), but I'd had enough experience implementing various languages that I was pretty confident I could get D working. I also by now had the huge advantage of the existing C++ compiler ecosystem to work with.
A couple years later, D first appeared on Slashdot and it rapidly started attracting users and collaborators. Turns out, I am hardly a unique person in what I want from a language! D grew dramatically in ambition, with collaborators from all over the world. It wasn't until a few months ago at Dconf2013 that we even knew what each other looked like. (This is one of the greatest aspects of the Internet revolution: You can work successfully with others while knowing nothing about their sex, age, looks, race, religion, language, culture, disabilities, histories, etc. It's as pure a meritocracy as it gets. Only your ideas, contributions, and how you present yourself matter.)
As it stands, D wouldn't exist without the Internet. How else could disparate enthusiasts have ever gotten together? The rise of collaborative tools like github and bugzilla have been absolutely critical to D's development (along with the open source model, of course!).
But I've had to learn things, too, often the hard way:
- My natural tendency is to work solo on things. My work performance reviews at the various jobs I've had usually included comments like "Walter needs to learn to work better with others." My desire to control everything nearly wrecked the D community at one point, and I've had to change.
- I've had to learn how to manage a project where people are all volunteers. Since I don't pay anybody anything, I can't tell anybody to do anything. I have to find other ways. I rank pretty close to "nerd" on personality tests,so motivating people is not natural for me. This is actually a fascinating challenge.
- In the D forums, proposals for new language features come every day. I have to say "no" a lot, which is hard to do, and harder for the proposers to hear.
- Management is a hard job, and I'm not well-suited for it. I didn't do a very good job of it at Zortech; and at various corporate jobs, I was (wisely) never promoted into management. But with D, it couldn't be avoided I have had to learn how to do it, and do it reasonably well. I'm pretty motivated to try and be a better manager, because I want D to succeed.
- As my friends know, I love to debate. I'm a participant in many Internet flame wars. But I can't do that with D because nobody ever changes their mind as a result of a flame war, no matter how wrong they are :-). All this experience actually lends a curious advantage I can tell when a conversation thread is starting to slip into unproductive argument, and take steps to lower the voltage and defuse it.
I still have to fight the urge to go "Shields up! Phasers on vaporize!" Of course, as anyone on the D forums will agree, this is a work in progress for me. D has been and continues to be an amazing experience it's borrowed the best ideas from many other languages, and is the summation of enormous and selfless effort by many, many collaborators from all over the world, who can all be justly proud of what they've accomplished.
There has been one result of all this that I especially treasure I love programming in D. It's the language I wish I'd always had available. I'm glad I didn't listen to the naysayers, the Debbie Downers, and of course that nameless C guru from long ago, whom I owe thanks to.
So, you want to write your own language? All I can say is: Certainty of death. Small chance of success. What are you waiting for?
Thanks to Andrei Alexandrescu for his helpful suggestions on this article.
Walter Bright is the designer of the D language. As mentioned here, he previously wrote the Datalight and Zortech C++ compilers, as well as several games. He regularly blogs for Dr. Dobb's.