There Must be Contest

Programming contests used to be fun. Now they're a commercial software development business—and they're still fun!


May 01, 2008
URL:http://www.drdobbs.com/web-development/there-must-be-contest/207404123

We improve ourselves by victories over ourselves. There must be contest, and we must win.

—Edward Gibbon

Have you ever competed in a programming contest? If not, you must have deliberately rejected the opportunity, because there is an amazing number and variety of competitions for programmers—more, it seems, than for just about any other (nonsports) professional.

If you haven't competed, you may not be aware that it's possible to make a six-figure income purely from contests—or that there are companies whose entire business model consists of running programming contests as the sole means of producing commercial software. There's a whole subculture of competitive software development within the industry, and it has many parallels with professional sports. (Except that nobody in competitive programming is talking about banning the use of performance-enhancing caffeinated beverages.)

This is not to suggest that programming contests are all just for pros: There are also contests specifically intended for college programmers and for high school students. The ACM International Collegiate Programming Contest is in its 32nd (!) year and just finished its World Finals in Banff, Alberta.

There are, in fact, all kinds of contests: regional and national and international contests, contests where you may compete in teams, contests where team programming is a requirement, and contests for solitary coders. There are 90-minute or five-hour dashes, 24-hour codeathons, month-long competitions, and continuous contests.

The rules and problems differ greatly, but the BME International 24-hour Programming Contest, put on by the Electrical Engineering Students' Hungarian Association and the Karoly Simonyi College for Advanced Studies, can serve as an example of one kind of contest. This event winnows out hundreds of participants from dozens of countries in an online challenge, reducing the field to 27 competitors (plus the three best from the previous year) who make their way to Budapest, where they spend 24 hours from 9:00am to 9:00am coding for glory and 5000 euros in prizes. There's even a book about the contest, including the problem sets from the first five years. The problem sets are definitely challenging: One year the teams had to design an integrated circuit chip design program and also create an AI agent that can play a simple strategy game against another AI agent. Other challenges involve controlling robots, managing virtual worlds, and cryptography. This year, the onsite competition took place May 2-4.

Often the choice of language is up to you, but not always. Pick a platform or a language and there's a contest for users of that platform or language: Java, C++, Ruby, Haskell, Mathematica. For a few years there was a competition for Eiffel programmers called the Eiffel Class Struggle, but I think that's over. You can compete to solve multicore or mobile problems, artificial life or fractal coding or robotics challenges, or participate in a Super Mario contest. I'm not sure what that last one is.

If you feel like brushing the cobwebs off your Apple II chops, you can easily find an Apple II competition. If you get bit by that "running light without overbyte" bug, there's a contest in which your entry must solve a set problem in no more than 1K of 6502 code. And 4K of Java. Dr. Dobb's has put on a variety of programming contests over the years: Currently, the Dr. Dobb's Challenge has a prize pool of $10,000.

Some contests are just not all that serious—like the International Obfuscated C Code Contest. The fact that the rules contain deliberate typos tells you a lot about that competition, which was jointly inspired by the Bulwer-Lytton fiction contest and some execrable examples of C code.

Then there's OMGWTF, the "Olympiad of Misguided Geeks at Worse Than Failure Programming Contest," where the goal is to solve a simple problem in the most obscenely convoluted way possible.

More seriously, there's the Loebner Prize competition. Win the ultimate challenge there and you'll get $100,000, a gold medal, and considerable fame—because the ultimate challenge-winning code merely has to pass the Turing Test. Programmers Heaven holds weekly contests. In short, there is no lack of opportunities to test or flaunt your programming skills.

TopCoder

Competition can be the most nerve-racking experience. Some people just thrive on it.

—Itzhak Perlman

So what's the experience of participating in these contests? "Exciting, exhilarating, heart pounding, brain crunching, mouth drying, do-not-want-to-miss-a-second-of-it-good," according to one veteran competitor, who participates regularly in TopCoder's competitions under the name AleaActaEst. "It can also be painful, dejecting, and anxiety ridden..."

Another thing that it can be is profitable. TopCoder competitor argolite made roughly half a million dollars competing last year, according to TopCoder's player stats, which, because the process is as much about evaluating individual programmers as it is about producing software, are remarkably extensive—arguably more detailed than baseball stats.

TopCoder is emblematic of this new contest-based approach to commercial software development. Back in 2001, the founders were running a contract software development operation that had grown to the point where they were managing a pool of over 600 programmers. They faced two challenges: To find a better way to evaluate programmers and to move in the direction of a true engineering process, based on a component architecture model. The solution they hit on for both problems was programming contests. "We think," says TopCoder's Chief Architect Dave Messinger, "that this is the best way to build software that is component based: component competitions." Competition, Messinger points out, is happening all the time among programmers. But TopCoder is one of only a few companies actually building commercial software through a competition model.

TopCoder winners.

And it's a real contest, or a steady succession of contests, with winners and losers. In TopCoder's competitions, only the first- and second-place finishers win money for their work, although there are several additional ways to win cash, including awards for reliability and for consistently good performance even if it doesn't win the big prizes.

Plus, "there are no penalties for signing up for a component and then not delivering it," AleaActaEst points out. "This way if life gets in the way we do not have to worry about broken promises." That doesn't often happen for him, one suspects. "The whole idea is that knowing that you are competing against some of the best makes you try harder, makes you try longer, makes you try smarter...There is nothing in my opinion better than a competition to make sure that you get it done."

Contests as a Career

The artillery department gave the specs for a new weapon. We had to improve on it or be eliminated from the contest.

—Mikhail Kalashnikov

TopCoder's first competition was a 90-minute algorithm test, measuring speed and accuracy. Today, they run several kinds of competitions—design contests, development contests in which every competitor writes the same code and the reviewers write test cases against that code, and assembly contests that produce full component-based applications.

TopCoder's contests generally take a week. A dedicated individual, participating part-time, can do one or two designs a week, AleaActaEst says, which could put up to $3200 in your pocket in a good month. TopCoder calibrates the awards in the contests to ensure, in argolite's words, "not only [that] someone will participate, but that the quality of the competition will be high." A competition may involve five or six competitors, meaning that you really do have to do your best to get one of the top spots, but it's not like winning the lottery: A talented programmer can count on racking up wins. Move up to full-time competing and the payoff ramps up. "Some of the top designers could make something like 40,000-80,000 per quarter. This is a combination of actual design winnings, reliability bonuses, and something known as Digital Run points, which pay per quarter for the top designers or developers who compete and deliver components."

TopCoder's Director of Communications Jim McKeown says that every programmer who participates gets something for the effort, even if it's only an education. This is because the entire process, including the reviewers' criteria, is highly transparent. AleaActaEst confirms that this is not just PR spin: "There is one amazing aspect of the TopCoder process that I really like: transparency and availability of winner's designs to peruse. The ability to look at the winner's design and learn from it is quite amazing. Imagine, especially for the newer technologies, that you have a full example of a working component...invaluable for anybody who wants to learn."

Win or lose, it seems like it's a win-win-win for the competitors, TopCoder, and TopCoder's clients. Argolite: "I would say it is akin to having an interview on the job, where if it does not pan out for an individual, there is no cost to TopCoder or their clients, since there will be someone that will succeed in providing a good product. Instead of wasting resources on advertising, interviewing, and then potetially seeing a resource not pan out, anyone is welcome to try to prove his/her worth."

Last year, TopCoder was granted a patent for their contest-based approach as a method of developing software.

Getting in the Game

The object of this competition is not to be mean to the losers but to find a winner.

—Simon Cowell

TopCoder is one model for using contests for serious business purposes, but there are others. Microsoft regularly screens potential employees through various contests and challenges. Its Imagine Cup winners get up to $33,000 and a trip to India to compete in the finals. Google's Code Jam has even higher awards, but in both of these contests the more likely payoff is that when you've finished and done a good job, you have significantly increased your chances of being hired by the sponsoring company.

So how do you learn more about programming contests? The websites for TopCoder (www.topcoder.com), Google Code Jam (www.google.com/codejam), Imagine Cup (imaginecup.com), the ACM International Collegiate Programming Contest (icpc.baylor.edu/icpc), the American Computer Science League for High School student competitors (www.acsl.org), and the International Olympiad in Informatics (www.ioinformatics.org) are good starting points. Also, the Algorithmist maintains a calendar of contests (www.algorithmist.com/index.php/Programming_Contest_Calendar); Ahmed Shamsul Arefin has written a book, Art of Programming Contest, to help students prepare for the ACM International Collegiate Programming Contest; and this year Andrew Trotman and Chris Handley published the paper "Programming Contest Strategy" (www.cs.otago.ac.nz/postgrads/andrew/2006-4.pdf).

One last point: Despite what I've said here about money and contests as a software-development model, the core of the experience is just plain competitive fun. Or in AleaActaEst's words: "This is beyond fun. This is uber-fantastic fun."

Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.