Jack J. Woehr is an independent consultant and programming team mentor. He can be contacted at http://www.softwoehr.com/softwoehr
Dan Sugalski has resigned as development lead for the Perl 6 Project.
From Dan Sugalski's Parrot Squawks blog http://www.sidhe.org/~dan/blog/archives/000400.html:
It's a funny old world....
Almost five years ago (five years next month), at TPC 4, Perl 6 got set in motion. We had a whole bunch of people volunteer to handle bits of the project -- I took on the development lead hat. (At this point I think, of the original volunteers, only Larry remains) A couple of months ago I gave up the hat, and now I'm giving up parrot development altogether. Chalk up another developer driven away.
I may spend some time writing up explanations of why things were done the way they were over the next few months, along with designs for some of the systems that never got (and likely never will get) implemented. I'll probably let the blog peter out to nothing after that. I'll leave it up since there are links into it, but if I feel the urge to blog I'll start up something fresh elsewhere, since it likely won't be particularly technical and it'll forestall the whining about me writing about things nobody cares about.
And no, my post-mortem on things won't be public, though I am going to put one together. It's always a good idea to look back on why you've bailed on a project.
Later, in response to queries on the perl6-internals list, Dan posted the following:
There was a lot of tension. It didn't go anywhere because I always insisted that the internals list stay professional and polite, and that applied to me as well. (And when I didn't, well... I've a collection of mail taking me to task about it).
No, things weren't particularly happy in parrot land. And no, you generally didn't see it. And no, it has nothing to do with Larry. And no, I'm not going to go into it here -- this isn't the place for it.
Apropos all this in general and particularly in a reponse to a query on the list, Larry Wall, the "Father of Perl", commented:
>What the heck is wrong with Parrot development?
I can't comment on aspects I am in all likelihood blind to, but I'll be glad to point out that one of the most irritating problems is people who snipe from the sidelines without having earned the right to do so by contributing. Dan has earned the right to carp, and deserves our respect for refusing to vent (much) in public, but anonymous cowards are generally just interested in making trouble without being held accountable. Feel free to prove otherwise.
We spoke with Dan by phone on June 6, 2005.
TPJ: Dan, can you give us a bit of a background? How did you get involved with Perl?
DS: I got serious with Perl in late 1997. I was working as a C programmer at Linn-Benton Community College in Oregon. We were a VMS shop. We had a spare DECStation and threw a webserver on it back when the Web was shiny and new and safe for children. I needed a guestbook -- I found one written in this language called Perl. So, I found the VMS port of Perl, built it, the guestbook worked. While building Perl, the compiler threw some warnings. We had the latest version of the DEC C compiler which at the time was the pickiest C compiler on the planet. I sent off patches, and that's where I started with Perl -- patching the core.
TPJ: "Look, someone touched the VMS port! We gotta talk to this guy!"
DS: The port itself was being managed by Charles Bailey, who is now a research pediatrician. l grabbed the latest beta and started patching. I hadn't written any actual Perl code at that point. It was just that I was on a VMS box, I like VMS, and dammit, this software should run on VMS! Then I started writing XS modules to interface to VMS system services. It was six months from the time I released my first module on CPAN before I actually wrote my first line of Perl code!
I got heavily involved with the first cut of threaded Perl, 5.005. I've been in the guts of Perl. I've done relatively little mundane Perl coding. Right now for work I'm writing a compiler in Perl. That I do. Web apps? I have to go grab the Mason book every time I do that.
At Perl Conference 4 in the summer of 2000 Perl 6 kicked off. The mug-throwing incident to fire off the project happened. Everybody who I thought was more qualified to head the development were busy, or burnt out. I was sitting there thinking, I wrangle the guys at work, we could have a committee, it's six months, it'll be fine. So I volunteered.
TPJ: What precisely were your responsibilities?
DS: At the time, I thought it was something like the "design team lead" for Perl 6, the Software, as opposed to Perl 6 the Language. I was going to lead the implementation team, Larry was going to do the language, someone else was handling PR. There were four or five project managers. Everything got divvied up, there was cheering and massive angst-ing. It was fun.
However, I think the paranoid rumors started before the meeting ended ...
TPJ: Which paranoid rumors?
DS: There were a bunch of them
- The world is coming to an end.
- The Dark Cabal is taking over Perl and secretly deciding its future.
- This is not Open Source.
- This is not democratic.
TPJ: When I was on the Forth Standards Committee, one commenter wrote to the president of the United States to complain about the process!
DS: After four and one-half years in this process I believe it.
At this point the plan was, over three or four months Larry would do the editing work on the language itself, then we'd start implementing it and life would be fine.
The redesign of Perl 6 turned out to be a much bigger job than we thought. I batted around a few ideas. I'd been deep in the guts of Perl 5 for years, including the rework that was necessary for getting threads and later Unicode in. We knew a lot of things we didn't want to do. We knew of things in Perl 5 that looked like good ideas but ultimately turned out to be bad ideas.
We knew things we needed, like objects, threads, Unicode. We didn't know what the syntax was going to be but we knew the semantics. Which is when we started writing Parrot. It was just called "the Perl 6 virtual machine". The name "Parrot" was because of Simon Cozen's 2001 April Fool's joke in which he claimed that Larry Wall and Guido van Rossum were going to get together and bury the hatchet and come up with a language that united Perl and Python. This came complete with a fake press release, fake interviews, the works ...
When OSCON came around that year, and Simon and I realized that, while it was a joke, what we were looking to do with Perl 6 software could actually do just that! Pretty much any semantic from any other language has been borg'ed into Perl. Any software that can run Perl can run Python and run Ruby and run C, because Larry has stolen every semantic in those languages. The only real difference is syntax.
So the next logical step was to steal the name from the joke!
TPJ: People have built virtual machines before, like IBM VM, COBOL, Pascal, Forth, OpenBoot, Java. Virtual machine writing is getting to be a routine yawn. What happened in this project? What has changed for you?
DS: It's been going on five years now. I thought we were going to get a bunch of the real clever folks handling Perl 5 development. I thought I was going to be a committee head coordinating a bunch of really smart people who already knew how to work together.
Everybody who I thought was going to be involved had an actual life and wasn't involved. It turned from being a committee chair position to an architect position. I wore that hat for four and one-half years before handing it over. I changed jobs a couple of times. I did the grant work for the Perl Foundation for six months. Travelled a bunch of places. Got a new job writing a compiler. I was getting really, really tired. It had been two months since I had done anything with Parrot. It was time to make it official, get out of the way. So we rounded up Chip Salzenberg who was willing to take on the architect hat.
That left me in a kibbitzing, user position. Ultimately, for reasons of personality, I decided it was time for me to bail. Reading the development list did nothing but send my blood pressure through the roof.
TPJ: Um, why?
DS: Leo Tötsch (the Perl 6 "pumpking", or release manager) and I have a personality difference. He's a nice guy, I met him in Paris a couple of years ago. Dealing with him drives me insane. I was fighting the same fights every six months. He'd bring up a topic, I'd say, "No", and six months later it would come up again.
TPJ: Like what?
DS: We've re-written the garbage collector three or four times. There are ideas about changing the basic architecture of Parrot that will get shot down, "No, this is not part of the design, let's implement something actually new, as opposed to re-painting the bike shed." Leo doesn't take "No" for an answer. He's very persistent, which can be a good quality. But given that I was supposed to be doing the architecture and he was supposed to be wrangling the source code, it got to be tiring.
Whenever I'd finally put my foot down, the next morning I'd have two or three messages in my inbox from people - usually not contributors - telling me what an utter bastard I was being, being mean to Leo when he was obviously right. It wasn't worth the hassle anymore. I wasn't designing Parrot for other people. I was designing Parrot because I promised Larry I'd do it, and because I wanted to do it. I wasn't under any obligation to anyone else, especially people who weren't doing anything.
One of us needed to go. Leo's done a lot for Parrot, a lot of very good work. We've both put a lot into the project. But I had backed out of development far enough that if I left, it would do less damage to the project than for Leo to leave.
TPJ: Is Perl 6 going to get finished? And is everyone going to be happy that Perl 6 got finished?
DS: I can answer the second question easily: Is everyone going be happy that Perl 6 got finished? Absolutely not! There will be people who will absolutely furious that Perl 6 was finished. No matter what you do, you will find someone who will be mad at you for it. You could end world hunger and someone will be mad at you for it.
As for "Will Perl 6 be finished?" my bet at this point is yes, definitely. On Thursday -- just don't ask me which one!
Autrijus Tang is a very sharp guy. He came in about four months ago. He started implementing from Larry's Apocalypses. He started implementing Perl 6 in Haskell as detailed in the Synopses, which is amazingly cool. He's gotten really far with it. It's not all there, and there are some speed issues. But it builds and it runs to the point where a lot of people who have been looking forward to Perl 6 can actually write Perl 6 code. It will compile your Perl 6 programs to Parrot code, but it will also run them on the Haskell engine.
Because the Parrot project has been running so long, we kind of gave up on Perl 6 ages ago.
TPJ: You stuck with just building the engine.
DS: Originally I was supposed to handle "the software", which includes the parser, language compilation, the back-end engine. I put the parsing aside, because there really wasn't any point in starting anything until Larry got the language design done. We could do the semantic part. I just put the syntax part aside. Then a couple of years ago I passed the syntax part on to Patrick Michaud. He took on getting the parsing and compilation parts done.
Larry's pretty much completed the language design now. My bet is Perl 6 will have a full alpha release between six and nine months.
TPJ: You said in your blog that you were going to write post-mortems about things that didn't get done. What didn't get done?
DS: A bunch of subsystems for Parrot either never got fully designed or never got fully implemented. Parrot was always slated to have a fully asynchronous I/O subsystem with layers, something like Sys V streams. I don't know if that's ever going to get finished. That was one of the pieces I posted a design for but which never got implemented.
I'm a VMS guy. I like asynchronous I/O, I'm well aware of how much power there is in asynchronous I/O, how much throughput you can get with it. Most of the folks involved with Parrot are Unix people, or worse yet, high-level language guys.
TPJ: They have no idea what asynchronous I/O is, they sit there and wait until something is done.
DS: Exactly. The only experience they have with asynchronous I/O is the Berkeley Socket system with polling and select and that's it. I've had more fun with root canals. So a bunch of people are, like, "We don't see the point." If it gets finished and gets out there, I think people will see how useful it is. You can layer a synchronous I/O system on top of an asynchronous I/O system, but you can't do it the other way around.
Also, Perl 6 was supposed to have a full event system. People like event systems so much there are like five different event systems for Perl. With Parrot, we knew what people were going to do because we saw what they had already done. So we were going to do it once and do it right so you didn't need five event systems.
Parrot was supposed to have a full notification system, modelled to some extent on Ruby. You could post watchers on things inside the engine to fire off notifications. A new global variable is put in the global variable store and if you've put a watcher on the store your code can do something. We were going to tie it in with the object system. People like to redefine classes at runtime, add a couple of instance variables, say. If you've already got instantiated objects you either have old objects that don't have all the slots, or you have to stop the world and add the slots to all the instances, or you have to have some really slow by-name lookup for the slots. One of the things people constantly come up with in Perl 5 is, "Can it get in the way of reads and writes on namespaces?"
TPJ: Kind of like the instrumentation they're putting into Java now.
DS: Something like that, yeah. People want to know when something happens internally. So we were going to have this notification system to allow you to add instance variables to instantiated objects. Since we were very careful about where you can allocate variables, on notification you could just walk through the variable pools looking for objects of the appropriate type and fix them. A fair amount of grunt work. It could take some time. A million instantiated objects and it could take a second or two. But what we were always shooting for was that normal things run as fast as possible, and if you wanted to do something horribly slow, you had asked for it and we were going to make sure you could do it and take the performance it. But that never got fully specified and it's not going to get in.
I don't think the scheme for multiple encodings and characters sets is ever going to go anywhere.
We always planned threads for Parrot since the beginning. I'm very fond of threads. But I'm not sure that Parrot's threading is ever going to get as sophisticated as I'd hoped it was going to get. It's another one of those things that never got fully specified for lack of time. It's going to be ad-hoc, clumsy, and have limitations that people aren't expecting.
So these were things that were off in the future, needed to get specified, needed to get implemented when were done with what we were working on, but never got done.
TPJ: There's no perfection in this world. To some extent, people's mystical fervor for programming languages kind of went out with the economy around the turn of the new millenium.
DS: The mystical fervor hasn't gone away. It has just shifted onto other things.