Al is DDJ's senior contributing editor. He can be contacted at email@example.com.
I have so much to write about this month. Changing to a bimonthly schedule means I get more work done between columnsone month's worth to be exactand, therefore, I have one additional month's worth of things to say. Who's idea was this anyway?
I'll start by telling you about a program I found that I cannot live without. Most musicians cannot live without this program eitherexcept most don't know it yet. The program is called Transcribe! (http://www.seventhstring.demon.co.uk/xscribe/), and the author is a self-employed musician, engineer, and programmer in London by the name of Andy Robinson. He wrote Transcribe! in Visual C++ with MFC. There is a version for the Mac, too. Transcribe! is shareware and the registration fee is $40.
Transcribe! is a digital signal processing (DSP) program that analyzes and plays back audio files. Its stated purpose is to support musicians who transcribe music from recordings. It has two features that will endear it to any piano player. This might be helpful for players of other instruments, too, but those players don't need to know about what God was playing. I'll explain in a minute.
Transcribe! slows down a passage and plays the passage in a loop. It does so as slowly as you like without changing the pitch and retaining enough fidelity that you can distinctly hear the notes. Any pianist who has ever tried to figure out an Art Tatum improvised arpeggio already knows how valuable this feature is. Those who claim to know all about DSP have told me that you can't do this. Transcribe! is cool because it does what I have needed for years and it does what some people say can't be done. Andy explains how he did it at http://www.seventhstring.demon.co.uk/xscribe/slowdown.html.
Transcribe! analyzes a marked passage and tells you what frequencies are in that passage, their duration, and their amplitude. What's more, it displays this information above a piano keyboard display. Any pianist who has ever tried to figure out a chord voicing that Art Tatum (or Bill Evans or Oscar Peterson...) played in some musical context can tell you the value of this feature. The feature also displays harmonics, of course, but you can tell which notes are harmonics by experimenting with the notes on a piano and comparing them to the recording.
Now, if it could only tell us the fingering...
Transcribe! enables something that pianists have been trying to do since the 1930s: Figure out what that magnificent, unbelievable, visually impaired piano player named Art Tatumyou know, the fellow whom other piano players call, simply, "God"was doing.
Next, I'll tell you about another program that knocks my pedal covers off, not so much because of what it does, but because of how it came about. In October, I wrote about using pipes in a GUI application to emulate the human user in a console application. I explained that I learned the technique by reading the Pascal code of another IDE program named Dev-C++. (I have since found an article that explains the technique in detail; see http://support.microsoft .com/default.aspx?scid=KB;EN-US;q190351.)
Someone told me about Visual-MinGW (http://visual-mingw.sourceforge.net/), another IDE that hosts the MinGW port of GCC. Visual-MinGW uses pipes to communicate with the compiler programs and a grep utility. It does not yet have debugger support in the IDE. I was interested in how the author built his program, a Win32 MDI application with splitter windows similar to Visual Studio's UI. I long ago concluded that you can't do that with MFC. The parts just don't fit together, no matter how I try. I downloaded and read the Visual-MinGW C++ code. The author built his own application framework class library that wraps the Win32 API and implements the split window interface. The class library resembles MFC in many ways, except that it is small, efficient, and effective. I contacted the author expecting to meet a seasoned Win32 programmer.
Visual-MinGW's author identifies himself as Manu, declining to be further identified. He is 31-years old and lives in Nantes, in the west of France. He works as an "electronician" in Le Mans, specializing in cordless numeric phones. When I asked about his education, this remarkable fellow said, "I'm mainly self taught in electronics." When I asked what prompted him to develop Visual-MinGW and its application framework, he said, "My goal was to learn programming..." Did you get that? Not to learn Win32 programming or C++ programming, but to "learn programming."
No wonder he wants to remain anonymous. The kids at Redmond might take out a contract on him.
Building a Better ConsoleApp
Also in October, I described a Win32 C++ class to implement running console applications from within a GUI application. The purpose is to let the Quincy 2002 GUI IDE run and interact with the command-line GDB debugger program that supports the MinGW compiler. I developed a simple ConsoleApp class that uses pipes as substitutes for the command-line program's standard input/output devices. The IDE writes debugger commands into the send end of GDB's standard input pipe and reads debugger output from the receive end of GDB's standard output pipe. The previous column was all about getting the pipes to work properly and building a stub GUI application to test the ConsoleApp class.
Quincy has two other command-line programs to run. One is grep, which searches source code for text matches. The other is the compiler program itself. Neither program is interactive, but both need to send standard output to the parent GUI application. Previously, Quincy managed this by building text files and passing their handles to the command-line applications to use as standard output. Then Quincy would read the text files and process the output messages from the console applications. With a properly operating ConsoleApp class that uses pipes, I decided to port these other two operations to the class.
That's right, even though I had something that worked, something that did not need fixing, I decided to fix it anyway. It seemed a good way to wring out the ConsoleApp class before I gave it the more complex job of running an interactive debugger. And fixing it appealed to me aesthetically; t'would be a better mousetrap because it would not depend on finding a place to keep those redirected temporary files.
To follow along with this narrative, you'll need the source code. More recent versions of consoleapp.h and consoleapp.cpp are available electronically; see "Resource Center," page 5. The console application interface worked much as I had designed it for grep. But the compiler interface did not. Running the compiler involves running gcc once to compile each source code file, perhaps running windres to compile resource scripts, and running gcc a final time to link the object files into an executable or to build a relocatable library module or dllwrap to build a DLL.
I use the generic term "compiler" to represent all the steps involved in building a binary. I know that, technically speaking, the compiler is only one component, but it's easier to call it that. We don't have a better word for the process anyway.
The ConsoleApp class got flaky when it was called upon to run a sequence of programs. The flakiness had to do with the standard output pipe and with timing. Frequently, the process would lose standard output messages, and it was always too slow. Compiling a program took much longer from the IDE than it did from the command line. That would never do.
The compiler process in Quincy queues the programs to be run along with their command lines and launches the programs one at a time, always watching for messages from standard output to be displayed in the gcc output window. At first, the program monitored the pipe from within the CWinapp::OnIdle loop. This was a mistake because the program spent too much time in the OnIdle loop, stealing valuable cycles that the compiler needed.
Consequently, I redesigned the ConsoleApp class to accept two more constructor parameters. The first parameter is a pointer to a callback function to be called when the console application has sent a line of text to its stdout. The callback function can then ConsoleApp::ReadConsole to get the line of text. The second new parameter is a callback function that notifies users of ConsoleApp that the console application has terminated. Both of these new parameters permit the using application to know when to do something without having to poll ConsoleApp from an idle loop.
The GDB Debacle
Quincy 2002 (http://www.alstevens.com/quincy.html) is an IDE that uses the ConsoleApp to integrate GDB, a command-line, source-level debugger, into its environment. In October, this was merely a hope. Now it is a reality. Quincy launches GDB and sends commands to it through a pipe. Quincy reads the piped responses from GDB and translates those responses into visual things that it does with the display of the debugged program's source code files. The interactive dialog happens under the covers, so to speak. To the user, Quincy is the debugger. It works great.
It did, that is, until I turned it over to Judy to test my suite of tutorial C++ programs for the new edition of Teach Yourself C++, authored by yours truly and coming to a major book store near you soonI hope. Judy is my tester, and she finds all my bugs. Just ask her. I copied the whole distribution, Quincy 2002, the compiler and debugger, and all the source files to Judy's computer. Then I returned to my office to dwell upon more meaningful and scholarly tasks than the mundane tedium of ensuring quality, something in which I had the greatest confidence, since I had only just completed a thorough test of everything myself. I wasn't even into the second hand of Freecell when the intercom buzzed.
"It isn't working," came the call. Figuring that Judy must have done something wrong, I gave forth a heavy sigh, pulled myself out of my chair, and trudged across the lawn to Judy's office.
"Isn't this thing supposed to say 'hello, world' or something?" she asked, pointing to the console window of the very first program in the tutorial suite.
Sure enough, the window was blank. I returned to my computer, a Windows XP Pro box, and reran the test. It worked. I moved everything to another XP computer. It worked. I moved it to a Windows 98 computer. It failed. Judy's computer, where it originally failed, runs Windows ME. A pattern seemed to be emerging.
Judy turned off her computer saying, "Let me know when you have something a little more reliable for me to test. Until then, I'll be busy watching Carol Duvall and doing some crafting."
Many tests later, I concluded that Quincy's GDB integration works on operating systems derived from NT and not on operating systems derived from 90 anything.
Quincy launches GDB by using CreateConsole, telling GDB to use pipe handles for stdin, stdout, and stderr. You can see that behavior by reading consoleapp.cpp. GDB launches the target application by using CreateConsole and the Win32 debugging API. I wasn't sure what GDB did other than that. This much was sure. When a GUI front end launches GDB, which launches a console application to debug, the console interface does not work on the 95/98/ME platforms and does work on the XP/2000/NT platforms. I concluded that the two platforms have inconsistent implementations of the CreateProcess function when the debugging API is used.
My conclusion was reinforced when I tested the VIDE program (http://www .objectcentral.com/vide.htm), another MinGW-hosting IDE. Sure enough, when debugging a console application, VIDE works okay on XP and fails to display the target's stdout in the console window on 9x. I notified Bruce Wampler, VIDE's author, and he replied that he does not use VIDE to develop console applications and was unaware of the problem.
I took the problem to the mailing list associated with MinGW. Next came several days of folks commenting, asking questions, making suggestions, and sending files. I must here and now acknowledge the unselfish and invaluable help given me by Greg Chicares, Earnie Boyd, Christopher Faylor, Norman Vine, Oscar Fuentes, Manu, and Marc Kealy. Whoever and wherever these folks are, I owe them big time. Having made a commitment to have this program running when the book is ready for publication, I was now looking at an insurmountable problem. It would not be acceptable to tell readers that they had to get XP to run the accompanying software. I don't look good in tar and feathers.
Someone suggested that I try the latest version of GDB. The version distributed with MinGW, 5.1.1, is an older one and doesn't do Standard C++ all that well. Porting GDB to MinGW is not an easy task, they said, and no one has found time to do it. But maybe, just maybe, the newer GDB works where the older one does not.
The cygwin project (http://www.cygwin .com/) includes a Win32 port of GDB 5.2.1. I downloaded cygwin and ran gdb. It starts up by default in its Insight mode, which is the project's GUI front end to the debugger. I did not try to see how Insight does it, because Insight does not launch GDB from a command-line executable. Instead, Insight is added to GDB with hooks to its GUI. You have to launch GDB with the -nw option ("no windows") to suppress the Insight GUI. The problem persisted with the new GDB. Oscar Fuentes sent me a copy of gdb.exe without the Insight code and he told me how to get the source code and build it myself. Still, debugged applications displayed no text when run on 98 and its fellows.
I peered into GDB's source code and made lots of changes in the way it launches the target. Nothing worked. Then Marc Kealy made an interesting suggestion. Before calling CreateProcess, GDB ought to set its own stdin/stdout/stderr handles to INVALID_HANDLE_VALUE. Then, when the target program is running, GDB ought to restore those handles to what they were before. What they were before were handles to Quincy's pipes. GDB was not telling the target to inherit its handles. And even if it was, the target's standard output was not coming through the pipe, nor would its standard input read the pipe. Everything told me that Marc's suggestion would not work, but I was desperate, so I tried it. Guess what? It worked.
Here is what I have concluded: The Win32 debugger API under 9x and ME has a bug, probably there for years. It causes a target console application to kind of inherit the debugging program's stdin/stdout/stderr handles even when you tell it not to. It is necessary to modify gdb to make it work in an environment such as Quincy's. And so I did.
And now I am ready to pry Judy away from her crafting studio and plunk her down at her computer again. We'll see what happens when she gets to exercise program number two.