How to Use Tcl and C Together
Cameron Laird
Mixing Tcl and C/C++ just got a whole lot easier.
“How do I call a C program from Tcl?” is one of the most frequent questions Tcl newcomers ask. As simple as the question seems, it has dozens of useful answers.
Understand, that’s not a defect in Tcl. There are situations where a choice between several alternatives hints that none of them is really complete or polished. That’s not the case for marriages between Tcl and C, though; in fact, a principle design goal for Tcl was for it to make good “glue” for C functions. To meet that goal, the Tcl core technology reuses a few simple principles to expose several different approaches to uniting the two languages. Each one is best for at least some applications. Note, by the way, that this article consistently uses C as an abbreviation for C/C++. All the technologies profiled here apply equally to C or C++.
An Easy Warm-Up
Let’s start with the simplest combination. You’re likely to run into Tcl in all kinds of architectures: within web servers, scripting IRC (Internet Relay Chat) sessions, embedded in proprietary hardware, and so on. The easiest way to begin your Tcl career, though, is with a standard distribution interpreter. The latest information on standard Tcl appears at the Tcl Developer Xchange, available at <http://tcl-tk.net/> or <www.tcl.tk/>.
After a quick download and installation to your Macintosh, Windows, or Unix desktop, you launch a Tcl interpreter or “shell.” It prompts you with a “%”. You begin:
% info tclversion 8.3 % puts "Hello, world." Hello, world.
The lines that begin with “%” here are ones you type. Think of the others as Tcl’s replies to you. Read more about how to begin to use Tcl at <http://wiki.tcl.tk/298>.
Typical Tcl development is highly interactive. In contrast to C’s familiar edit-compile-link-execute cycle, Tcl calculations feel immediate.
In particular, Tcl only needs a single line to retrieve results from a C-coded application. Here’s an example:
% set result [exec ping -c 3 tcl-tk.net] PING tcl-tk.net (212.73.209.252): 56 data bytes 64 bytes from 212.73.209.252: icmp_seq=0 ttl=244 time=107.6 ms 64 bytes from 212.73.209.252: icmp_seq=1 ttl=244 time=107.3 ms 64 bytes from 212.73.209.252: icmp_seq=2 ttl=244 time=107.6 ms —- tcl-tk.net ping statistics —- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 107.3/107.5/107.6 ms
That’s a typical result when using Linux or a similar Unix host. If you’re on Windows, drop the -c 3 arguments.
There! You’ve just invoked a C-coded function — the main that implements ping — from Tcl.
What’s the Point?
This example is a better one than might first appear. Many Tcl beginners see how easily Tcl manages external processes and conclude that exec and related conveniences aren’t for serious programming. That’s a mistake. One of Tcl’s best uses is to “wrap” existing programs. Many programs in the world do important work, but clumsily. Preserve them for the good they do by wrapping them inside a friendlier, more flexible Tcl interface. By keeping the legacy C-coded programs unchanged, you eliminate the need to configure and test them for a new deployment. You can enhance and customize them purely through the wrapping code. Because Tcl is a succinct, high-level language, this will almost always be more productive than modifications to the legacy programs.
Dataplot is an example of this approach. Dataplot (<http://wiki.tcl.tk/dataplot>) is a Fortran-coded statistical package originally written at the National Bureau of Standards (now known as the National Institute of Standards and Technology) in the 1970s. Its command-line version is still in use on several platforms, essentially none of which existed when Dataplot was first coded. For modern users, a GUI written in Tcl wraps the command-line Dataplot.
If you’re working with C and Tcl, your first instinct should be to look for the simplest feasible partnership between them. In many cases, construction of a stand-alone C executable, managed by Tcl as an external process, will meet all your needs. Tcl’s exec recognizes flags for passing command-line variables and input and also can receive output (both stdout and stderr) as well as the managed process’s “exit value.” exec’s flexibility and polish are quite remarkable. Consider, for comparison, Tcl’s peer languages, Python and Perl. While both also have mechanisms for controlling external processes, they’re quite a bit “clunkier”: more obscure, and less succinct and portable. Documentation at <http://wiki.tcl.tk/exec> gives examples.
exec’s process control is the simplest possible: Tcl starts an external process and then “blocks” while the external process is active. Only when the other process is completely done does Tcl regain control and restart its own calculations, based on the result of the external process. exec also recognizes an option for launching long-running “service” or “daemon” processes “in the background.” In this case, though, there’s no direct way to receive data back.
open is a Tcl command that offers a bit more sophistication. open makes it possible to launch an external process and then resume Tcl operations. Tcl and the external process act concurrently.
Process-Level Concurrency
Several pieces need to be in place to demonstrate open’s power. Perhaps the best way is to build a simple GUI that simultaneously displays results from an external process, while remaining responsive to such user actions as keystrokes. An example is this:
package require Tk entry .e text .t pack .e .t proc receive {} { if [eof $::fp] { set line DONE close $::fp } else { gets $::fp line } .t insert end $line\n } set fp [open "|ping -n 5 tcl-tk.net"] fileevent $fp readable receive
If you run this script under Windows, you’ll see a GUI screen that looks like Figure 1. For Unix, change the -n 5 to -c 5 (but see sidebar on portability). As you execute this small program, notice that the GUI stays “live;” you can type in the entry at the top at the same time as ping is gathering its results.
exec and open are the most important built-in facilities for teaming two different processes, one coded in Tcl, and the other in C. They’re far from the only ones, though. Under MacOS, Tcl speaks Applescript. On Windows, Tcl knows DDE (Dynamic Data Exchange), and popular extensions provide COM (Component Object Model) capabilities. Under all operating systems, Tcl has a lovely socket-oriented network-programming interface, one that makes it easy to build network servers and clients. Each of these technologies, along with the others mentioned in <http://wiki.tcl.tk/1228>, has a place in constructing architectures for teamwork between Tcl- and C-coded processes.
Bare-Fisted Coding
None of this, though, gets at what most Tcl newcomers think they want in regard to combining C and Tcl: in-process means to pass program execution back and forth between modules coded in the two languages. The rest of this article surveys these possibilities.
Even restricting focus to same-process architectures leaves several distinct engineering designs. Early in Tcl’s history, it was common to “extend” the standard Tcl processor. That means that developers would build a new Tcl interpreter, based on the standard one, but with a few extra C functions exposed as Tcl commands. This is essentially the difference between Tcl and Tk, as most users see them: the latter is just a bigger executable, with more available commands.
For the past few years, though, it’s been most common to use an unmodified, “stock” Tcl interpreter. New C functions are packaged into loadable objects, which the standard interpreter loads dynamically, at run time.
Tcl documentation is sufficiently lucid that many programmers learn how to interface C functions just by reading material in the standard distribution. Most, though, have studied one of two books that detail how to build new commands: Tcl and the Tk Toolkit (by John K. Ousterhout, Addison-Wesley Professional Computing, 1994), or Practical Programming in Tcl and Tk (by Brent B. Welch, Prentice Hall, 1999). Find out more about these and other Tcl-related books at <http://wiki.tcl.tk/book>. The two chapters in Practical Programming in Tcl and Tk most important for using C functions are available at <www.beedub.com/book/3rd/bookTOC.html>.
An Example C-Coded Command
In this section, I’ll show you how to integrate a C function into Tcl as a loadable object. This example assumes gcc or a similar Unix-oriented compiler. Use these commands:
cc -c example.c -o example.o ld -shared -expect_unresolved \ -o mycommand.so example.o
to compile this source file, example.c:
#include <tcl.h> int MyCommand(ClientData clientData, Tcl_Interp *interp, int argc, char *argv[]) { char *datum, temp; datum = ""; if (argc >= 2) { datum = argv[1]; if (strlen(datum) > 3) { temp = datum[0]; datum[0] = datum[1]; datum[1] = temp; } } Tcl_SetResult(interp, datum, NULL); return TCL_OK; } int Mycommand_Init(Tcl_Interp *interp) { if (Tcl_InitStubs(interp, "8.1", 0) == NULL) { return TCL_ERROR; } Tcl_CreateCommand(interp, "mycommand", MyCommand, (ClientData) NULL, (Tcl_CmdDeleteProc *) NULL); return TCL_OK; }
Once you’ve created mycommand.so, you can exercise it this way:
% mycommand abcd invalid command name "mycommand" % load mycommand.so % mycommand abcd bacd % mycommand abc abc % mycommand
The first invocation of mycommand abcd demonstrates that mycommand is not known to the standard Tcl interpreter. After loading in the mycommand.so module, though, we can use mycommand as freely as any other Tcl command. (The mycommand function returns a string based on its input, but with the first two characters transposed, if the string exceeds a minimum length.)
This recipe requires only minor tweaks for proper configuration. The C code is slightly simpler if you’re working with a Tcl version before 8.0; on the other hand, the advantage of version 8.1 and later is that a loadable module made with any newer release works properly with any other recent Tcl release. This is the benefit of Tcl’s “Stubs” (<http://wiki.tcl.tk/stubs>) technology: when you make an extension with Tcl v8.4, for example, you can safely pass it on to a friend or customer who’s replying on 8.2. Also, be aware you might have to give your compilation a “hint” such as cc -I/usr/local/include.
As exciting as it is to be able to alloy two different languages with so little trouble, there are three reasons why commercial-grade implementations are likely to look different from this example.
But I Want My C Function!
Tcl has a “package” structure for managing external modules. A refined C-coded command generally includes a bit more information to help it fit in the package framework.
Also, note that the example above didn’t really make your function available as a Tcl command; what it did was to register a function with a specific argument signature, starting with:
ClientData clientData
In any practical example, you’ll certainly need to wrap up the C function you’ve already written inside another one, which translates to the Tcl calling conventions:
int MyCommand(ClientData clientData, Tcl_Interp *interp, int argc, char *argv[]) { ... my_real_function(...) ... }
Other programmers have passed this way before, though, and have created several specialized tools for automating Tcl-C combinations.
The example code above is a good basis for adding one or two new commands to Tcl. If you’re working with a broad API, though, like a whole new Internet protocol, or the command set to control new hardware, you’ll want to learn about SWIG (Simplified Wrapper and Interface Generator), <www.swig.com>. SWIG automates creation of the “glue” code between Tcl and a collection of C functions. SWIG is actively supported and widely used, not just between Tcl and C, but for many other combinations of languages, including C++, Perl, Python, and so on.
Many Variations
That’s not all, though. Along with writing Tcl-C glue yourself, or using SWIG, there are a handful of other specialized approaches in productive use at many sites.
One particularly apt for C/C++ Users Journal readers is ffidl (Foreign Function Interface with Dynamic Loading), <http://wiki.tcl.tk/ffidl>. This is an extension to Tcl that makes it possible to call C functions directly, without any auxiliary code. Although ffidl’s author, Roger Critchlow, labels it “experimental,” it’s already the basis for several applications doing good service for end users.
This article has presented all its examples as cases of a Tcl program controlling a C program or function. Tcl was designed from the beginning to be “invertible,” though. D. Richard Hipp is a consultant whose work emphasizes the “dual” approach of C programs that manage small Tcl scripts. mktclapp (<http://wiki.tcl.tk/mktclapp>) is his tool for simplifying “the task of constructing a stand-alone program that uses both C and Tcl/Tk.”
These are ways to bind together Tcl and C that are most apt to appeal immediately to readers coming from a C background. Database specialist Jean-Claude Wippler’s latest of many contributions to Tcl programming, CriTcl (<http://wiki.tcl.tk/critcl>), promises to revolutionize the way Tcl and C are used together.
Cheap Idea, Deep Consequences
CriTcl is reminiscent of the Inline module Brian Ingerson pioneered for Perl, and Wippler credits Inline as a partial inspiration. With CriTcl, a programmer only writes Tcl source. If he or she needs C code, that, too, is part of the Tcl source. Here’s a demonstration:
package require critcl namespace import critcl::* cproc cube {int x} int { return x * x * x; } proc report number { puts "The cube of $number is [cube $number]." } report 4 report 13
This program reports the cubes of 4 and 13, after computing them directly through C-coded calculations.
The superficial appeal of CriTcl, then, is that programs can be managed in a more unified way. A conventional application built with both Tcl and C involves at least Tcl sources, C sources, and compilation scripts certain to vary from one operating system to another. With CriTcl, there are only Tcl sources.
In all this, CriTcl is like Inline. Wippler explains, though, “The difference is that Tcl’s ‘stub’ architecture makes it much simpler to combine the results — the resulting shared library does not care where the Tcl runtime is or what version it is, yet this design supports callbacks between Tcl and C in both directions, and provides full access to the Tcl C API. This simplicity comes back in the form of generality, portability, and robustness. As an example of the latter, CriTcl does not need to figure out how to link to a runtime, meaning that there’s one less source of potential problems/variability.”
“Stub” here is one of Wippler’s earlier co-inventions. It was Tcl_InitStubs in the example code above that lets mycommand work across versions.
Developers with large-project experience appreciate how important the release-neutrality afforded by CriTcl is. That’s just “plumbing,” though, as beneficial as it is. Wippler anticipates bigger consequences, as Tcl sites accept CriTcl. As he told me, “There are many implications of having a system like CriTcl in place. First of all, one can now surgically replace hot spots in a large body of Tcl code to get speed improvements. Second, there is no longer a need to set up a build system: source code is recompiled when it changes, and libs are cached when it does not. The complexity of managing software snippets goes down, because small dabs of C ‘grease’ can be inside the Tcl scripts, next to the Tcl code that uses, tests, and wraps it.”
The ultimate achievement he foresees is a possibility to re-factor the architecture of the Tcl interpreter radically. While it’s beyond the scope of this article to explain CriTcl fully, the effect will be something like Perl 6’s proposed exposure of its own parser and extension interface: designers and architects will solve problems dynamically, and at higher levels, while retaining full ability to program with the power and efficiency of C.
As of November 2001, CriTcl v0.18 works for Linux and MinGW-endowed Windows. Other operating systems should follow before long.
Variations
This presentation slightly simplifies Tcl’s story to a few points to make for an easier introduction. The most crucial have to do with portability and the Tcl-C programming interface.
The maintainers of the standard Tcl distribution are committed to keeping the language fully current on MacOS, Unix, and Windows. There are times when one of these — usually MacOS — lags the others by a bit. In general, though, they all move forward together.
Moreover, specialists maintain ports of Tcl to other operating systems such as BeOS, OpenVMS, handheld computers, and so on.
Also, the section on interfacing C and Tcl above mentions that there are three ways the example differs from most professionally engineered Tcl extensions. The first two, already mentioned, have to do with package hooks and the difference between an arbitrary C function signature and the signature Tcl requires.
The third difference has to do with Tcl’s “object interface.” Tcl is “dual-ported;” it provides two distinct APIs for in-process communication between C and Tcl. This article presents the “classic” interface, which both recommended books cover. In the last few years, though, most serious Tcl programming has switched over to a variant interface, which is slightly more complex, but often improves performance. The references throughout this article detail its use.
Summary
C and Tcl make good partners. Working together, they’re able to get more done faster than either can working alone. While there are over forty systems for combining the speed and efficiency of C with Tcl’s expressiveness and dynamism, readers of this article can learn in one sitting the most important and broadly applicable.
For more on this and related subjects, see at <http://phaseit.net/claird/comp.lang.tcl/HowToC.html>.
Cameron Laird is vice president of Phaseit, Inc. <www.phaseit.net/>. Tcl is the single computing language he most often uses, although most of his projects combine at least two of a dozen or so different languages. Cameron has contributed to several books on Tcl and often teaches classes on the language. Write to him at [email protected].
