Expect occupies a place in systems administrators' toolboxes much like that of marking pencils for carpenters, or axes for firemen: it hasn't changed in years, it's small and inexpensive, it's utterly unglamorous, there are substitutes that make it possible to live without it, but it makes a big difference in the situations that call for it.
Exploring Expect is the book that documents Expect. Incredibly, it remains in its first edition, materially unchanged since author and Expect inventor Don Libes first published it almost a decade ago. While Expect is a full-fledged, general-purpose programming language- — a superset of Tcl, in fact — its value is most apparent when there's a requirement "for dealing with crappy interfaces," in Libes' words. An abundance of applications have intractable or inconvenient interfaces, so there's plenty of need for Expect.
Here's an example of what Expect can do for you: suppose you need to
reset all the passwords for a collection of users, or update the passwords
to new randomized values, or ... well, you get the idea. This is just
the sort of chore that turns up in the daily life of a systems administrator.
It's also typical in that it's far more challenging than it first appears.
Your instinct should be to write a script that invokes passwd
to make each change. The problem with that is that passwd
is unscriptable, at least in conventional terms. You cannot successfully
redirect input to passwd the way most Unix programs encourage.
Expect to the rescue
Many people begin at this point to mutter about hacking the source
to getpasswd.c or other extreme measures. It's not
that hard, though. The standard Expect distribution includes a
couple dozen fascinating examples, including this small but potent
building block, called autopasswd:
#!../expect -f # wrapper to make passwd(1) be non-interactive # username is passed as 1st arg, passwd as 2nd set password [lindex $argv 1] spawn passwd [lindex $argv 0] expect "password:" send "$password\r" expect "password:" send "$password\r" expect eof
Do you see what that gives you? It's a password-changer easily driven
from the command-line. That's it: with these 11 lines and the Expect executable,
you can script all your password-maintenance solutions. As long as you
stay on one host, of course; we all know how cumbersome it can be to synchronize
operations across a network. If only there were some way to script telnet
or ssh ...
There is, of course. That's the message of this introduction: any time you think, "well, I can do this computer operation 'by hand', but I sure wish there were a way to automate it", your immediate reaction should be, "This is a problem for Expect!".
Anything having to do with passwords is likely to be such a
problem. Another of the examples the Expect distribution
includes is the 210-line passmass, which knows
how to update a user's password on a whole list of hosts.
Its heart is just as simple as autopasswd, but
it includes intelligence to handle rlogin,
ssh, telnet, and many other
variations that arise in connecting from one machine to
another.
The place of Expect
If you have good taste, this sort of description might make you uncomfortable. Isn't a distributed directory system like NIS+ or LDAP the right way to maintain accounts across a network? Don't these scripts I'm recommending have password information "in the clear", inviting misbehavior?
Yes, absolutely. LDAP indeed meets the requirement to provide consistent user account information across a network, and does much more beside. It's been carefully engineered for security, scalability, interoperability, and other desirable features.
That's why, despite its gloss as the "Lightweight Directory Access Protocol", LDAP is so heavy, certainly in comparison with Expect. LDAP deployments typically take months. I often go to a new site, download Expect, build it for the first time, write a solution, and execute it, within an afternoon. When your requirements stabilize, set up a system that's designed for the long term. Move your citizens from their flammable huts into safe, fireproof houses. Until you've done so, though, it's a good idea to have the flexibility and quick response of a few axes and fire hoses on hand. That's what Expect represents.
Your own start
Just as Expect is very light-weight in operation, it demands little to become a user. It is open source software, with a liberal BSD-style license. A few companies offer training classes — some of them excellent — and I recommend Exploring Expect to a colleague at least weekly. These aren't strictly necessary, though; I know plenty of systems administrators who've used Expect for as much as a decade simply by invoking its example scripts in their own shell-coded source.
Many modern Unix distributions include Expect by default. If yours don't,
consider installing Activestate's fee-free ActiveTcl binary.
While Expect has traditionally been distributed as a high-quality source
bundle, a few subtleties in co-existence
with Tcl installation(s) can confuse newcomers, even those experienced
with "configure; make; make test;
make install" sequences. ActiveTcl jumps past those obstacles.
Do explore the example/ directory. You'll find
small but complete programs to process passwords in a CGI
context, to enforce keyboard time-outs for carpal-tunnel
syndrome victims, to mirror xterms on two different displays
(handy for customer support!), to perform regression tests on
character-mode applications using a scripted terminal emulator,
and much more.
When you go beyond using Expect and its packaged utilities to writing your own scripts in the Expect language, you should find it straightforward. Expect's Tcl base is a general-purpose language, with constructs for arithmetic, file reading and writing, branching, looping, and so on, similar to those in shell, Perl, and other languages. Expect augments these with special commands in three areas:
- Debugging — Expect offers a rich array of introspective switches that report on the state of a program in progress;
- Pty management — "pseudoterminal" (pty) is the abstraction Unix uses to handle terminal service. Expect is particularly wise in dealing with ptys, and all their variations across different flavors of Unix;
- Character-oriented dialogues — Expect builds in
send,expect, and a few related commands to express the back-and-forth of a typical session when using a program: the computer process says something, you type a command, it responds, you type more, and so on.
Suppose you wanted to write your own program, one that would read a
list of newline-separated user names from the file account_list,
and change the password of each user to be the name itself, with "00"
as a suffix. Thus, if the account "robert" appeared on the list, this
utility would change "robert"'s password to "robert00". To do this takes
no more than:
package require Expect
set filename account_list
set fp [open $filename]
set content [read $fp]
close $fp
foreach account [split $content \n] {
set password ${account}00
# Notice that this tool must be run as
# the root user (or equivalent).
spawn passwd $account
expect "password:"
send "$password\r"
expect "password:"
send "$password\r"
expect eof
}
Does that look easy? Maybe you're ready to be an Expect programmer!
Looking ahead
The other installments in this series will cover "Lightweight regression testing" and "Unknown Expect". They expect a "cultural" knowledge of Expect, and will refer to Expect for examples, but both will be valuable even to systems administrators who choose not to use Expect in their own practices. You're welcome to write to me, and questions (that reach me in time) that relate to the next two topics will appear in the follow-up installments.
