Dr. Dobb's | GNU Autoconf | January 01, 2006

GNU Autoconf

Build processes can be frustratingly brittle. If you find yourself making predictable, repetitive changes to your build procedure to suit a given environment, Autoconf may be for you.

January 01, 2006
URL:http://www.drdobbs.com/gnu-autoconf/184402060

January, 2006: GNUAutoconf

Ethan McCallum is a freelance technology consultant who spends a lot of time with Linux, C++, and Java. He can be reached at ethanqm@ penguinmail.com.

Although we design software to be flexible and adaptable at runtime, our build processes are often still rigid—once a project moves beyond the most trivial of examples or stub tests, changes to the build environment cause a snowball effect throughout the directory tree. Addressing headaches like this, GNU Autoconf (http://www.gnu.org/software/autoconf/) provides a uniform interface to creating your makefiles at build time. If you find yourself making predictable, repetitive changes to your build procedure to suit a given environment—even through a homegrown sed/Perl tool—Autoconf should help.

All but the most simple of software is driven by some sort of file or command-line parameters to control its behavior—network listener ports, process owners, number of threads to spawn, and the like. Deferring such configuration from compile time to runtime provides tremendous flexibility to the program because the product of a single build is adaptable to myriad environments.

Autoconf encourages this adaptability within the build process by similarly deferring decisions until build time—the location of supporting tools, flags for OS-specific compilers, installation directories, file ownership, and so on. Results of queries against the build host are merged with templates to create makefiles (actually, Autoconf can create any kind of file based on a template, but is typically used to create makefiles), and command-line tasks can be wrapped inside a tool such that you need not even remember to run them individually.

Having a clean, standard process to manage this is ideal for both developers and end users. (Raise your hand if you remember hand-editing makefiles and, in some cases, header files, to get a piece of software to build on your system.) An added benefit of letting Autoconf handle this sort of work is that it minimizes the chances of error introduced by human element; manual edits or lists of build procedures, for example.

If you frequently download and build open-source tools from source, chances are you are already familiar with Autoconf. Autoconf generates the configure of the configure/make/make-install routine. The configure process queries the host and uses its findings to bootstrap the makefiles, customizing them for the target build environment.

What may be surprising is that configure is a shell script—and not a fancy shell script at that, just the most basic of Bourne shell syntax. Autoconf creates the configure script by expanding a series of m4 macros, as specified by users.

Configure.ac

Autoconf parses a file—typically called "configure.ac"—and expands any m4 macros it encounters. The macros are fragments of shell logic, which is why the resultant configure is a shell script. Similar to a preprocessor, anything Autoconf does not recognize as a macro it adds to configure untouched, which means a configure.ac could contain a mix of macros and raw shell code. Purists might eschew this notion, but the realists see this as a means to bootstrap Autoconf integration or perform a proof-of-concept. Over time, such quick-hit shell actions may be wrapped in proper m4 macros, so the lofty goal of a "pure" configure.ac can still be achieved. To introduce raw shell code into configure.ac is typically a precursor to writing your own site- or project-specific Autoconf macros.

The shell syntax to be used here is plain Bourne shell (/bin/sh)—not C Shell, Korn Shell, or Bash, but very plain, portable Bourne shell. I emphasize the word "portable" because Bourne shells are not the same across all operating systems.

While the Autoconf manual says configure.ac must contain at minimum two specific Autoconf macros, you need three macro calls for Autoconf to be of any real use:

AC_INIT specifies the name and version of your package. Optionally, you can include an e-mail address for filing bug reports.

AC_INIT([myproject], \
	[3.5.1], \
	[bugs-myproject@mydomain])

This information is included in configure's help message and is available as expansion variables, which are what Autoconf uses to turn makefile templates into usable makefiles.
AC_CONFIG_FILES lists the files configure creates at the end of its run. The filenames passed to AC_CONFIG_FILES need not exist—in fact, they will be overwritten when configure runs—but their templates most certainly should. A template is named for the file in question, plus a .in suffix. For example, in this call to AC_CONFIG_FILES:

AC_CONFIG_FILES([Makefile \
	base/Makefile \
	network/Makefile])

the files Makefile.in, base/Makefile.in, and network/Makefile.in must exist relative to the location of the configure script.
AC_OUTPUT performs variable substitution on the templates indicated by the call to AC_CONFIG_FILES. It takes no parameters.

	AC_OUTPUT

The square brackets are included in the text. While some technical manuals have adopted the square brackets to indicate an optional parameter in examples, Autoconf (really, m4) sees them as quoting characters—they act as boundaries such that values are not misinterpreted. I recommend that any parameter, other than a single word, be quoted.

Autoconf is also sensitive about spaces—there should be no spaces between a square brace and the text it encloses, nor should there be space between a closing square brace and a following comma. In cases where you would like to move a value to the next line, use tabs instead of spaces.

Creating Configure.ac

Once you have a few Autoconf implementations under your belt, you will have the built-in macro names memorized and create configure.ac from scratch. For the neophytes (or, in my case, the forgetful), the Autoconf distribution includes the helper tool autoscan. Run autoscan at the base of your source tree and it recursively scans the directory structure for source and header files. The resultant configure.scan file is a template containing Autoconf macros relevant to header files, function calls, and compiler checks based on the scan of your source code. This file should be copied to configure.ac, which you would then customize.

Autoscan is not clairvoyant, however, so you must supplement its labor with some tests of your own. In fact, many macros (such as AC_ARG_WITH) are akin to abstract base classes in C++: They are not to be run as-is, but extended and customized with your own code. Some of the basic macros and output variables include:

AC_PROG_CC and AC_PROG_CXX check the command search path for a set of given C or C++ compilers, respectively. Once a particular compiler is located, set this value to the makefile variable $CC for C, or $CXX for C++. The entire list is quoted in m4 fashion, not individual elements:

	AC_PROG_CXX([ gcc3 , gcc , cc ])

AC_CHECK_TYPES checks for each variable type in the list, defining the preprocessor macro HAVE_{TYPE} to "1" if found. Optionally, provide some shell code or m4 macros to run for either case of whether the type is found. If a list of header files is not provided as the fourth argument, the macro scans some predetermined system header files.

	AC_CHECK_TYPES(
		[uid_t , struct stat SSL],
		[true],
		[AC_MSG_NOTICE([missing a key \
		data type, this build may \
		fail])]
	)

AC_FUNC_MALLOC and AC_FUNC_REALLOC check for the functions malloc() and realloc(), respectively.
AC_ARG_WITH provides a framework to handle the --with-{arg} values passed to configure on the command line. The value passed via --with-arg is available as the shell variable $withval. The second argument is what is printed in configure's help message; here I use the built-in AC_HELP_STRING macro to provide proper spacing and line wrapping such that the help message looks uniform:

	AC_ARG_WITH(
		[ssl],
		[AC_HELP_STRING([—with-ssl], \
		[provide the location of your \
		SSL toolkit])],
		[
			## ... code to execute if \
			## this flag is passed to 
			## configure
		],
		[
			## ... code to execute if \
			## this flag is NOT passed \
			## to configure
		]
	)

variable @configure_input@ is a blanket warning that the file in question was generated by configure. When you see this, you should know to edit this file's Autoconf template (its .in file) and not the one you are currently reading.
variables @CC@, @CXX@, and @CPP@ represent the C compiler, C++ compiler, and preprocessor, respectively.
variable @target@ is the target system type in the canonical GNU format of {cpu}-{vendor}-{os}.

Listing 1 is the full version of this macro. If users pass --with-ssl=/some/path to configure, it sets the output variables @CXXFLAG_SSL@ and @LDFLAG_SSL@ for C++ compiler and linker flags, respectively, based on the value of the --with-ssl flag; otherwise, the output variables are left blank.

Also note its deficiencies: You have to trust the user input (or lack thereof) to provide the location of a required library. If users omit the --with-ssl flag or provide an incorrect value, the build fails, perhaps at the tail end of a time-consuming compilation phase. It is preferable to catch such a problem during configure's tests. Autoconf has tools to attempt a sample compile/link process to confirm the availability of external headers and libraries.

Writing Custom Macros

Autoconf's bundled macros suit a variety of purposes, and as such you may find yourself in need of some functionality specific to your project. Creating your own m4 macros to run through Autoconf is a straightforward affair, as long as you are familiar with shell programming and have a grasp of m4 quoting rules.

As previously stated, Autoconf macros use the most basic, portable Bourne shell syntax, and your homegrown macros should follow suit. If you use Autoconf solely for internal build purposes, you can probably get away with using shell constructs of Bourne derivatives (such as ksh and bash, but not csh or tcsh), but you would then have to explicitly invoke that shell when running configure (but this is strongly discouraged):

ksh ./configure

Just as Autoconf by default looks for its input in configure.ac, it seeks project-specific macros in a file named "aclocal.m4." (Autoconf also supports sitewide macros in a file called "acsite.m4," thus allowing you to create a hierarchy within your organization, much like a code library.)

In addition to shell fragments, your macro may also call existing macros. Common built-in macros of this sort include:

Members of the MSG_{type} family print a message to the console, where {type} is one of CHECKING, RESULT, ERROR, FAILURE, NOTICE, or WARN. It is preferred that these facilities be used instead of using the shell's built-in echo command.
AC_SUBST is used to make a value available as an output variable; that is, available in template files as @variable_name@. Be sure to properly quote (shell quote, not m4 quote) any values that have spaces or may be otherwise misinterpreted by the shell.

Keep in mind, macros can do anything you could normally call from the shell; they need not necessarily check the target host or create any sort of output variable. For example, you could have some conditional logic that calls other macros based on the presence of a debug flag.

All good code should include comments, and Autoconf macros are no exception. Autoconf macros support two types of comments: Those lines beginning with "#" are ignored by Autoconf's macro expansion parsing but are included in the resultant configure script. Lines beginning with "dnl" are for quoting the m4 text itself, and are therefore omitted from configure.

Lastly, naming convention counts here. All of the built-in Autoconf macros' names begin with the prefix "AC_" so you should consider that reserved for Autoconf use to prevent clashes. For project-specific macros, you could name the prefix for the product's acronym. Sitewide Autoconf macros and variables (such as those you would see in acsite.m4 rather than aclocal.m4) could be named for your company or team. In these examples, I have chosen the generic MYPROJECT_ prefix for all custom macros and variables.

To define a custom macro, call the macro AC_DEFUN from your aclocal.m4 file. AC_DEFUN takes the new macro's name and contents (body), quoted in m4 fashion, as its arguments:

AC_DEFUN(
	[MYPROJECT_MACRONAME],
	[
		## ... macro actions ...
	]
)

Listing 2 implements a custom Autoconf macro based on the built-in AC_ARG_ENABLE macro, which permits conditional inclusion of compile-time features based on whether the --with-{feature} flag is passed to configure. The feature in question is a compiler flag that enables runtime profiling.

The macro in Listing 2 uses the AC_MSG_INFO macro to communicate with the end user, rather than pure shell echo statements. This provides consistent output for the configure script. The results of the test are available in makefiles as the substitution variable @CXXFLAGS_PROFILING@, which will be either -pg or blank, based on whether the flag --with-profiling is passed to configure.

Listings 3 and 4 are a sample configure.ac and Makfefile.in, respectively, which include our custom macros from Listings 1 and 2. I intentionally used a basic makefile for brevity. For example, instead of creating makefile variables for the compiler flags, I include the Autoconf output variables inline with the compiler commands.

Once you've written configure.ac, creating configure is a snap: Simply run autoconf in the directory where you have placed configure.ac (most likely at the root of your project's source tree). Be sure to rerun the command each time you update configure.ac, or your changes will not be incorporated into the configure script.

Implementing Autoconf: Existing Projects

There are myriad ways to integrate Autoconf into an existing build process. In all cases, it would behoove you to thoroughly review and, if necessary, clean up your existing build process before making any changes. This should reveal where you make your hand-edits, and which makefile definitions should be put under Autoconf control. In turn, this will make the initial Autoconf introductions brief, providing a relatively low downtime for your build process. If you already have a semiautomated process for performing your makefile substitutions, you could wrap that with Autoconf and phase it out over time.

Next, rename all files that will be updated by Autoconf such that they are recognized as templates: Add a .in suffix to their names. (Be sure to do this within your source control system as well.) Educate local development staff such that everyone understands that only changes to the template files will survive a source-code check-in. Include these filenames in the AC_CONFIG_FILES section of configure.ac. This way, from the start, each call to configure will create the target files.

Mark up the new templates using Autoconf output variable syntax @variable_name@. Be sure to review both the Autoconf manual and your aclocal.m4 to confirm the names of the output variables and the macros in which they are defined. At this point, if your build process is too delicate to call proper macros from configure.ac, your first version of this file can simply contain a series of AC_SUBST calls that use your default values. Listing 5 includes an example of such a configure.ac.

In Listing 5, the call to AC_CONFIG_FILES uses tabs and newlines to separate its arguments for clarity. Also, the values passed to AC_SUBST are double quoted. Technically, only the CFLAGS substitution requires quotes, as its value contains a space; but to quote all values follows good shell form.

With the ability to perform variable substitutions, you can view your makefiles as a (limited) type of source code: "That macro is subject to change and is found in seven makefiles; let's put it under Autoconf."

Implementing Autoconf: New Projects

Using Autoconf in new projects (that is, making it part of the build process from the beginning) follows a path similar to integrating it into existing projects. Most likely the project will begin as stub code, so you can run autoscan on that to extract special headers or definitions you will use. From there, it becomes an iterative process: As the source tree grows, new directories are added, thus new files must be added to AC_CONFIG_FILES. If a new tool is required for a particular module, a new macro call is added to configure.ac (perhaps a custom macro is created in aclocal.m4) and the new output variables are added to that module's makefile. As part of release planning, further updates are made to configure.ac and the main makefile. In essence, Autoconf management can take on a life of its own, a project within your project. As time progresses, though, the Autoconf maintenance should subside even as the actual development moves along at full steam.

Some practices apply to both new and existing projects:

Make a note of how Autoconf checks the build environment's architecture, and the form in which it reports the value. Keep a (static) list of architectures under which your product has been successfully built. When configure is run, make note of the architecture and let your end user know whether a compile has been attempted under this architecture before (that is, denote the chances of a successful build).
Keep an eye on future porting efforts: Provide reasonable defaults when the target host does not match any of your previous build specifications or, if this is not possible, create a means for configure to fail gracefully if it has not been built on this sort of platform before. If you do this, be sure to include another configure flag to override the failure/exit at this point.
Tailor Autoconf design to your expected mode of variance: If you port to a wide variety of architectures, then make your focus architecture dependencies.
Keep both configure.ac and configure in source control. This is especially helpful if you distribute your source code to customers or the general public: Such people need not have Autoconf installed on their machine to build your product as long as the configure script is available.

Using Autoconf

Authors of open-source tools are acutely aware of varying build environments; it is no coincidence, then, that Autoconf drives the build process for so many open-source tools today. This needn't limit Autoconf to builds of open-source tools, however. Anyone who has ever designed a build environment for a multideveloper project can see the advantages of adaptable build-time configuration.

All of this is good, but skeptics might raise an eyebrow: "Is all of this overkill for my project?" Perhaps, but not likely. If your project consists of a single makefile with definitions that never change, makes use of no external products/libraries, and you've never changed environments or compilers, then Autoconf probably is overkill. (Then again, who has a project like that and what are the chances it will remain as such?)

Even if Autoconf sounds like a great idea for your project, there are other considerations. From a technical standpoint, your first task is to determine whether Autoconf runs in your environment.

Also consider the political ramifications, as determined by your team or company's stance toward free software. Some sites are wary of tools for which there are no explicit contracts, nor technical support contacts. To that, you may be able to quell management's fears by explaining Autoconf's ubiquity: It is part of the build process of a great number and wide variety of open-source packages, including the long-since-stable crowd of applications that are used, free of support contracts, every day in production environments.

Even then, you may want to have the local legal staff review the license such that they are aware of any sticking points. My understanding of the Autoconf license is that the configure script generated by the tool is not covered by the GNU Public License (GPL), nor is the code built by the process bootstrapped by configure; but it would still be to your advantage to have the license reviewed by a competent attorney.

Once you've confirmed Autoconf is right for you, the next hurdle is to implement it. There's no time like the present, and this article has provided some tips on integrating Autoconf into an existing project. As with the introduction of any widespread change, you must decide between "baby steps" and "leaps and bounds"; in other words, it comes down to how much disruption you can afford and how much confidence you have in the changes you make. There is no one true answer here. For the faint of heart, Autoconf's design supports an incremental introduction, in which most of the changes may take place behind the scenes.

Conclusion

Like most development tools, Autoconf requires an initial investment of time and effort, but as it takes care of some of the drudgework, the payoff is that you get to spend more time doing what you want to do in the first place—write code.

CUJ

January, 2006: GNUAutoconf

Listing 1

dnl
dnl MYPROJECT_CHECK_SSL
dnl
dnl Allow the caller to specify the base tree for
dnl ssl, and set compiler/linker flags accordingly
dnl
dnl Sets output variables: @LDFLAG_SSL@ and @CXXFLAG_SSL@
dnl

AC_DEFUN(
[MYPROJECT_CHECK_SSL],
[
AC_ARG_WITH(
   [ssl],
   [AC_HELP_STRING([--with-ssl], [provide the location of your SSL toolkit])],
    [
        ## if provided, set -L and -I accordingly
        MYPROJECT_DIST_SSL=${withval}
        MYPROJECT_LDFLAG_SSL="-L${MYPROJECT_DIST_SSL}/lib"
        MYPROJECT_CXXFLAG_SSL="-I${MYPROJECT_DIST_SSL}/include"

        AC_MSG_NOTICE([We'll look for ssl under ${MYPROJECT_DIST_SSL}])
        AC_SUBST(LDFLAG_SSL, [${MYPROJECT_LDFLAG_SSL}])
        AC_SUBST(CXXFLAG_SSL, [${MYPROJECT_CXXFLAG_SSL}])
    ],
   [
       ## if it's in a system path, there's no need for the "-L" or "-I" flags
       ## this means, if the tool isn't available in default header/library
       ## search paths, the build will fail.
       AC_MSG_NOTICE([We'll look for ssl in the default 
                                               preprocessor/linker paths])
       AC_SUBST(LDFLAG_SSL, [""])
       AC_SUBST(CXXFLAG_SSL, [""])
    ]
)
]
)

January, 2006: GNUAutoconf

Listing 2

dnl
dnl MYPROJECT_ENABLE_PROFILING
dnl
dnl sets output variables to be used as 
dnl compiler flags related to profiling, e.g.
dnl with GNU gprof.
dnl
dnl Sets outcatput variable: @CXXFLAG_PROFILING@
dnl

AC_DEFUN(
[MYPROJECT_ENABLE_PROFILING],
[
AC_ARG_ENABLE(
    [profiling],
    AC_HELP_STRING([--enable-profiling], [build executables for profiling]),
    [
        AC_MSG_NOTICE([Enabling runtime profiling flags])
        AC_SUBST(CXXFLAG_PROFILING, ["-pg"])
    ],
    [
        ## no need to send a message if we're not profiling
        AC_SUBST(CXXFLAG_PROFILING, [""])
    ]
)
]
)

January, 2006: GNUAutoconf

Listing 3

##
## Autoconf input for project MyProject
##

AC_INIT([myproject],[3.5.1],[bugs-myproject@mydomain])

dnl - compiler check
AC_PROG_CXX( g++3 c++3 g++ c++ )

dnl - basic header files
AC_HEADER_STDC
AC_CHECK_HEADERS(unistd.h)

dnl - OS-related types
AC_TYPE_UID_T
AC_TYPE_MODE_T

dnl - C-based functions we will need
AC_CHECK_FUNCS([strdup])

dnl - where is the SSL toolkit?
MYPROJECT_CHECK_SSL

dnl - should we enable runtime profiling?
MYPROJECT_ENABLE_PROFILING

dnl - the roundup
AC_CONFIG_FILES([
    Makefile
    base/Makefile
    net/Makefile
    net/defaults.h
])

dnl - write out the files based on the templates + configure's queries
AC_OUTPUT

January, 2006: GNUAutoconf

Listing 4

##
## toplevel Makefile for MyProject
##
## WARNING:
##  @configure_input@
##

all: myproject

myproject:
    @CXX@ -o myproject @CXXFLAG_PROFILING@ @CXXFLAG_SSL@ @LDFLAG_SSL@ -lssl
## EOF

January, 2006: GNUAutoconf

Listing 5


AC_INIT([myproject],[5.1],[bugs-myproject@mydomain])
AC_SUBST(CC,["/usr/local/bin/gcc3"])
AC_SUBST(CFLAGS,["-O3 -pg"])
AC_SUBST(DEFAULT_OWNER,["nobody"])
AC_SUBST(DEFAULT_PORT,["2345"])
AC_CONFIG_FILES([
    Makefile
    base/Makefile
    net/Makefile
    net/defaults.h
])