source: trunk/doc/build-system @ 14489

This document explains how to build packages from the Athena source
tree, and then describes guidelines for designing the build system for
code we maintain in the Athena source tree.  A discussion of building
packages we get from third parties is in the file "third-party" in
this directory.

This file contains the following sections:

        Building packages from the source tree
        Building the tree
        Generic build system guidelines
        Using Autoconf
        Using a straight Makefile

Building packages from the source tree
--------------------------------------

To build a package from the Athena source tree, you should generally
first attach the source tree that you want.  You can either attach a
locker like "source-8.0" or "source-8.1" to get the sources for a
particular release, or you can attach "source" to get the current
development sources.  Releases prior to 8.1 will not build using the
procedure below.  Note that you can only build whole packages, not
arbitrary subdirectories; package names are listed in
packs/build/packages.

After attaching the correct locker, you can build a package by doing:

        set src = /mit/source           # or /mit/source-8.1, etc.
        mkdir /var/tmp/PACKAGE-NAME
        cd /var/tmp/PACKAGE-NAME
        synctree -s $src/PATH-TO-PACKAGE -d . -a $src/.rconf
        sh $src/packs/build/do.sh dist
        sh $src/packs/build/do.sh prepare
        sh $src/packs/build/do.sh all

If you are building the from the current sources (that is, $src is
/mit/source), the package may have dependencies on changes which have
been made since the last release.  In this case, use the "-c" flag to
do.sh to make the build use a recent build for tools and libraries.
The last three commands are then:

        sh $src/packs/build/do.sh -c dist
        sh $src/packs/build/do.sh -c prepare
        sh $src/packs/build/do.sh -c all

"do" accepts the following options (-c and -t are mutually exclusive):

        -s SRCDIR       Specifies the source hierarchy, if not default
        -d DESTDIR      Specifies the install hierarchy, if not
                        /afs/dev.mit.edu/system/$ATHENA_SYS/srvd-current
        -t TOOLDIR      Specifies the tool hierarchy
        -c              Sets the tool hierarchy to the install
                        hierarchy
        -n              Show what build steps would be done, but don't
                        do them

The default location for SRCDIR is the source locker you attached
(/mit/source, /mit/source-8.1, etc.).  "do" supports the following
build operations:

        dist            Copy in or create any materials necessary to
                        make source tree suitable for distribution
        prepare         Set up the source tree for building
        clean           Clean any files generated by the "all" operation
        all             Create automatically generated files
        check           Perform automated tests, if any
        install         Install in DESTDIR specified by -d (or default)

Building the tree
-----------------

To build the entire tree by hand, attach an appropriate source locker
and do something like the following:

        set src = /mit/source
        mkdir /var/tmp/build
        cd /var/tmp/build
        synctree -s $src -d .
        sh $src/packs/build/build.sh -b `pwd` -d /var/tmp/inst

"build" accepts the following options:

        -s SRCDIR       Specifies the source hierarchy, if not default
        -b BUILDDIR     Specifies the build hierarchy, if not /build
        -d DESTDIR      Specifies the install hierarchy, if not /.srvd
        -k              Keep going if a package fails to build
        -n              Only display a list of packages to be built
        -l              Log build output to a file.

As with "do", the default location for SRCDIR is the source locker you
attached.

Building the tree has the following requirements right now:

        * The system passwd file must have "pop" and "discuss" users.
          Athena machines should typically have these lines in them
          already, but you may have to add them on non-Athena systems.

        * You must have write access to the destination hierarchy, and
          be able to chown files and make setuid root files in that
          tree.

        * On Solaris, you need system:authuser access to the Athena
          cell in order to build third/gcc (since it is bootstrapped
          with the sunsoft compiler).

Therefore, if you have a machine newly installed with the vendor
operating system and AFS, and you want to build the Athena source tree
for the first time, you have to do a little work beforehand:

        * Make a temporary directory to put binaries in:

                mkdir /var/tmp/bin
                set path = ($path /var/tmp/bin)

        * You need a functioning synctree.  The sources are in
          athena/etc/synctree; because they are not world-readable,
          you may need to copy the sources somewhere temporary from
          another workstation in order to get them onto the machine
          you're building on.  Once you have a copy of the sources,
          building them is relatively easy:

                set src = /afs/dev.mit.edu/source/src-current
                ./configure
                make
                cp synctree /var/tmp/bin
                rehash

        * You need a functioning kinit.  In the worst case, you may
          need to do a partial build of third/krb5.

                cd /var/tmp; mkdir krb5; cd krb5
                synctree -s $src/third/krb5 -d . -a $src/.rconf
                make -f Makefile.athena prepare
                make -f Makefile.athena all

                cp src/clients/kinit/kinit /var/tmp/bin
                cp $src/packs/config/krb5.conf /etc
                mkdir /etc/athena
                cp $src/packs/config/krb.conf /etc/athena
                cp $src/packs/config/krb.realms /etc/athena

        * For Solaris, you need a functioning aklog.

                cd /var/tmp; mkdir aklog; cd aklog
                synctree -s $src/athena/bin/aklog -d . -a $src/.rconf
                ./configure --with-afs=/usr/afsws # or wherever
                make
                cp aklog /var/tmp/bin

        * For a few packages (third/perl and some things under packs),
          /os must point to an image of the operating system.  A
          symlink from /os to / will usually service if a pristine
          operating system image is unavailable.

With these tools, you should be able to set up the build environment
as documented above.

Generic build system guidelines
-------------------------------

Here are some desirable properties of a build system for programs or
libraries, regardless of how it is implemented:

        * It should use feature tests, not platform tests.

        * Once the appropriate configuration steps (e.g. running
          configure or imake) have been done, "make all" should
          generate all automatically generated files without ever
          modifying any source files.  (For example, "make all" should
          not try to regenerate configure from configure.in, and a
          build system should not wait until "make install" time to
          generate a file which is going to be installed.)

        * "make install" should install all files appropriate for the
          target directories without modifying any files in the build
          tree.  In particular, it should not depend on the "all"
          target.

          "make install" should use the install command to install
          files, not cp or tar or shell redirection.  To install
          multiple files in the same way, use a single install command
          rather than a for loop.

        * "make clean" should clean all files generated by "make all."
          "make distclean" should also delete files generated by the
          configuration steps.

        * "make check" should run self-tests.  It should not assume
          that the program has been installed, but it should assume
          that the program has been built.  It should not modify the
          build tree in any way; in particular, it should not depend
          on the "all" target.  If there are no self-tests, "make
          check" should not generate an error.

        * In a multi-directory source tree, it should be possible to
          change to a subdirectory of the build tree and run any of
          the aforementioned make targets with the same results as
          having descended into that directory from above.  It is okay
          to assume that previous parts of the build tree have been
          built, but it is not okay to rely on make command-line
          parameters passed in from the parent directory.

Using Autoconf
--------------

For packages in the "athena" hierarchy, we use autoconf.  Autoconf is
not perfect, but it's the only reasonable package out there for doing
feature tests as opposed to platform tests.  See the Autoconf info
pages (available in the default emacs info menu) for information about
using Autoconf in general.

When we build the program for the Athena environment, we will use a
config.site file (already written; you don't need to write it or point
to it) which does the following:

        * Sets the shell variable "athena" to "true".

        * Sets the appropriate compiler (using the CC environment
          variable) as well as other compilation tools (yacc, lex) for
          the platform in question.

        * Sets the appropriate X include path and library path for use
          with AC_PATH_X.

        * Sets the appropriate with_* variables for finding Motif, AFS
          libraries, Kerberos 4, Kerberos 5, com_err, and ss.  If your
          package uses any of those libraries, copy the aclocal.m4
          file from packs/build and use the macros contained within.

        * Sets prefix to /usr/athena, datadir to '${prefix}/lib',
          sbindir to '${prefix}/etc', and sysconfdir to '/etc/athena'.
          You do not need to take these into account in configure.in;
          just put in your Makefile.in:

                prefix=@prefix@
                exec_prefix=@exec_prefix@
                datadir=@datadir@
                sbindir=@sbindir@
                sysconfdir=@sysconfdir@

        * Sets shell variables lbindir and lsbindir to /bin/athena and
          /etc/athena respectively.  If your build system needs to
          install files in /bin/athena or /etc/athena when built for
          the Athena environment, use the following in your configure.in:

                test -z "$lbindir" && lbindir='${bindir}'
                test -z "$lsbindir" && lsbindir='${sbindir}'
                AC_SUBST(lbindir)
                AC_SUBST(lsbindir)

          And then in your Makefile.in:

                prefix=@prefix@
                exec_prefix=@exec_prefix@
                bindir=@bindir@
                sbindir=@sbindir@
                lbindir=@lbindir@
                lsbindir=@lsbindir@

The build environment will ensure that the following things are true:

        * The environment variable ATHENA_SYS is set to the current
          platform's desired ATHENA_SYS value (e.g. "sun4m_54").

        * The environment variable HOSTTYPE is set to the current
          platform name (e.g. "sun").

        * The environment variable CONFIG_SITE points to the Athena
          config.site file.

Using a straight Makefile
-------------------------

For packages in the "packs" hierarchy, and in some parts of the
"third" hierarchy where we construct the build system by hand, we use
a straight Makefile, since feature tests are not necessary for the
packs hierarchy.

The build system will ensure that the same environment variables are
set as for using Autoconf.  In addition, the make variable CC will be
set to the appropriate compiler for the current platform when the
"all" target is invoked.  You should be sparing in your use of C
source code in packages using a straight Makefile, but for small,
portable C programs it's okay.

The subdirectories "platform/$HOSTTYPE" or "arch/$ATHENA_SYS" should
be used for platform or system-dependent components of a package using
a straight Makefile.

Start each Makefile with "# $Id: $" and "SHELL=/bin/sh".  The latter
is to work around a bug in the Irix make where it runs commands under
${SHELL} rather than /bin/sh.