Linux/IRAF Site Manager's Guide

3.8. Updating the full IRAF system

This section will describe how to recompile or relink IRAF. Before we get into this however, it should be emphasized that most users will never need to recompile or relink IRAF. In fact, this is not something that one should attempt lightly - don't do it unless you have some special circumstance which requires a custom build of the system (such as a port). Even then you might want to set up a second copy of IRAF to be used for the experiment, keeping the production system around as the standard system. If you change the system it is a good idea to make sure that you can undo the change.

While the procedure for building IRAF is straightforward, it is easy to make a mistake and without considerable knowledge of IRAF it may be difficult to recover from such a a mistake (for example, running out of disk space during a build, or an architecture mismatch resulting in a corrupted library). More seriously, the software - Linux, the GCC and F2C compilers, the local system configuration, and IRAF - is changing constantly. A build of IRAF brings all these things together at one time, and every build needs to be independently and carefully tested. An OS upgrade or a new version of the GCC/F2C compilers may not yet be supported by the version of IRAF you have locally. Any problems with the Linux configuration can cause a build to fail, or introduce bugs.

The precompiled binaries we ship with IRAF have been carefully prepared and tested, usually over a period of months. They are the same as are used at NOAO and at most IRAF sites, so even if there are bugs they will likely have already been seen elsewhere and a workaround determined. If the bugs are new then since we have the exact same IRAF system we are more likely to be able to reproduce and fix the bug. Often the bug is not in the IRAF software at all but in the host system or IRAF configuration. As soon as an executable is rebuilt (even something as simple as a relink) you have new, untested, software.

3.8.1. The BOOTSTRAP

To fully build IRAF from the sources is a three step process. First the system is "bootstrapped", which builds the host system interface (HSI) executables. A "sysgen" of the core system is then performed; this compiles all the system libraries and builds the core system applications. Finally, the bootstrap is then repeated, to make use of some of the functions from the IRAF libraries compiled in step two.

To bootstrap Linux/IRAF, login as IRAF and enter the commands shown below. This takes a while and generates a lot of output, so the output should be spooled in a file.

For a PC Linux (using uncompressed binaries) system:

% cd $iraf
% mkpkg linux
% cd $iraf/unix
% reboot >& spool &

There are two types of bootstrap, the initial bootstrap starting from a source only system, called the NOVOS bootstrap, and the final or VOS bootstrap, performed once the IRAF system libraries libsys.a and libvops.a exist. The bootstrap script reboot will automatically determine whether or not the VOS libraries are available and will perform a NOVOS bootstrap if the libraries cannot be found. It is important to restore the linux architecture before attempting a bootstrap, as otherwise a NOVOS bootstrap will be performed.

3.8.2. The SYSGEN

By sysgen we refer to an update of the core IRAF system - all of the files comprising the runtime system, excluding the HSI which is generated by the bootstrap. On a source only system, the sysgen will fully recompile the core system, build all libraries and applications, and link and install the executables. On an already built system, the sysgen scans the full IRAF directory tree to see if anything is out of date, recompiles any files that need it, then relinks and installs new executables.

To do a full sysgen of IRAF one merely runs mkpkg at the IRAF root. If the system is configured for multiple architecture support one must repeat the sysgen for each architecture. Each sysgen builds or updates a single BIN directory. Since a full sysgen takes a long time and generates a lot of output which later has to be reviewed, it is best to run the job in batch mode with the output redirected. For example, to update the linux binaries:

% cd $iraf
% mkpkg linux
% mkpkg >& spool &

To watch what is going on after this command has been submitted and while it is running, try

% tail -f spool

Sysgens are restartable, so if the sysgen aborts for any reason, simply fix the problem and start it up again. Modules that have already been compiled should not need to be recompiled. How long the sysgen takes depends upon how much work it has to do. The worst case is if the system and applications libraries have to be fully recompiled. If the system libraries already exist they will merely be updated. Once the system libraries are up to date the sysgen will relink the core system executables.

A full sysgen generates a lot of output, too much to be safely reviewed for errors by simply paging the spool file. Enter the following command to review the output (this assumes that the output has been saved in a file named "spool").

% mkpkg summary

It is normal for a number of compiler messages warning about assigning character data to an integer variable to appear in the spooled output if the full system has been compiled. There should be no serious error messages if a supported and tested system is being recompiled.

The above procedure only updates the core IRAF system. To update a layered product one must repeat the sysgen process for the layered system. For example, to update the linux binaries for the NOAO package:

% cd $iraf/noao
% mkpkg linux
% mkpkg -p noao >& spool &

This must be repeated for each supported architecture. Layered systems are independent of one another and hence must be updated separately.

To force a full recompile of the core system or a layered package, one can use rmbin to delete the objects, libraries, etc. scattered throughout the system, or do a "mkpkg generic" and then delete the OBJS.arc.Z file in the BIN one wishes to regenerate (the latter approach is probably safest).

A full IRAF core system sysgen currently takes anywhere from 3 to 30 hours or more, depending upon the system (e.g. from 30 hours on a VAX 11/750, to 3 hours on a big modern server). On most systems a full sysgen is a good job to run overnight.

3.8.3. Localized software changes

The bootstrap and the sysgen are unusual in that they update the entire HSI, core IRAF system, or layered package. Many software changes are more localized. If only a few files are changed a sysgen will pick up the changes and update whatever needs to be updated, but for localized changes a sysgen really does more than it needs to (if the changes are scattered all over the system an incremental sysgen-relink will still be best).

To make a localized change to a core system VOS library and update the linked applications to reflect the change all one really needs to do is change the desired source files, run mkpkg in the library source directory to compile the modules and update the affected libraries, and then relink the system binaries.

For example, assume we want to make a minor change to some files in the VOS interface IMIO, compiling for the linux architecture. We could do this as follows (this assumes that one is logged in as IRAF and that the usual IRAF environment is defined).

% whoami
iraf
% cd $iraf
% mkpkg linux
% cd imio
     (edit the files)
% mkpkg                          # update IMIO libraries (libex)
%
% cd $iraf/bin.linux             # save copy of old libex.a
% cp libex.a libex.a.V210
%
% mkpkg relink                   # relink and install new binaries

Changing applications is even easier. Ensure that the system architecture is set correctly (i.e. "mkpkg linux" at the iraf or layered package root), edit the affected files in the package source directory, and type "mkpkg -p <pkgname> update" in the root directory of the package being edited. This will compile any modified files, and link and install a new executable. You can do this from within the CL and immediately run the revised program.

We should emphasize again that, although we document the procedures for making changes to the software here, to avoid introducing bugs we do not recommend changing any of the IRAF software except in unusual (or at least carefully controlled) circumstances. To make custom changes to an application, it is best to make a local copy of the full package somewhere outside the standard IRAF system. If changes are made to the IRAF system software it is best to set up an entire new copy of IRAF on a machine separate from the normal production installation, so that one can experiment at will without affecting the standard system. An alternative which does not require duplicating the full system is to use the IRAFULIB environment variable. This can be used to safely experiment with custom changes to the IRAF system software outside the main system; IRAFULIB lets you define a private directory to be searched for IRAF global include files, libraries, executables, etc., allowing you to have your own private versions of any of these. See the system notes files for further information on how to use IRAFULIB.