STSDAS Tables Package Uses FITSIO

P. Hodge

Space Telescope Science Institute

Abstract:

1. Introduction

The STSDAS tables package is an IRAF ``external package,'' that is, it runs under IRAF but is distributed separately from the core IRAF system. The tables package contains a variety of tasks pertaining to tables, such as sorting, searching, plotting, and conversion to/from FITS. These tasks are built upon a set of SPP-callable I/O routines, which are included in the package and can be used by anyone to write new IRAF tasks. A Fortran interface to these SPP routines is also included in the tables package.

FITSIO, written by William D. Pence for the HEASARC project (High Energy Astrophysics Science Archive Research Center), is a set of Fortran subroutines for reading and writing FITS files, including primary array and IMAGE, TABLE and BINTABLE extensions. The package includes a C interface and an SPP interface as well. HEASARC has also written an IRAF external package called FTOOLS containing many useful tasks for working with FITS files.

2. Discussion

The STSDAS tables package originally read and wrote only binary tables in a format that is internal to STSDAS. A few years ago, support was added for plain ASCII tables, text files with data in rows and columns, including standard input and output. This was done by changing the table I/O routines, so the tasks that call those routines did not need to be modified, only relinked.

Recently the table I/O routines were modified to call routines in the FITSIO library for the case that the file containing the table is a FITS file. Again, the tasks that call the table I/O routines only needed to be relinked, for the most part. All tasks that use STSDAS tables can therefore operate on FITS tables as well. The SPP version of the FITSIO library is included in the table I/O object library, and the source code is distributed with the STSDAS tables package.

Tables in FITS files are stored in extensions; the primary array, if it exists, is an image. There are several types of FITS extensions, including IMAGE, TABLE and BINTABLE. The TABLE extension is an ASCII format, while the BINTABLE extension is binary. The STSDAS table routines write only BINTABLE extensions, but they can read either TABLE or BINTABLE. The extensions in a FITS file can be uniquely identified by number, as described in the ``Further Details'' section.

While the STSDAS table I/O routines distinguish between STSDAS binary tables and text files by the contents of the file, FITS files are indicated by filename extension ``.fits'', with ``.fit'' and ``.??f'' also recognized. This is the same convention used by the FITS image kernel (see the paper by Zarate and Greenfield) which provides access to the primary image and image extensions in a FITS file for all IRAF tasks that operate on images. The present work for tables complements the FITS image kernel.

The Fortran version of FITSIO uses Fortran I/O and is used to write stand-alone Fortran programs. The SPP version, on the other hand, uses IRAF I/O routines and is linked into tasks that run under the IRAF CL. Since IRAF tasks cannot use Fortran I/O, it was the availability of an SPP version that made it possible to incorporate FITSIO into the tables package.

3. Further Details

Multiple BINTABLE extensions may be included in one FITS file, and these may be intermingled with extensions of other types, so it is necessary to be able to uniquely identify a particular extension. When opening an existing table, the extension number enclosed in brackets can be appended to the FITS file name. The indexing convention is that the first extension---not the primary array---is number one. If the desired extension includes the EXTNAME header keyword, then the value of that keyword can be used instead of the extension number. Case is ignored when checking EXTNAME values. Note, however, that if more than one extension has the same EXTNAME, the first extension with that name will be opened, with no warning about the lack of uniqueness. If the file name is given with neither name nor number, the first extension that is a table will be opened.

When a task creates a new output FITS table, a new extension will always be created. That is, the ``overwrite'' option in the FITS image kernel is not available for tables. If the output FITS file itself does not exist, the file will be created, a minimal primary header will be written, and the table will be written as the first extension. There is therefore no need to specify the extension number. An extension name may be given, however, in which case an EXTNAME keyword with that value will be added to the header when it is created. There is no check that the name is unique.

If a table is opened read-write, then the table need not be the last extension in a FITS file, and the table may be modified in-place. The SPM_quottedit" task does this, for example. Rows may be added or deleted, and new columns may be defined.

The FITS image kernel includes the option of specifying the EXTVER keyword (version number) in addition to EXTNAME. This makes it possible to distinguish between extensions with the same EXTNAME by assigning different values of EXTVER. We intend to add the EXTVER option for tables in the near future.

The SPM_quottcopy" task (or the SPM_quottbtcpy" subroutine) can now change the table type as it copies a table. SPM_quottcopy" can also read from SPM_quotSTDIN" or write to SPM_quotSTDOUT"; for example, you can SPM_quottcopy" to SPM_quotSTDOUT" to print a table. The output table will be a FITS table if the filename extension is ``.fits'', and the output will be an STSDAS binary table if the filename extension is ``.tab''; otherwise, the output will be either an STSDAS table or a text table, according to the input file type. If the input to SPM_quottcopy" is a FITS file, one particular extension must be specified; the entire file will not be copied. The primary header will not be copied either, since it is not associated with any table. Use SPM_quotcopy" to copy the entire FITS file.

4. Programming Interface

The programming interface for STSDAS tables allows users to write IRAF tasks in SPP or Fortran that read or write tables. There is a list of the user-level subroutines and a description of their calling sequences in tables$doc/tbtables.tex. The directory tables$lib/tbtables/doc contains other descriptive material, including sample SPP source files ``ex.x'' and ``example.doc''.

The STSDAS tables interface is roughly comparable to the high level interface for tables in FITSIO, or perhaps even a little simpler. The STSDAS tables interface is appropriate to use if you would like transparent access to different table types, FITS, STSDAS, and text. FITSIO is clearly more powerful, though. FITSIO gives access to both images and tables, for example, and it does not require closing and reopening the file to access different extensions. FITSIO also supports columns with data types byte and complex, as well as variable-length arrays. If you need that flexibility then you should use the FITSIO interface directly.

Even though transparent access to different table types is one of the features of the STSDAS tables package, there are a few cases where the table type does matter to the programmer. One has to do with filename extensions. A task cannot assign an arbitrary filename extension if the output is to be a FITS file, due to the naming convention the tables package uses for FITS files. Other cases are due to the fact that one FITS file may contain more than one table. If a task checks the filename extension, for example, the task must allow for the possibility of a name or number in brackets appended to the file name. (A subroutine is provided to extract such information.) Another case is when making a temporary copy of a table. This is often done in order to modify the table in-place and then rename it back to the original. If a single extension were extracted to a temporary file, modified, and then renamed back to the original FITS file, the rest of the original file would be lost. The simplest solution is to copy the entire FITS file. For another example, suppose a task has two output tables and writes to both at the same time. After IRAF opens a file for writing, the file cannot be opened a second time for writing without first closing it. Therefore the two output tables in this example cannot be in the same FITS file. Multiple extensions can be written one at a time, of course, and if they are opened read-only then several extensions in the same file can be open simultaneously.

5. Examples

tt> tcopy  STDIN  t.fits[test]
tt> tcopy  t.fits[test]  STDOUT
tt> tprint  t.fits                # These
tt> tprint  t.fits[1]             # four
tt> tprint  t.fits\[1]            # are
tt> tprint  t.fits[test]          # equivalent.

# Suppose a directory listing shows
#      bright.tab   example.tab  ngc5272.txt
tt> tcopy  *.txt,*.tab  all.fits
tt> tinfo  all.fits[ngc5272.txt]
tt> tlcol  all.fits[example.tab]
tt> tprint  all.fits[bright.tab] rows="5-8" col="hr name ra dec"

# Create a FITS table from an ASCII data file "civ.dat".
tt> tcreate  spec.fits[civ]  civ.cd  civ.dat

# The "civ.cd" file that we just used defines two columns,
# each row of which is a 22-element array.  "civ.dat" contains
# 22 wavelengths and 22 fluxes for the first table row,
# followed by 22 wavelengths and 22 fluxes for the second row,
# etc.
tt> type  civ.cd
wavelength   d[22]   %10.3f  Angstroms
flux         r[22]   %12.5g

# Extract the flux values for the first three rows to scalar
# columns, using a separate output file for each row.
tt> taextract  spec.fits[civ]  row1.fits  1   flux
tt> taextract  spec.fits[civ]  row2.fits  2   flux
tt> taextract  spec.fits[civ]  row3.fits  3   flux

# Plot the fluxes.
tt> sgraph  "row?.fits flux"  stack=yes

# Copy the FITS table to an STSDAS table.
tt> tcopy  spec.fits[1]  spec.tab
spec.fits[1] -> spec.tab

# This would fail because text tables don't support arrays.
tt> tcopy  spec.fits[civ]  STDOUT

# Print the array of fluxes in row two.
tt> tdump  spec.fits[civ]  datafile="STDOUT"  col=flux  row=2