STB

Save the Bits Archive


Goal
To transparently archive the raw data from all KPNO/NSO nighttime telescopes on Kitt Peak. This has recently expanded to include data from CTIO, as well. The WIYN telescope board has approved a CDrom based upgrade to Save the Bits  that will also serve as the primary data distribution mechanism for the telescope, in addition to the archival functions. The central data center for both CTIO and WIYN is at NOAO headquarters in Tucson.

Current Status Reports for Kitt Peak National Observatory
The Archive status reports the current state of the archive queue and tape drives.

The KPNO domes report summarizes the health of the queue and disk space in each dome, as well as lists the latest archive transactions from each telescope computer.

The daily and monthly archive statistics.

Retrieving archive data
To request data from the Save the Bits  archives, you may fill out the provided web-based form. Post-proprietary access to the data will only be provided on a case-by-case basis while scientific grade database facilities are being implemented. If you have any questions, please contact Rob Seaman at the address, email address, or phone number below.

IAU-22 Poster
An IAU-22 poster gave an overview of Nighttime Data Archiving at NOAO, including a flowchart of the steps involved in archiving the images.

Yearly Archive Estimate
During the first full year of operation, Kitt Peak telescopes archived a little over 400 Gigabytes of imaging data. This represents 224,000 images. The Save the bits  FITS header catalog for a year's images approaches a half Gigabyte, the index that cross references the archive tapes and the catalog is about 10 Megabytes/year.

Allowing for the WIYN telescope which will begin regular operation (and archiving) this year, and also compensating for the fact that the IR instruments on Kitt Peak were only archiving their data for half of the first year, a comfortable estimate is one-half Terabyte and 250,000 images/year.

Basic Design
The domes feed a central archive, queuing is provided by a unix print spooler.

Tape Format
The archive tapes are written using exabyte 8505 FITS image extension format, about 4 Gigabytes per tape. A number of images are concatenated from the various telescopes and packed into ~50 Megabyte tape files.

Catalog
The headers are written to the catalog file, a separate index cross references the catalog and tapes.

Multiple Drives
An lpd filter, bitf , writes the tapes and swaps automatically between an arbitrary number of drives.

Swapping Tapes
The bitmon  program provides archive and dome status information, and allows for swapping tapes. The Observing Tech staff are responsible for swapping the archive tapes daily.

Recovering Data
Although not the main purpose, 24 nights' data had been recovered following observer mishaps after the first year. One observer had his data stolen from his office!

System requirements
The archive hardware requirements are minimal - a networked workstation with one or more tape drives. Unix and a Berkeley style line printer daemon are system requirements, although a similar design would be workable on many other platforms. NOAO currently runs the archive under SunOS 4.1.3, and will support Solaris operations in the future using the System V printer daemon.

Software Availability
The current version of the Save the Bits software can be retrieved from ftp://iraf.noao.edu/misc/bits.tar.Z.

For more information
Contact seaman@noao.edu  to discuss using the NOAO archive software for your own observatory projects.

Who are we?


Since loading this page, big number bits have been saved.


IRAF Project
National Optical Astronomy Observatories
950 North Cherry Avenue
Tucson, Arizona 85719
(520) 318-8000
iraf@noao.edu