Next: Remote Observations in the Near Infrared
Previous: Remote Eavesdropping via the World Wide Web
Table of Contents --- Search ---
PS reprint
Séverin Gaudet
CADC/DAO/HIA/NRC, 5071 West Saanich Rd., Victoria BC, Canada,
V8X 4M6
The Gemini Project is an international partnership to build two 8-meter telescopes, one on Mauna Kea, Hawaii and one on Cerro Pachon, Chile. Gemini will support remote, queue, and classical observing modes. The data will be archived. Instrument and software development is distributed among different groups in the member countries.
The Gemini Software Controls Group (McGonegal 1996) is responsible for the software control systems. Included in this group are the Observatory Control System (OCS) (Gillies 1996), the Telescope Control System (TCS), the Instrument Control Systems (ICSs) and the Data Handling System (DHS).
Given that the Gemini instruments are being built by various groups, it was important to design a mechanism to avoid duplication of software and to prevent the hybridization of interfaces between instruments and data storage and display. The philosophy guiding the design of the DHS was to have centrally managed data storage, data processing and data display where the knowledge of the data is centralized. The goals of the DHS are:
The DHS will provide a Data Storage server to store the data generated by the scientific instruments. The server will receive pixel and header data from one or more sources using SDS data format (Bailey 1993) and the IMP messaging system (Shortridge, Farrell, & Baily 1993). It will then combine the data into a standard FITS format file. The server will also parse the header information and using a data dictionary, will verify the consistency of the header. The header data will then be stored in DHS database as a record of the observation. The server will be entirely data-driven and will not be sequenced by the OCS.
The DHS will provide a Quick Look system to display data from instruments to allow observers to judge the quality of the data. The display will be synchronized with data acquisition and will always show the latest frame. Simple processing operations will be supported such as applying arithmetically a specified calibration frame, calculation of simple statistics, color table selection, etc. The DHS will allow for more than one display per data source as well as remote displays for eavesdropping.
To allow the operator and the observer to calibrate the telescope and instruments, the DHS will provide a command-driven Synchronous Data Processing server. This will be capable of executing scripts provided by other Gemini systems on data within the DHS and to return a result. The script may or may not require user interaction. This will have a higher priority over other data processing tasks.
Once the data for an observation has been received and formatted by the Data Storage server, it is automatically entered into the On-line Data Processing queue. The purpose of this component is to allow the observer to make qualitative assessments of the data by removing telescope and instrument signatures. The goal is not to produce publication quality data. The system will run automatically using rules to identify relevant input files and applying recipes to control the steps executed in the reduction. The observer however will be able to control the process using a console to modify the queue order, to enable/disable steps in the recipes, to use alternate recipes and to override the rules governing the choice of input files. The on-line data processing system will use an existing data reduction package (IRAF).
The observers will be able to use a favored data reduction system off-line from the the DHS data reduction queue. The DHS will make accessible on magnetic disk recent data files to the users of those systems.
The Data Transport component of the DHS is responsible for providing users with of copy of their data including the observing log, all raw data, all calibration data and all processed data belonging to a particular science program. In addition, all data designated for archiving will be written to permanent media as part of this component.
The DHS is also responsible for collecting various types of logging information. These include history logs for other Gemini systems, observing logs and engineering logs.
The DHS will provide access to external (non-Gemini) databases such as guide star catalogues, digital sky surveys, spectral and flux standard catalogues, etc. This will be used by the OCS to support observation preparation and by the DHS itself to support data reduction. The baseline for access is ANSI SQL.
A CADC-style archive center for public access to Gemini data is not part of the DHS. Although writing data to a permanent or archive medium is included in the DHS, the decision to create such a center will be made at some time in 1996. It is clear however that the data pipeline in large telescope projects no longer stops at the investigator but continues on to the archive user. Gemini systems will support queue-scheduled observing where the investigator is not the observer. The data necessary to support that investigator is the same as that required by an archive researcher. The DHS is important to the archive-ability of the data by ensuring the completeness and consistency of data being written to the permanent medium and to the DHS database.
The first prototype of the DHS will be delivered in mid-1996. The final system will be delivered in the latter part of 1997.
Many people have participated in the development of the DHS concepts and design---the Gemini Software Design Team who proposed the initial high-level design, the NOAO IRAF group for their input on the data processing, and the members of the CADC.
Gillies, K. 1996, this volume
McGonegal, R. 1996, this volume
Wampler, S., Gillies, K., Beard, S., Johnson, A., Laing, R., Mayer, C., McGehee, P., McGonegal, R., & Wallace, P. 1994, Gemini document SPE-C-G0037, Software Design Document
Shortridge, F., Farrell, T., & Bailey, J. 1993, in Astronomical Data Analysis Software and Systems II, ASP Conf. Ser., Vol. 52, eds. R. J. Hanisch, R. J. V. Brissenden, & J. Barnes (San Francisco, ASP), p. 295