Next: Starbase: A User Centered Database for Astronomy
Previous: WISARD: A Web Interface for Searching Archival Research Data
Table of Contents --- Search ---
PS reprint
E. C. Olson, F. Girouard, A. Hopkins
Center for EUV Astrophysics,
2150 Kittredge Street,
University of California,
Berkeley, CA 94720-5030
Recently NASA has started to emphasize the importance of low-cost mission operations. Software makes up a significant fraction of these costs.
The relative cost of software will rise, unless software components that handle mission-critical operations can be reused. The broader software industry is also trying to determine how to get a handle on software development costs. Many new technologies address the reuse issue, but today, no generally accepted techniques exist for creating reusable software components for large, complex systems.
The Center for EUV Astrophysics, Space Science Laboratory at the University of California, Berkeley, operates the scientific telescopes and instrumentation on board the Extreme Ultraviolet Explorer ( EUVE) satellite (Bowyer, Malina, & Marshall 1988; Haisch, Bowyer, & Malina 1993). Like earlier missions, CEA developed the ground system for acquiring, monitoring, and archiving the EUVE science data. This system was specifically designed to support the EUVE satellite. The system was designed in an ``end-to-end'' system concept, so effectively it has no software components that can be reused in other missions.
More recently, the Ames Research Center (ARC) funded CEA to collaborate with Boston University (BU) and AeroAstro in building the TERRIERS ground system (Vickers et al. 1993).
CEA developed a three-point strategy for the TERRIERS mission. First, we would ensure that the primary scientific mission of TERRIERS would be met by meeting BU's aggressive system development timeline. Second, we would provide the interfaces to ARC so that they can demonstrate the effectiveness of their analysis systems. Third, we would design and develop a multi-mission archive software component. Although the TERRIERS ground system is still being developed, it appears that the first two goals will be met. This paper discusses the difficulties and issues discovered in addressing the third goal.
The first question we had to address was what should be the logical division of a multi-mission archive software component. The TERRIERS ground system certainly included an archival capability while ARC required access to a TERRIERS ``data server.'' But what part of a TERRIERS archive could reasonably be used on other missions? We decided that a reasonable evaluation criterion would be to identify an archival component that could be applied to the EUVE mission as well. In addition to the general-purpose archive requirements, we felt that a mission archive had to have specific capabilities for handling satellite telemetry. Telemetry handling includes receiving the telemetry into the archive, indexing, and decomposing the telemetry for application software.
Based on our experience with EUVE, we knew that the primary difficulties in creating a multi-mission archive would be in the decommutation of the telemetry. The bulk of our design analysis focused on how to describe telemetry formats. In practice, it turned out that EUVE and TERRIERS telemetry formats were fundamentally different. The EUVE telemetry is structured around fixed-length records called ``frames.'' These frames are telemetered from the satellite to the ground station with quality flags. The TERRIERS telemetry is packet-oriented: the satellite and the ground station use a communications protocol to ensure that the packets are properly transferred, not unlike a ground network. Whereas an EUVE frame has a 128 minor variations in the telemetry format, TERRIERS has literally hundreds of widely varying packets. Further, sequences of packets are variable in size from tens of bytes to megabytes. This huge variation in format has greatly increased the design complexity of a multi-mission archive.
A closer inspection of the EUVE telemetry raised further issues. In addition to the primary EUV science instruments, EUVE contains two secondary missions. One of these, a Global Positioning Satellite (GPS) experiment, shares the EUVE telemetry stream. However, since the GPS is a completely independent system, its telemetry is not synchronized with the platform---effectively making it an independent sub-stream of telemetry. Also, the EUV science data, time-tagged photon events, have a stream-oriented characteristic. However, the photon events are synchronized with the EUVE telemetry stream and do not form an independent sub-stream.
The basic steps involved in decommutating telemetry are extraction, assembly, and conversion. In general, in order to properly assemble and convert values, a notion of the current state of the telemetry must be maintained. For example, when converting an electric current monitor you must know whether the primary or secondary relays are currently in use. For EUVE this information is telemetered on a periodic basis, whereas for TERRIERS it is only available from whenever the required packet of information was last telemetered. Maintaining complete state information is much more difficult for TERRIERS than EUVE. On EUVE, one simply backs up 128 frames to rebuild the state. On TERRIERS, there is no time limit on how long ago a particular monitor was telemetered. This ability to recover the most recent state of the telemetry is critically important for ground systems operations---especially when ground systems are recovering from a system crash. The state information is also important for the archival use of data. For example, for engineering or science analysis, someone might want to select data only when the backup relays were enabled. In practice, careful selection of appropriate state information on which to build an archival index is required. The current state information is used in a sense as an indirect index back into the historical telemetry archive. For EUVE, the state information can be easily maintained in the memory of contemporary computer systems. However, for TERRIERS, where values can be multi-megabyte images, maintaining these values in memory can be problematic.
In order to provide a common model for accessing telemetry, we determined that frames could easily be handled as packets, but that the reverse was not feasible. The TERRIERS documentation indicated that some packets were of variable length, but the packets themselves only indicated a sequence number without indicating how many total packets formed the sequence. Later, the point was clarified and packets of a particular type were determined to always have a specific length. However, in general the problem of how to handle variable length data is open ended.
Another problem induced by packet-oriented streams is how to create reasonably sized indices of the telemetry. In practice it is not feasible to record information in an index about each individual packet of information. For EUVE, we record indexing information about intervals of telemetry. For TERRIERS, we will maintain two types of indices. The first will contain detailed information about individual packets to be used as the packets are received from the satellite. The second will contain information about which packets of information were collected during a time interval. From time to time, the detailed index information will be coalesced into the second index after an appropriate amount of time has elapsed. The time will be determined by the likelihood of having a packet delivered out of order.
Not surprisingly, we determined that the primary index for telemetry data should be required to be a value closely related to time. It is important that the indexing value be basically linear with time, but it does not have to be exactly linear. It must be monotonic and should be in integral values. Building telemetry indices for multiple missions is quite feasible.
Preliminary work has been done on providing a specification file for describing telemetry formats---both frame-oriented and packet-oriented telemetry. The initial effort focused on a table-driven, descriptive specification. This has proven difficult because telemetry has self-labeling tags for marking variable-length records to be decommutated. This suggests that an algorithmic specification file may be necessary.
To describe the telemetry format we have developed the following nomenclature. First, the archive accepts a stream of telemetry comprising ``parcels'' of data. These parcels may contain ``sub-streams'' of telemetry, like the GPS data in the EUVE telemetry stream or other parcels. Sub-streams are used for truly independent telemetry streams simply encapsulated in the parent ``stream.'' Parcels reflect the logical structure of the telemetry. ``Atoms'' are the smallest unit of useful information of a certain type. Atoms are created by assembling and extracting information contained in parcels called segments. ``Segments'' are parts of atoms contained in a single parcel that has associated quality information. Atoms consist of one or more segments. Quality information is associated with individual segments. An atom is assembled from good quality segments. To determine equality, atoms need to have a notion of the ``first'' and ``last'' segments that form the atom. Each atom maintains ``pins,'' which are set on the first segment, and each one is knocked down when a good segment is observed. On the last segment, the pins are checked to ensure that they have all been knocked down and that no data dropouts occurred. ``Items'' are converted from one or more atoms. Conversion may require table lookups, nonlinear equations, etc.
The quality of an item is determined by the quality information associated with its atoms. The telemetry specification must describe how to identify parcels, segments, atoms, and items in the telemetry.
Providing a multi-mission archive that handles packet-oriented and frame-oriented telemetry is, at best, a difficult problem and may not be feasible. However, providing a frame-oriented multi-mission archive, while still a difficult problem, might have feasible solutions. CEA is continuing to evaluate the possibility of creating a reusable archive software component. Certain parts of the archive, for example indexing, seem to be quite feasible while other areas need additional analysis before determining an appropriate implementation. Some of these problems may be solved in part by object-oriented database systems. In particular the Scalable Heterogeneous Object Repository (SHORE; Carey et al. 1994) and Paradise systems are currently under development.
This work was supported by NASA contracts NCC2-838 and NAS5-29298. CEA is a division of the Space Sciences Laboratory at UC Berkeley.
Bowyer, C. S., Malina, R. F., & Marshall, H. L. 1988, JBIS, 41, 357
Haisch, B., Bowyer, C. S., & Malina, R. F. 1993, JBIS, 46(9), 330
Vickers, J. S., Cotton, D. M., Cook, T. A., & Chakabarti, S. 1993, Opt. Eng., 32, 3126