BOF Session---Future of Astronomical Data Analysis Systems (FADS)

Jan E. Noordam, William T. S. Deich

Netherlands Foundation for Research in Astronomy, P. O. Box 2, 7990 AA Dwingeloo, The Netherlands

Abstract:

1. Introduction

Computer hardware and software technology are changing rapidly, the data volumes increase all the time, and algorithms become more sophisticated. Because of all this, the existing astronomical software systems are facing difficulties in meeting the rising demands and expectations of their users.

Since ADASS has effectively become the world forum for astronomical software, it seemed appropriate to organize a special BOF session to address this fundamental problem. The first goal was to discuss the likely structure of software systems in the future. The second goal was to determine whether any collective action by the astronomical software community is desirable and possible.

2. Preliminary E-mail Debate

In order to stimulate participants to think about the issue beforehand, a preliminary e-mail debate was conducted in the month before ADASS. This proved to be a very useful device. Although there was not much real discussion, there were many thoughtful contributions. This greatly helped to formulate the key question to be addressed at the BOF itself, and to identify some key players to give short (5 min) introductory presentations. The full text of the e-mail contributions can be obtained by sending an e-mail message with the words send digest to fads-request@nfra.nl. An alternative source is Peter Teuben's fads home-page http://www.astro.umd.edu/teuben/fads.

3. Discussion Framework

Because of the almost limitless scope of the subject, a key issue was identified by the organizers, and used as a framework for discussion. The proposition was that it is advantageous to separate the three types of software services (Control, Data Servers, Computation and Standard Services). Rather than being limited to particular implementations of these services in monolithic packages, the user should be able to put together a customized package from his favorite implementations of each service. Of course this means that all services must share communication protocols (but not data structures!).

James Coggins has represented this idea as a triangular `Gaming Table' (see Figure 1), on which the collaboration between services is negotiated.

 
Figure 1: The Gaming Table.
Figure 1: PS 6 Kb

A tentative evolution scenario towards the Gaming Table situation was also part of the discussion framework:

`Agents' will be needed in the wild and wonderful world of the future to help the users choose from the many possibilities, and to to put them together.

Other communities are justifiably envious of our FITS `standard', but further attempts at standardization and interoperability have not been successful so far. Nevertheless, there are compelling reasons to keep trying. The emerging technology of distributed objects may be the key.

4. The BOF Discussion Itself

Even though the conference was officially closed, more than 120 ADASS participants (out of 285) had stayed for this BOF session. First, a number of key players were asked to set the stage by giving 5-minute introductions. (NB: Brian Glendenning could not be present at the BOF, but influenced the discussion framework. As the chief architect of AIPS++, he is already putting some of the Gaming Table ideas in practice within the limits of available technology).

• James Coggins emphasised the importance of distributed services, on which he is doing research. He outlined how the services around the Gaming Table could interact. He gave some examples of configurations in which the software services could be put together: hierarchical, pipeline, monitored pipeline, interactive, etc. Finally, he suggested a list of relevant technical subjects for future ADASS's.

• Doug Tody argued that, in order to deal with many applications, we will still need big (but not necessarily monolithic!) systems. Users who want to program themselves need the choice between a full environment and `mix-and-match' libraries. Of course systems will have to become more modular, with separate UI, data structures, standard services, etc. Distributed objects are much easier to integrate and network than a traditional API. IRAF is already going in the direction sketched by Coggins.

• Eric Mandel. Cooperation is necessary, but has political, social and economic implications. Software systems can work together without sacrificing their independence. Let us start on a small scale. `Think globally, but act locally: That means you!'

• Allen Farris. We need a precisely defined, and commonly accepted, definition of basic measured entities. For instance, the definition of an Astronomical Image should formally specify public functions that supply specific information, not how the data is arranged. Use the CORBA Interface Definition Language as the formal mechanism.

Then followed 45 minutes of open discussion. Except for a few minor lapses, all participants were very cooperative in sticking to the given framework. The following is a summary which does not specify who said what, but hopefully will be recognizable to those who were there.

It came as a bit of a surprise that there was virtually no discussion about what the future would look like. There seems to be a consensus that something like the Gaming Table will emerge soon, roughly following the suggested scenario. There was also little disagreement about technical issues. It is generally assumed that distributed objects will play a key role, and if standards are required they will not be difficult to formulate. The real discussion was about how we should move towards this future as a community. It was felt that software developers will only collaborate (i.e., standardize) if they are pushed by strong forces. An attempt was made to identify such forces:

• The users, who do not like to learn new systems. They are a captive audience, without much real influence.

• The institutes that `sell' astronomical data (and employ software developers). They see software as a means for astronomers to do great things with their data. But they are powerless because managers do not understand software well enough.

• Industry standards (only at lower level).

• The increasing role of digital libraries.

• Decreasing budgets.

• The new generation of software developers.

It was stated that `people willingly standardize only on those things that they do not really care about'. This will be different for different groups: e.g., users will be quite happy about a standard data format, but they will never accept a standard user interface.

Forces that oppose collaboration are the pressure of meeting short-term user needs with the existing packages, and the very real fear of `ossification' by making the wrong choices. Also, many developers are too busy to follow what is happening elsewhere (although that does not prevent them from criticizing). The point was made that collaboration starts bilaterally, usually between two influential institutes. Others will follow if the result is a success. This is consistent with the assertion that good standards evolve out of small-group efforts, to be embraced by the larger community. Can we identify the groups that are making such decisions at this moment?

5. Conclusions

The BOF did not achieve anything tangible in the form of a specific proposal, or even a committee. Nevertheless, it was regarded as a useful `exercise in promoting awareness'. Isolated thinkers now realize that they are not alone in their concerns, and that there is a broad consensus about what the future will look like. They also realize that lots of things are going on, and that their pet concerns may already have been (partially) addressed by others. The participants were released with a number of questions: Does `the Community' want to collaborate? Who are `the Community'? What is the minimum amount of standardization that you yourself would tolerate, and to what level? What are you personally going to do about it in the coming year?

Acknowledgments:

The organizers of the BOF are grateful to the people that contributed to the preliminary e-mail discussion. In chronological order: Choudhary, Downey, Coggins, Harrington, Glendenning, Arnaud, Mandel, Shannon, Hjellming, Glazebrook, Shortridge, Tody, Groner. Of particular help in formulating the basic theme of the discussion were James Coggins, Doug Tody and Brian Glendenning.