Internal Technical Memorandum ITM-1999-03
A New Paradigm for
User Support and Software Tools
Glenn Miller, Anuradha Koratkar, Dan Golombek
April 20, 1999 8:27am - Version 1
ABSTRACT
Our current approach for providing user support, both expert human support and software tools, is
needlessly fragmented. In this paper we outline an integrated approach which will allow us to
maintain and improve the current quality of user support, yet substantially reduce the cost of
development and operations. STScI, a recognized leader in user and observatory support, is in a
unique position to implement the paradigm presented here.
1. Motivation
As we move into an era of multi-mission operations (HST, NGST and others), the STScI is
re-examining its strategy for observatory operations. In this context, our user support paradigm needs to be more effective while continuing to provide outstanding service. Our
basic goals should be:
• At least 80% of the observations provided by observers should not require any manual intervention by our staff.
• Our observers should primarily spend time on scientific investigation, not the
mechanics of using the system.
• Our expert staff should primarily spend time on innovation, not routine matters.
• Our staff should be able to efficiently handle multiple observatory operations.
The above goals can be achieved by making the observer more self-sufficient, i.e. providing them with information and tools that make the astronomical investigative process more
intuitive and observatory operations less cumbersome. This approach encourages “returning the eye to the telescope” to allow the observer to more directly interact with the object
being studied. Further, the goals must be accomplished in the context of pushing the
observatory to achieve cutting-edge science.
1.1 A Possible Future
Imagine an astronomer working a few years from now. She has just received some data
from recent observations and notices an interesting feature. Using the feature as a filter, a
search is conducted of both literature and observations. The search returns a few relevant
papers from the literature as well as data from other observatories. She concludes that fur-
1
Internal Technical Memorandum ITM-1999-03
ther investigation is promising and uses integrated observing tools to develop new
observations to provide more definitive data. These tools work in the same environment as
the data analysis, archive and search tools, and allow generation of simulated data. The
astronomer is assisted by the tool in selecting the best observatory and instrument. All the
relevant information is stored in such a way that it can be directly used in a proposal. After
a short time, the proposal has won time from the observatory. The astronomer refines the
proposal, in the integrated tools environment to generate an observing program. The
observing tools not only help in improving the signal-to-noise, but guide her in avoiding a
particular detector defect that could affect the data quality. It then also suggests observational strategies that generate an efficient program. The observations are executed and data
are returned to the astronomer so that she can verify her original idea.
This sketch of the future is in contrast to today’s environment for using HST and other
observatories. Astronomers are given a variety of tools (Exposure Time Calculators, Phase
I template, RPS2, Starview, STSDAS) which do not seamlessly communicate with each
other, and often have significant learning curves. In addition, the astronomer must be
familiar with hundreds of pages of documentation (Call for Proposals, Phase II Instructions, Instrument Handbooks, Data Handbook,...). As a consequence of this complexity, at
the STScI, Program Coordinators and Contact Scientists must review every observation,
provide extensive user support, and write a large amount of documentation. These shortcomings are exacerbated if an investigation requires observations with more than one
observatory because each observatory has different sets of tools and documentation.
2. The Process and Vision
2
Internal Technical Memorandum ITM-1999-03
An integrated toolset is possible because the different facets of astronomical investigation
(both archival research and acquisition of new observations) are related in a circular flow
of data. Figure 1 illustrates the various relationships possible. The blue arrows indicate the
flow for a fully archive-oriented project. The red arrows indicate the flow for a purely
observational project. A project that involves both the archive and observatory is a combination of the red and blue arrows. An important aspect of the investigative process is that it
is not compartmental and seamlessly flows from one aspect to another.
A fundamental service of an observatory is to support its users in the investigative process
so that the user can move from one facet to another (and back!) as intuitively as possible.
We see two major areas where STScI user support (both software tool and expert human
support) can be improved so that astronomers do not encounter barriers as they move
through the investigative process:
1. The current approach for software development builds tools with artificial
boundaries. We propose that tools should be constructed with the complete
investigative cycle in mind.
2. Expert human support is currently focused on routine activities. The primary
value of human to human interaction is the innovative and extraordinary, not the
routine. It must be easier for expert observatory staff to communicate their
expertise to the user and to the software tools.
These are subtle but dramatic paradigm shifts. We look at each in turn.
2.1 One Toolset From Idea to Publication
Although we provide a number of tools to astronomers, these tools are not well integrated.
Our users are required to learn one set of tools for proposal preparation, another set for
observation development, another set for archive use and yet another set for data analysis.
Although each system strives for good user interface design, the basic paradigm is selfcontained tools with limited interoperability, different presentation styles and different terminology. This decoupling of tools that are similar in function leads to inefficiencies both
for the user and the observatory staff including re-entering information and adjustment to
a new environment.
We can take an alternate view by not looking at the phases in the investigative process, but
rather at what kinds of tools are used:
Visualization tools provide a means to display information and can be used for:
• selection of targets
• determination of observing constraints (orientations, timing, solar avoidance,...)
• display of data
• determination of the usability of archival data
Analysis tools provide means to generate and manipulate data and can be used for the:
• development of proposed observations (signal to noise ratio, exposure times,...)
3
Internal Technical Memorandum ITM-1999-03
• creation of simulated data (tinyTIM PSFs, spectra,...)
• development of an observing plan (simulate instrument and telescope operations)
• reduction and analysis of data
The common thread is that one tool may be used at different stages. For example, a tool
which displays a target on the sky can be used during Phase I proposal development, Phase
II program implementation, archive browsing, data analysis, and publication.
Another important aspect of software tools is documentation, which we take in a broad
sense to include help files, manuals, WWW pages, etc. With few exceptions, at present,
the information contained in documentation is largely unavailable to the user while running the software, and this has important consequences. For example, reference
information (e.g. filter transmission curves) is currently stored in two forms: one for
humans (in the Instrument Handbooks), and one for software (Synphot). These two
expressions of the same data must be maintained and synchronized. As another example,
since documentation is largely unstructured text, software can provide very little assistance to answer user’s questions. This directly increases work load of STScI staff.
Documentation needs to be considered as an integral part of the toolset and structured to allow straightforward access not only to users but also to software and
search tools.
2.2 Simplify the Process of Providing Expert Knowledge to Users and Tools
Direct staff support is currently a manually intensive (and therefore costly) process,
whether it is direct interaction with observers or writing documentation. One cause for this
is the lack of integration described above: since tools are not integrated, a larger than necessary amount of support is needed for routine use. However, there is another, more subtle
(and therefore more problematic) root cause: it is currently cumbersome to put expert
knowledge into a form which is easily used by humans and software. As a result we
continue to rely on human support in areas even when it could be automated. For example,
we currently maintain a Web page of “RPS2 advisories” (http://presto.stsci.edu/public/
rps2advisories/rps2problems.html). Many of the advisories could be eliminated if the GO
version of RPS2 software were updated as fixes became available, as is the case with the
internal version of RPS2. We also maintain documents with recommended observing
strategies for instruments (e.g. dithering, target acquisition), yet this information is not
incorporated into RPS2.
As an observatory, we need to facilitate the transfer of expert information to users and
tools. This not only benefits the user, but makes better use of staff whose expertise is innovation. Adding structure to documentation has already been mentioned as one way to
achieve this. We should also use better techniques for providing help to users. For example, the Microsoft “Office Assistant” provides tips to users and finds documentation based
on a user’s questions. Expert systems technology, such as those being explored in the Scientist’s Expert Assistant is another promising avenue.
4
Internal Technical Memorandum ITM-1999-03
An integrated system such as the one proposed in this document will also lift the artificial
barriers that now exist between the elaboration of the idea, the submission of the proposal
and the development of the observations. That is, we can remove the artificial distinctions
between “Phase I” and “Phase II”. In principle, a Phase II proposal should be a refinement
and update of the information in Phase I. Currently the transition requires re-entering
information and using a different set of tools.
3. The Environment
Given the goals of better integration of tools and better use of expert knowledge, we can
define several high-level requirements. The key theme is transparency: the observing
tools and support should bring the observer as close as possible to the astronomical objects
being studied, with the minimum of distractions due to the mechanics of observing and
analysis.
1. Astronomical orientation - All components of the system (not just the user interface) should use terms and concepts which are meaningful to astronomers: astronomical objects, instruments, detectors, physical constraints, etc.
2. Scientific feedback - The system should provide the information needed to make
scientific trade-offs. The impact of a choice needs to be apparent and shown in a
meaningful way. Users should be given the tools to be self-sufficient.
3. Uniformity - All tools should use consistent terminologies and have a similar
“look and feel”. Such uniformity reduces the learning curve.
4. Interoperability - Tools should be able to share information, alleviating users
from having to manually enter and re-enter data and re-process information.
5. Observatory independence - Tools should be able to be adapted to a different
observatory by reusing common components and customizing only on observatory-specific features.
6. Utility of documentation - Documentation should be an integral part of the
toolset and must be structured to allow efficient access by humans and software
tools.
7. Capturing expert knowledge - Methods which capture human expertise and
make it usable to humans and software should be employed.
8. Open architecture - It should be easy for everyone to enhance tools or include
fixes (the “bazaar” software paradigm). The initial system created and developed
should work like a kernel around which the facility will grow following the
needs of the community.
9. Immediacy - Whenever possible, results of user actions should be available
immediately.
10.Common environment - Observers and observatory staff should use the same
tool environment.
5
Internal Technical Memorandum ITM-1999-03
4. Benefits
We believe that the paradigm outlined in this paper provides benefits to the observatory
and the observer. It allows the observatory to streamline operations and offer improved
support by providing the right tools and information. Observatory staff will spend more
time on innovative applications and exceptional circumstances rather than on routine questions and mundane operations, thus making efficient use of staff. For the observer, this
approach encourages “returning the eye to the telescope”. The observer is able to more
directly interact with the object being studied rather than be distracted with poorly integrated tools and hard to find information.
This latter point is particularly important for the future of astronomy. The concern has
been voiced that observing tools and new modes of observing such as “queue scheduling”
remove the observer from the data and will prevent graduate students from gaining practical experience in the process of scientific observation. The paradigm presented here
fosters the development of good observers by reinforcing the connection between the scientist and the science and eliminating artificial distractions from software, documentation
and processes.
Another long-term benefit of this approach is to encourage collaborative investigation. As
tools become more integrated, and more interoperable, it becomes easier to share information and results with collaborators whether they are across the hall or across the planet.
The barriers between wavelength domains can be lowered by integration, and will enable
multi-wavelength, multi-observatory investigations. This is in contrast to the primitive
state of collaboration available to most astronomers today: email and ftp-ing data files and
programs. Moreover, effective collaboration will also benefit software developers at different observatories by encouraging common approaches and reuse of ideas and tools.
5. Why Now? What’s Different This Time?
There are a number of compelling reasons why now is the right time to design and implement these changes. Several software systems at the STScI are considering next
generation architectures including RPS2, Starview and STSDAS. In addition, the STScI is
trying to develop a practical strategy for supplying tools and user support for NGST at a
much lower cost than HST. The STScI would like to operate other missions in addition to
HST and NGST.
The theme of reuse and common tools is being embraced by other observatories as well, as
demonstrated by interest at the Workshop on Observing Tools (STScI/GSFC Oct 98), the
Proposal Process Workshop (NOAO Aug 98), the SPIE Conference on Observatory Operations to Optimize Scientific Return (Mar 98). STScI’s Spike planning and scheduling
tools are being used by FUSE, AXAF, SIRTF, ESO/VLT and Subaru. (The latter two are
notable since they are ground-based, not space-based missions.) The MAST archive
serves multiple missions. Although the “not invented here” syndrome can inhibit reuse of
tools, it is clear that several major observatories have found that reuse and commonality
have powerful advantages.
6
Internal Technical Memorandum ITM-1999-03
In many cases, the technology available in the late 1980s or early 1990s still limits our
thinking today. However the landscape of information processing has changed dramatically over the past 5 years. Languages such as Java allow tools to run on many different
types of computers and over the WWW. Scripting languages (e.g. Python, Tcl) and request
brokers (e.g. CORBA) make it relatively easy to seamlessly combine tools written in different languages. One side effect of the WWW is that we are accustomed to “tagging” our
documents with HTML (Hypertext Markup Language), either directly or indirectly. The
next generation, XML (Extensible Markup Language) provides a more powerful and flexible scheme which will allow us to offer context-sensitive help and searches which return
focused answers.
In addition to these technological changes, there are also conceptual enablers. Object-oriented programming emphasizes the approach of building tools which model real objects
and processes in contrast to early programming models which had very limited ways to
represent data and relationships. Projects like Linux demonstrate that software can be
developed by many people at different geographic locations. The commercial success of
the Internet and WWW provides reliable networking, browsers and other building blocks
at very low cost.
A clear example of an integrated environment was popularized by the Macintosh operating system and subsequently adopted by the Microsoft Windows operating systems, and
others. Key points are:
• consistent “look and feel” - different programs present data and operations in similar ways, allowing users to leverage knowledge gained in one application into others
• interoperability - different programs can share information transparently, e.g. cutting and pasting text, numbers, tables and illustrations between different programs
A local example of success is the transition from RPSS to RPS2 in Cycle 5. RPS2 was a
great advance over RPSS by providing users with more scientific insight into their observations and better tools for constructing observations. Another example is the
development of WWW-based exposure time calculators which replaced manual calculation from formulae and tables in the Instrument Handbooks. It is time for the next step in
the evolution of our tools and support.
The authors of this memo want to make it clear that we don’t claim to have originated all
the ideas presented here. What we are trying to do is to clarify the common themes that
can enable better scientific investigation and better use of observatory staff. We do believe
that as a recognized leader, the STScI is in a unique position to implement this new
paradigm.
6. A Next Step
The purpose of this memo is to foster a critical examination of the STScI’s software tools
and user support by all those involved including management, scientists, operations, and
development. The result should be a clear statement of our software and user support
7
Internal Technical Memorandum ITM-1999-03
strategy for HST, NGST and other missions. A discussion is particularly timely in light
of the result of the 1999 Science Staff Survey that 53% of the staff responding to the survey feel that “Cheap Ops” will degrade our service to the community. We should consider
the following questions:
1. How do we achieve our goals for user support in a multi-mission environment?
• How do we communicate information to users?
• What is our support infrastructure?
• Under “Cheap Ops”, how can we provide service to HST users that is at least as
good as we provide now?
• What steps can we take to increase communication between STScI groups to realize an integrated environment?
2. How do we refine the paradigm presented here into more specific requirements?
3. What is the impact of emerging technologies on how we provide user service?
• What technologies are promising and how do we evaluate them?
• How do these technologies change our user support paradigm?
4. How do we encourage cooperation between observatories?
• Multi-observatory programs are now common. How do we ease the task of proposing for and scheduling multi-observatory observations?
8