GLAST LAT Project
Gamma-ray Large
Area Space
Telescope
ISOC CDR, 4 August 2004
GLAST Large Area Telescope:
Instrument Science Operations Center
CDR
Section 3.4
Science Data Products and
Science Analysis Software
Richard Dubois
SAS System Manager
Document: LAT-PR-04500
Section 3.4
1
GLAST LAT Project
ISOC CDR, 4 August 2004
Outline
• Introduction to SAS Scope and Requirements
• Overall Test Plan
• Data Challenges
• DC1 Summary
Document: LAT-PR-04500
Section 3.4
2
GLAST LAT Project
ISOC CDR, 4 August 2004
Science Analysis Software Overview
• Data Pipelines
– Prompt processing of Level 0 data through to Level 1 event
quantities
– Providing near real time monitoring information to the ISOC
– Transient sources
– Monitoring and updating instrument calibrations
– Reprocessing of instrument data
• Performing bulk production of Monte Carlo simulations
• Higher Level Analysis
– Creating high level science products from Level 1
• Point source catalogue
– Providing access to event and photon data for higher level data
analysis
• Interfacing with other sites (sharing data and analysis tool
development)
– Mirror PI team site(s)
– SSC
• Supporting Engineering Model and Calibration tests
• Supporting the collaboration for the use of the tools
Document: LAT-PR-04500
Section 3.4
3
GLAST LAT Project
ISOC CDR, 4 August 2004
Level III Requirements Summary
Ref: LAT-SS-00020
Function
Flight Ground Processing
Instrument Design Support
High Level Tools
Mission Support
Document: LAT-PR-04500
Requirement
perform prompt processing
from Level 0 through Level 1
provide near-real time
monitoring to IOC
maintain state and
performance tracking
facilitate monitoring and
updating of iinstrument
calibrations
archive all data passing
through
Create simulation tool, based
on instrument geometry, that
reproduces the interactions of
photons and background
Create physics model of
expected photons and
backgrounds incident upon
the instrument
Create algorithms to interpret
the data from the instrument
to identify the interaction and
estimate photon direction and
energy
Create algorithms to generate
calibration constants for the
subsystem components
Interface with the SSC and PI
mirror sites, sharing selected
data and algorithms
Create High-Level Science
products. Development of
analysis tools
Support the Software system
for the life of the mission
Expected Performance (if
applicable)
keep pace with up to 10 GB
Level 0 per day and deliver to
SSC within 24 hrs
within 6 hrs
Verification
demonstration
demonstration
demonstration
demonstration
> 50 TB on disk and tape
backup
demonstration
system test comparison to
balloon flight and
existing data
system test comparison to
balloon flight and
existing data
system test comparison to
engineering model
tests
system test - in
conjunction with
engineering model
tests
demonstration
.
demonstration
demonstration
Section 3.4
4
GLAST LAT Project
ISOC CDR, 4 August 2004
RD
LAT sim/recon
System Architect
ScienceTools
Simulation
Recon
Event Interpretation
ACD
Architects
Infrastructure
Analysis Tools
Code Distribution
Release Manager
ACD
Likelihood
System Tests
Data Server
CAL
Pulsars
Issues Tracker
Documentation
TKR
GRBs
SLAC Linux
environment
SLAC Windows
Environment
GEANT4
Obs Sim
Sundry Utilities
Caliibrations
Event Display
Code Management
Tool
User Interface
Release Manager
I/O
CAL
TKR
System Tests
Release Manager
Flight Int
Support
I/O
Document: LAT-PR-04500
There is a name for each box, except Infrastructure lead.
Asking for SLAC to fill this position. Making do for now.
Section 3.4
5
GLAST LAT Project
ISOC CDR, 4 August 2004
SAS in the Ground System
Pipeline (Gino) is robotic
backbone of ISOC process
handling –
Performs L1 & L2
processing
Level 0 & HSK
Telemetry
Mission Operations Center
Instrument Operations Center
(IOC)
Nominal
Commanding
Science
Support
Center
Commanding
Data Processing Facility
Housekeeping
Data
Instrument Monitoring
Gino server and database
can handle
multiple arbitrary
sequences of tasks:
L1 pipeline; reprocessing;
MC; ….
Contingency
Commanding
Science Data
Production
Level 1 Products
Instrument
Performance
Evaluation Group
Operation Communication & Interfaces
Flight Software
Support Group
Science Analysis
Software Group
LAT User
Community
LAT Operations Steering Committee
(Suggested)
Science Analysis
Burst Alert
Messages via GCN
LAT TEAM
Keep everything on disk and archive to tape
Document: LAT-PR-04500
Section 3.4
6
GLAST LAT Project
ISOC CDR, 4 August 2004
Pipeline Spec
•
•
•
•
Function
– The Pipeline has five major functions
• automatically process Level 0 data through reconstruction (Level 1)
• provide near real-time feedback to IOC
• facilitate the verification and generation of new calibration constants
• produce bulk Monte Carlo simulations
• backup all data that passes through
Must be able to perform these functions in parallel
Fully configurable, parallel task chains allow great flexibility for use online as
well as offline
– Will test the online capabilities during Flight Integration
The pipeline database and server, and diagnostics database have been
specified (will need revision after prototype experience!)
– database: LAT-TD-00553
– server: LAT-TD-00773
– diagnostics: LAT-TD-00876
Document: LAT-PR-04500
Section 3.4
7
GLAST LAT Project
ISOC CDR, 4 August 2004
Pipeline in Pictures
State machine +
complete processing
record
Configurable
linked list of
applications
to run
Document: LAT-PR-04500
Expandable and
configurable set of
processing nodes
Section 3.4
8
GLAST LAT Project
ISOC CDR, 4 August 2004
Overall Test Plan
•
Combination of Engineering Model tests, Data Challenges and LAT
Integration Support
•
EM tests
– EM1 demonstrated ability to simulate/reconstruct real data from
single (non-standard) tower
• All within standard code framework/tools
• Data analyzed with SAS tools
Data Challenges
– End to end tests of sky simulation through astro analysis
– Exercise pipeline
•
•
LAT Flight Integration
– Combine tools from EM & DC applications
– Sim/recon/analysis & pipeline processing and record keeping
Document: LAT-PR-04500
Section 3.4
9
GLAST LAT Project
ISOC CDR, 4 August 2004
Processing Flow
Data Pipeline
Level 0
Sim
Raw Data
Recon: interpret LAT readout
and estimate directions and
energies; flag background
Sim: full modeling of e/g/p
interactions and readout in
the LAT
Recon
Level 1
Science Tools
Level 2
Document: LAT-PR-04500
Section 3.4
10
GLAST LAT Project
ISOC CDR, 4 August 2004
Level 1 Sim/Recon Chain
3 GeV g
Source
Fluxes
Particle
Transport
Real Data
“Raw”
Data
Recon
Geometry
Background
Rejection
Particle ID
Level 1
Document: LAT-PR-04500
Section 3.4
11
GLAST LAT Project
ISOC CDR, 4 August 2004
Sim/Recon Toolkit
Package
Description
Provider
Status
ACD, CAL, TKR
Recon
Data
reconstruction
LAT
90% done
In use
ACD, CAL, TKR
Sim
Instrument sim
LAT
95% done
In use
GEANT4
Particle transport
sim
G4 worldwide
collaboration
In use
xml
Parameters
World standard
In use
Root
C++ object I/O
HEP standard
In use
Gaudi
Code skeleton
CERN standard
In use
doxygen
Code doc tool
World standard
In use
Visual C++/gnu
Development envs
World standards
In use
CMT
Code mgmt tool
HEP standard
In use
cvsweb
cvs web viewer
World standard
In use
cvs
File version mgmt
World standard
In use
Document: LAT-PR-04500
Section 3.4
12
GLAST LAT Project
ISOC CDR, 4 August 2004
Software Development Approach
• Enable distributed development via cvs repository
• Extensive use of electronic communications
– Web conferencing (VRVS), Instant Messaging (icq), Wikis
• CMT tool permits equal development on Windows and Linux
– Coupled with cvs gives machinery enables config control
– Superior development environment on Windows; compute
cycles on linux
• documentation and coding reviews enforce coding rules
• JIRA issue tracking system
• “Continuous integration”
– Eliminate surprises for incoming code releases
– Build code every night; alert owners to failures in build or
running of unit tests. Results tracked in database.
– Developing comprehensive system tests in multiple source
configurations. Track results in database; web viewable.
Document: LAT-PR-04500
Section 3.4
13
GLAST LAT Project
Document: LAT-PR-04500
ISOC CDR, 4 August 2004
Using Jira Issue Tracker
Section 3.4
14
GLAST LAT Project
ISOC CDR, 4 August 2004
Nightly Builds
Performing builds
for Science Tools
now also
Past release
Display created from database query
Release in progress
Build status
Unit test status
Future release
Document: LAT-PR-04500
Section 3.4
15
GLAST LAT Project
ISOC CDR, 4 August 2004
System Tests
Comparison of
current to previous
release.
Document: LAT-PR-04500
Section 3.4
16
GLAST LAT Project
ISOC CDR, 4 August 2004
Main Science Tools
Package
Likelihood
Level 1 database
access
Description
First Use
Workhorse model fitting for detection &
characterization of cosmic gamma-ray sources
DC1
Extracts desired event data
DC1
Exposure calculation
Uses IRFs, pointing, livetime etc. for deriving
calibrated source fluxes
DC1
Source identification
Identifies gamma-ray sources with cataloged
counterparts at other wavelengths
DC2
GRB analysis
Temporal and spectral analyses of burst profiles
DC1
Pulsar analysis
Phase folding & period searching of gamma-ray
pulsars and candidates
DC2
High level simulation of observations of the gamma-ray
sky with the LAT
DC1
Observation
simulator
Document: LAT-PR-04500
Section 3.4
17
GLAST LAT Project
ISOC CDR, 4 August 2004
Science Tools Toolkit
Package
Description
Provider
Status
PIL, PIL++
IRAF parameter
access
HEASARC
In use
cfitsio, CCFits
FITS file
manipulation
HEASARC
In use
XSPEC, Sherpa
For GRB spectral
modeling
HEA standards
Under consideration
Root
gui etc
HEP standard
Under consideration
python
Scripting
World standard
Under consideration
doxygen
Code doc tool
World standard
In use
Visual C++/gnu
Development envs
World standards
In use
CMT
Code mgmt tool
HEP standard
In use
cvsweb
Cvs web viewer
World standard
In use
cvs
File version mgmt
World standard
In use
Document: LAT-PR-04500
Section 3.4
18
GLAST LAT Project
ISOC CDR, 4 August 2004
Data Products
DC1
DC1
DC1
DC2
DC2
DC2
DC2/3
?
under
discussion
Document: LAT-PR-04500
Section 3.4
19
GLAST LAT Project
ISOC CDR, 4 August 2004
Purposes of the Data Challenges
S.Ritz
• “End-to-end” testing of analysis software.
• Familiarize team with data content, formats, tools and realistic
details of analysis issues (both instrumental and astrophysical).
• If needed, develop additional methods for analyzing LAT data,
encouraging alternatives that fit within the existing framework.
• Provide feedback to the SAS group on what works and what is
missing from the data formats and tools.
• Uncover systematic effects in reconstruction and analysis.
Support readiness by launch time to do all first-year science.
Document: LAT-PR-04500
Section 3.4
20
GLAST LAT Project
ISOC CDR, 4 August 2004
SAS Checklist
Detailed Simulation
Event Reconstruction
Instrument Calibration
ACD
CAL
TKR
Processing Pipeline
MC
I&T
Event Classification
Re-processing
User Support
High Level Analysis
Code distribution
High Level Instr Diags
Data Distribution
Quicklook
GRBs,
AGN,
Pulsars
Catalogue,
Diffuse
Institutional Mirrors
SSC
LAT Mirrors
DC1
DC2
DC3
Document: LAT-PR-04500
I&T EM
I&T Flight
Section 3.4
21
GLAST LAT Project
ISOC CDR, 4 August 2004
Data Challenge Planning Approach
S.Ritz
• Walk before running: design a progression of studies.
• DC1. Modest goals. Contains most essential features of a data
challenge. Original plan:
•
•
•
•
•
1 simulated day all-sky survey simulation, including backgrounds
find flaring AGN, a GRB
recognize simple hardware problem(s)
a few physics surprises
exercise:
– exposure, orbit/attitude handling, data processing pipeline components,
analysis tools
• DC2, start end of CY04. More ambitious goals. Encourage further
development, based on lessons from DC1. One simulated month.
• DC3. Support for flight science production.
Document: LAT-PR-04500
Section 3.4
22
GLAST LAT Project
ISOC CDR, 4 August 2004
Data Challenge 1 Closeout
12-13 Feb 2004
http://www-glast.slac.stanford.edu/software/Workshops/Feb04DC1CloseOut/coverpage.htm
Document: LAT-PR-04500
Section 3.4
23
GLAST LAT Project
ISOC CDR, 4 August 2004
DC1 Components
•
Focal point for many threads
– Orbit, rocking, celestial coordinates, pointing history
– Plausible model of the sky
– Background rejection and event selection
– Instrument Response Functions
– Data formats for input to high level tools
– First look at major science tools – Likelihood, Observation Simulator
– Generation of datasets
– Populate and exercise data servers at SSC & LAT
– Code distribution on windows and
•
Involve new users
•
Teamwork!
Document: LAT-PR-04500
Section 3.4
24
GLAST LAT Project
S.Digel and P.Nolan
ISOC CDR, 4 August 2004
Science Tools in DC1 DC3
The big picture:
Details are changing, but
still basically right
Level 0.5
Level 1 (D1)
Pointing/livetime
history (D2)
Alternative source
for testing highlevel analysis
Event
display (UI1)
Pulsar
ephem. (D4)
Pulsar period
search (A4)
Ephemeris
extract (U11)
Arrival time
correction (U10)
Pulsar phase
assign (U12)
Standard Analysis Environment
Data extract
(U1)
Pt.ing/livetime
extractor (U3)
Alternative for
making additional
cuts on alreadyretrieved event
data
Observation
simulator (O2)
Data subselection (U2)
Pt.ing/livetime
simulator (O1)
Pt.ing/livetime
extractor (U3)
Exposure
calc. (U4)
Document: LAT-PR-04500
Source model
def. tool (U7)
Catalog
Access (U9)
Likelihood (A1)
Astron.
catalogs (D6)
Src. ID (A2)
Interstellar em.
model (U5)
Map gen
(U6)
IRFs (D3)
IRF visualization (U8)
GRB LAT DRM
gen. (U14)
1
This tool also performs periodicity tests and
the results can be used to refine ephemerides
2 These tools can also take as input binned data
from other instruments, e.g., GBM; the
corresponding DRMs must also be available.
LAT Point source
catalog (D5)
Pulsar
profiles (A3)1
GRB event
binning (A5)
GRB spectral
analysis (A8)2
GRB rebinning
(A6)2
User Interface aspects of the standard
analysis environment, such as
GRB unbinned
Image/plot display (UI2), Command
spectral analysis (A9) line interface & scripting (UI4), and
GUI & Web access (UI5) are not
GRB spectral-temporal shown explicitly.
modeling (A10)
GRB visualization (U13)
GRB temporal
analysis (A7)2
Section 3.4
14 Sept 2002
25
GLAST LAT Project
S.Digel and P.Nolan
•
ISOC CDR, 4 August 2004
Science Tools in DC1
All components are still prototypes
Level 1 (D1)
Pointing/livetime
history (D2)
Alternative source
for testing highlevel analysis
Data extract
(U1)
Pt.ing/livetime
extractor (U3)
Alternative for
making additional
cuts on alreadyretrieved event
data
Observation
simulator (O2)
Data subselection (U2)
Pt.ing/livetime
simulator (O1)
Pt.ing/livetime
extractor (U3)
Source model
def. tool (U7)
Exposure
calc. (U4)
Likelihood (A1)
Interstellar em.
model (U5)
Map gen
(U6)
IRFs (D3)
GRB LAT DRM
gen. (U14)
The DC1 functionality is
Data extraction
Limited visualization
Model definition
Model fitting
Observation simulation
GRB spectral
analysis (A8)2
GRB event
binning (A5)
Document: LAT-PR-04500
Section 3.4
26
GLAST LAT Project
ISOC CDR, 4 August 2004
The data
T.Burnett
Document: LAT-PR-04500
on to
individual
components!
Section 3.4
27
GLAST LAT Project
ISOC CDR, 4 August 2004
http://www-glast.slac.stanford.edu/software/DataChallenges/DC1/DC1_StatusAndInfo.htm
Documentation
• User’s Guide
• Data Description
• Likelihood Tutorial
DC-1 Discussion List
Analysis Code
download sites
Wiki page for
sharing results!
Document: LAT-PR-04500
Section 3.4
28
GLAST LAT Project
ISOC CDR, 4 August 2004
Lessons Learned
•
Analysis Issues
– Astrophysical data analysis
– Software usage and
reliability
– Documentation
– Data access and data server
usage
– UI stuff
– Software installation and
release
– Software infrastructure &
framework
– Communication and Time
frame
•
Infrastructure Issues
– SciTools did not run on
windows at the last minute
– We discovered problems
with sources and ACD
ribbons late
– Manual handling of the
processing
– No checking of file integrity
– Large failure rate in batch
jobs (~10%)
– Tools are not checking
inputs much
– Code distribution scripts
were written manually
Issues have been addressed
Document: LAT-PR-04500
Section 3.4
29
GLAST LAT Project
ISOC CDR, 4 August 2004
Strawperson Updated Plan for DC2
• DC2, based on lessons from DC1
–
–
–
–
–
S.Ritz
1 simulated month of all-sky survey gammas (backgrounds: see next
slide)
key sky addition: source variability
• AGN variability, including bright flares, quiescent periods
• expand burst variety (and include GBM? see later slides)
• pulsars, including Gemingas, w/ orbit position effects.
more realistic all-sky attitude profile
background rate varies with orbit position
more physics surprises, and add nominal hardware problems (and
misalignments?), add deadtime effects and corrections
– Analysis Goals:
• produce toy 1-month catalog and transient releases
• detailed point source sensitivity and localization studies
• first systematic pulsar searches (timing!); detailed diffuse analyses
• recognize simple hardware problems (connect with ISOC/SOG)
– benchmark:
• processing
Document:
LAT-PR-04500 times, data volume, data transfers.
Section 3.4
30
GLAST LAT Project
ISOC CDR, 4 August 2004
SAS Support of Flight Integration
•
Expected Deliverables
– Automated processing pipeline with bookkeeping
• Gino: in test now
• Revised draft pipeline spec at
http://confluence.slac.stanford.edu/display/pipe/Functional+Spec+for+Processing+Pipeline
–
Stable Simulation/Reconstruction package
• Frozen as of 8/4 (more versions will be necessary before actual use)
• Special system tests for surface muon data
–
Calibration algorithms for TKR, CAL – due by 1 Sept
• TKR hot/dead strips
– Later: ToT gains, inter-tower alignment
• CAL peds, gains, light tapers
–
Calibration infrastructure for input to recon - rdbGui
• Next step is allowing for binary format of parameters (800k ToT gains!)
–
Event display – FRED
• Ready
–
Will inaugurate CCB process for code used in Pipeline
–
Plan to extend System Tests scheme to deliver the High Level Diagnostics
Document: LAT-PR-04500
Section 3.4
31
GLAST LAT Project
ISOC CDR, 4 August 2004
Pipeline View
Document: LAT-PR-04500
Section 3.4
32
GLAST LAT Project
ISOC CDR, 4 August 2004
Example of FRED
Document: LAT-PR-04500
Section 3.4
33
GLAST LAT Project
ISOC CDR, 4 August 2004
Summary
• We believe that EMs, DCs and Flight Integration will leave us
ready for flight
• EM1 worked with our tools
• DC1 worked well, showing very good capabilities from sky
modeling through astronomical analysis
– Plenty of work still to so, but reasonably understood
– Will be demonstrated in DC2, 3 and LAT Integration, 16tower cosmic ray tests and the beam test prior to launch
• LAT Flight Integration prep in full swing now
• DC2 within a year (being negotiated)
Document: LAT-PR-04500
Section 3.4
34