GLAST LAT Project
Gamma-ray Large
Area Space
Telescope
CDR/CD-3 Review May 12-16, 2003
GLAST Large Area Telescope:
Science Analysis Software
Richard Dubois
Stanford Linear Accelerator Center
richard@slac.stanford.edu
Document: LAT-PR-01967
Section 6 SAS Subsystem
1
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Outline
•
Introduction to SAS
•
SAS Mission as defined by Level 3 Requirements and Milestones
•
•
Instrument Simulation and Event Reconstruction
Support of Engineering Tests
•
•
•
Level 1 Pipeline progress
High Level Science Tools development progress
Mission Ground Systems End-to-end testing
•
•
•
Preparation for LAT Ground System Peer Review and CDR
Cost and Schedule
Concerns
•
Summary
Document: LAT-PR-01967
Section 6 SAS Subsystem
2
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Level III Requirements Summary
Ref: LAT-SS-00020
Function
Flight Ground Processing
Instrument Design Support
High Level Tools
Mission Support
Document: LAT-PR-01967
Requirement
perform prompt processing
from Level 0 through Level 1
provide near-real time
monitoring to IOC
maintain state and
performance tracking
facilitate monitoring and
updating of iinstrument
calibrations
archive all data passing
through
Create simulation tool, based
on instrument geometry, that
reproduces the interactions of
photons and background
Create physics model of
expected photons and
backgrounds incident upon
the instrument
Create algorithms to interpret
the data from the instrument
to identify the interaction and
estimate photon direction and
energy
Create algorithms to generate
calibration constants for the
subsystem components
Interface with the SSC and PI
mirror sites, sharing selected
data and algorithms
Create High-Level Science
products. Development of
analysis tools
Support the Software system
for the life of the mission
Expected Performance (if
applicable)
keep pace with up to 10 GB
Level 0 per day and deliver to
SSC within 24 hrs
within 6 hrs
Verification
demonstration
demonstration
demonstration
demonstration
> 50 TB on disk and tape
backup
demonstration
system test comparison to
balloon flight and
existing data
system test comparison to
balloon flight and
existing data
system test comparison to
engineering model
tests
system test - in
conjunction with
engineering model
tests
demonstration
.
demonstration
demonstration
Section 6 SAS Subsystem
3
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Science Analysis Software Overview
• Data Pipeline
– Prompt processing of Level 0 data through to Level 1 event quantities
– Providing near real time monitoring information to the IOC
– Monitoring and updating instrument calibrations
– Reprocessing of instrument data
• Performing bulk production of Monte Carlo simulations
• Higher Level Analysis
– Creating high level science products from Level 1 for the PI team
• Transient sources
• Point source catalogue
– Providing access to event and photon data for higher level data analysis
• Interfacing with other sites (sharing data and analysis tool
development)
– Mirror PI team site(s)
– SSC
• Supporting Engineering Model and Calibration tests
• Supporting the collaboration for the use of the tools
Document: LAT-PR-01967
Section 6 SAS Subsystem
4
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
SAS Organization
Instrument Project Office
R.Dubois
Manager
4.1.D
SLAC
T.Burnett
Sim/Recon
4.1.D.1
UW
T.Burnett
Architect
UW
E.do Couto e Silva
Calibrations
4.1.D.6
SLAC
S.Digel
Science Tools
4.1.D.4
Stanford
H.Kelly
Analysis Tools
4.1.D.2
GSFC
H.Kelly
ACD
4.1.D.1.5
GSFC
R.Schaefer (SSC)
K.Young (SLAC)
Databases
M.Strickman
CAL
4.1.D.1.6
NRL, France
J.Chiang (UMBC)
T.Burnett
Observation Simulatior
T.Usher
TKR
4.1.D.1.7
SLAC, UCSC, Italy
D.Band (SSC)
S.Digel (SU)
Analysis Tools
T.Burnett
Sources
4.1.D.1.1
UW
P.Nolan (SU)
Source Detecttion
F.Longo
GEANT4
4.1.D.1.4
Italy
I.Grenier (CEA/Saclay)
Catalog Analysis
Performance
Metrics
in conjunction with
S.Ritz GSFC
A. Schlessinger
DPF
4.1.D.5
SLAC
A.Schlessinger
Release MAnagement
4.1.D.2.9
SLAC
M.Hirayama (SSC)
Pulsar Analysis
D.Band (SSC)
GRB Analysis
Document: LAT-PR-01967
Section 6 SAS Subsystem
5
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
SAS in the Ground System
DPF is robotic backbone of
IOC/SAS process handling
–
Performs L1 & L2
processing
Level 0 & HSK
Telemetry
Mission Operations Center
Instrument Operations Center
(IOC)
Nominal
Commanding
Science
Support
Center
Commanding
Data Processing Facility
Housekeeping
Data
Instrument Monitoring
DPF server and database
can handle
multiple arbitrary
sequences of tasks:
L1 pipeline; reprocessing;
MC; ….
Contingency
Commanding
Science Data
Production
Level 1 Products
Instrument
Performance
Evaluation Group
Operation Communication & Interfaces
Flight Software
Support Group
Science Analysis
Software Group
LAT User
Community
LAT Operations Steering Committee
(Suggested)
Science Analysis
Burst Alert
Messages via GCN
LAT TEAM
Keep everything on disk
Document: LAT-PR-01967
Section 6 SAS Subsystem
6
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Processing Flow
Data Pipeline
Level 0
Sim
Raw Data
Recon
Level 1
Science Tools
Level 2
Document: LAT-PR-01967
Section 6 SAS Subsystem
7
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Level 1 Sim/Recon Chain
3 GeV g
Source
Fluxes
Particle
Transport
Real Data
“Raw”
Data
Recon
Geometry
Background
Rejection
Particle ID
Level 1
Document: LAT-PR-01967
Section 6 SAS Subsystem
8
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Multiple Scattering in Converter Layers
•
•
•
100 MeV gammas
– Mean angle: ~17 mr
– Separation at next layer: ~550 mm
– Strip pitch 228 mm
– Barely resolvable into separate strip hits
@100 MeV!
MS blows up the opening angle significantly!
– Mean angle: ~ 140 mr
– Separation at next layer: ~4.5 mm
– Easily resolvable
Note design:
– Blue is “front” 12 3% X0 layers
– Green is “back” 4 18% X0 layers
• Last 2 have no radiator
– To optimize interaction rate vs resolution
100 MeV g
vertical
x2 scale change!
Multiple scattering critical to
tracking at low E
Apparent opening angle
Use Kalman filter to account for
large MS contributions
T.Usher
Document: LAT-PR-01967
Section 6 SAS Subsystem
9
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Tracking Reconstruction Example
100 MeV Gamma
T.Usher
Document: LAT-PR-01967
Section 6 SAS Subsystem
10
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Sim/Recon Toolkit
Package
Description
Provider
Status
ACD, CAL, TKR
Recon
Data
reconstruction
LAT
90% done
In use
ACD, CAL, TKR
Sim
Instrument sim
LAT
95% done
In use
GEANT4
Particle transport
sim
G4 worldwide
collaboration
In use
xml
Parameters
World standard
In use
Root
C++ object I/O
HEP standard
In use
Gaudi
Code skeleton
CERN standard
In use
doxygen
Code doc tool
HEP standard
In use
Visual C++/gnu
Development envs
World standards
In use
CMT
Code mgmt tool
HEP standard
In use
cvsweb
cvs web viewer
HEP standard
In use
cvs
File version mgmt
World standard
In use
Document: LAT-PR-01967
Section 6 SAS Subsystem
11
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Software Development
• Enable distributed development via cvs repository
• Extensive use of electronic communications
– Web conferencing (VRVS), Instant Messaging (icq)
• CMT tool permits equal development on Windows and Linux
– Superior development environment on Windows; compute
cycles on linux
• documentation and coding reviews enforce coding rules
• “Continuous integration”
– Eliminate surprises for incoming code releases
– Build code every night; alert owners to failures in build or
running of unit tests. Results tracked in database.
– Developing comprehensive system tests in multiple source
configurations. Track results in database; web viewable.
Document: LAT-PR-01967
Section 6 SAS Subsystem
12
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Nightly Builds
Performing builds
for Science Tools
now also
Display created from database query
Past release
Build status
Unit test status
Release in progress
Future release
Document: LAT-PR-01967
Section 6 SAS Subsystem
13
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
System Tests
Comparison of
current to previous
release.
Document: LAT-PR-01967
Section 6 SAS Subsystem
14
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
SAS Timeline
LAT-GS
Peer Rev
LAT-GS
CDR
LAT
CDR
LAT Cosmic
Ray Tests
DC 1
2003
2004
GRT 4
2005
2006
GRT 1
EM
DC 2
CU Beam
Test
FSW FE-Sim
MC
Sim/recon,
Proto pipeline
Document: LAT-PR-01967
Sim/recon,
Proto SciTools,
Pipeline,
Data xfer to SSC
DC 3
CU-Validated Sim/recon,
SciTools,
Final pipeline,
Data xfer to SSC
Section 6 SAS Subsystem
15
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Engineering Tests Support – EM – mid 2003
•
References
– LAT-MD-00446 – SVAC Plan
– LAT-MD-01587 - SVAC EM Tests spec, section 6.1
– LAT-MD-00570 – I&T – SAS ICD for EM
– LAT-TD-01340 – SAS Calibration Infrastructure
– LAT-TD-01588 – Calibration Algorithms for EM
– LAT-TD-00582 – EM Geometry for Simulations
•
Required deliverables
– TKR, CAL subsystem calibration algorithms
– Calibration infrastructure for time dependent parameters
– Flexible geometry facility to describe EM unit
– Reasonable fidelity simulation/reconstruction
– Disk & CPU resources for simulation and analysis
– Would like to run processing with the pipeline. Not required.
Complete
Complete
Complete
Complete
Complete
Ready for EM
Document: LAT-PR-01967
Section 6 SAS Subsystem
16
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
EM - 18 MeV on-axis photon (from VDG)
Engineering Model
Mini-Tower
(5 trays of material,
3 pairs of active
silicon)
Document: LAT-PR-01967
Section 6 SAS Subsystem
17
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
TKR – number of CLUSTERS
TKR - Number of TRACKS
m
m
g
g
Cuts: TKR trigger
Cuts: TKR trigger
Signal dominates
Negative values
are not shown
Document: LAT-PR-01967
Differential distribution
I&T / E. do Couto e Silva
Differential distribution
Signal dominates
Negative values
are not shown
Section 6 SAS Subsystem
18
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
FSW MC Support for FE-Sim – late 2003
•
•
FSW has requested a full orbit’s worth of background to test the
Front End Simulator
– ~50 Million events
– ~1200 CPU-days @ 2 secs per event
– ~500 GB output
Needed around Aug 2003 – resources in place
Complete
•
MC/Sim already in place
•
Must interface FSW code to output flight format data
Complete
Document: LAT-PR-01967
Section 6 SAS Subsystem
In progress
1 man-week
19
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Engineering Tests Support – CU – mid 2005
•
See
– LAT-MD-00446 – SVAC Plan
– LAT-MD-01587 - SVAC EM Tests spec, section 6.1
– LAT-MD-00571 – I&T – SAS ICD for CU
– LAT-TD-01589 – Calibration Algorithms for CU
– LAT-TD-00583 – CU Geometry for Simulations
•
Required deliverables
In planning – 1 man month
– ACD subsystem calibration algorithms
Complete
– Flexible geometry facility to describe CU
– Good fidelity simulation/reconstruction
Complete
– Disk & CPU resources for simulation and analysis Promised by SLAC
– Processing Pipeline and Data Catalogue
Database complete
Document: LAT-PR-01967
Section 6 SAS Subsystem
20
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
CU – 500 MeV angled electron (from test beam)
500 MeV e-
Document: LAT-PR-01967
Section 6 SAS Subsystem
21
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Level 1 Pipeline
•
Goal is to do early prototyping using EM and MC simulation runs as
undemanding clients
– Provide a general robot that can be configured to run any of the task
chains we need
• L1, L2 processing
• MC simulations
Docs:
• Data reprocessing
database:
LAT-TD-00553
• I&T/IOC tasks
server:
LAT-TD-00773
– Underlying database design complete
diagnostics:
Functional Reqs in draft now
Evaluating STScI/Hubble OPUS pipeline
Heritage from SLD experiment at SLAC
•
Design interfaces to make the pipeline portable
– Generic database usage
– Interfaces to submit processes to do the work
•
Done by DC1 if OPUS works out
– In use by Hubble, FUSE, Integral, Chandra, BeppoSax, and SIRTF
– 1 FTE-year budgeted to write from scratch if not – ready for CU
Document: LAT-PR-01967
LAT-TD-00876
Section 6 SAS Subsystem
22
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Pipeline Server Layout
Document: LAT-PR-01967
Section 6 SAS Subsystem
23
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Working with Mission Ground Systems
• Contact via biweekly GOWG meetings
• Series of End-to-End tests being planned
– SAS involved with GRT1 and GRT4
– GRT1 (11/04)
• First transmission of Level 0 data from MOC to IOCs
– GRT4 (9/05)
• Required Level 1 processing with transfer of results to
SSC
• Will have been done in CU and DC1
– Support Mission GS PDR etc
Document: LAT-PR-01967
Section 6 SAS Subsystem
24
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Development of Science Tools
•
Extensive planning on which tools are needed to do science - and
their requirements
– One set of tools for all – “astronomy standard”
– Had external review (9/2002) to see if we are on the right track
• No major problems noted
http://www-glast.slac.stanford.edu/ScienceTools/reviews/sept02/report/review_091602.pdf
•
In progress with the SSC
– Joint oversight group
– Sorted out technical basis (HEASARC standards; support of
community; re-use of LAT developments)
•
Effort ramping up now
•
Selected Level 1 database technology
– Meets performance requirements
– Starting to implement at GSFC
Document: LAT-PR-01967
Section 6 SAS Subsystem
25
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Science Tools for High-Level Analysis
Gamma rays in 1-day scanning
observation (~150k >30 MeV),
color coded by energy
Bright diffuse emission of the Milky Way + Galactic
and extragalactic point source populations
Document: LAT-PR-01967
Annual rate (all energies)
~108 gamma rays/year
Hundreds of sources even in this
short time: What are their fluxes?
Which are flaring?
Section 6 SAS Subsystem
26
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Main Science Tools
Package
Likelihood
Level 1 database access
Description
Workhorse model fitting for detection &
characterization of cosmic gamma-ray sources
Extracts desired event data
Exposure calculation
Uses IRFs, pointing, livetime etc. for deriving
calibrated source fluxes
Source identification
Identifies gamma-ray sources with cataloged
counterparts at other wavelengths
GRB analysis
Temporal and spectral analyses of burst profiles
Pulsar analysis
Phase folding & period searching of gamma-ray
pulsars and candidates
Observation simulator
Document: LAT-PR-01967
High level simulation of observations of the gammaray sky with the LAT
Section 6 SAS Subsystem
27
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Science Tools Toolkit
Package
Description
Provider
Status
PIL, PIL++
IRAF parameter
access
HEASARC
In use
cfitsio, CCFits
FITS file
manipulation
HEASARC
In use
XSPEC, Sherpa
For GRB spectral
modeling
HEA standards
Under consideration
Root
gui etc
HEP standard
Under consideration
python
Scripting
World standard
Under consideration
doxygen
Code doc tool
World standard
In use
Visual C++/gnu
Development envs
World standards
In use
CMT
Code mgmt tool
HEP standard
In use
cvsweb
Cvs web viewer
World standard
In use
cvs
File version mgmt
World standard
In use
Document: LAT-PR-01967
Section 6 SAS Subsystem
28
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Data Challenges
• Now traditional in HEP experiments
– exercise the full analysis chain prior to needing it
– involve the collaboration in science prep early
• Doing planning now
– Fall 2003 - DC1
• 1 day’s data through full instrument simulation and first look
at Science Tools
– Fall 2004 – DC2
• 1 month’s background/1 year signal
• Test more Science Tools; improved Pipeline
– Spring 2006 – DC3
• run up to flight – test it all!
– DC1 Plans
• Focus effort through Analysis Group (S.Ritz) and kickoff
workshop in mid-summer
– Including geometry and simulation validation
• Sept collaboration meeting as milestone for start
Document: LAT-PR-01967
Section 6 SAS Subsystem
29
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Prep for GS Peer Review and CDR
• SAS was baselined in PDR
• Ground Systems CDR has been scheduled for 2/2004, with Peer
Review in 11/2003
• Expectations for Peer Review
– Successful EM support
– Level 1 Prototype operational
• Functional requirements; Design documents ready
– Science Tools
• Major components understood, with schedule, manpower and
milestones
• Plan to schedule next external review to be coincident with
Peer Review
– ICD with SSC
Document: LAT-PR-01967
Section 6 SAS Subsystem
30
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Processing Flow Recap
Data Pipeline
~80-90% done
if Opus works
out
Level 0
95% done
In use
Sim
Raw Data
Prototype
database being
implemented
Recon
90% done
In use
Level 1
Science Tools
• In planning/progress
• Estimate 40 man-Yr effort
• Available from SSC and LAT
collaboration
Document: LAT-PR-01967
Level 2
Section 6 SAS Subsystem
31
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
4.1.D Science Analysis Software
Cost/Schedule Summary
Document: LAT-PR-01967
Section 6 SAS Subsystem
32
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
CCB Actions Affecting 4.1.D
Change Request #
Description
Status
LAT-XR-01146-01
UW Manpower
Approved, $283K
LAT-XR-01148-02
NRL Resource
Leveling
Approved, $0K
LAT-XR-01752-02
SLAC/HEPL Labor
Escalation Rates
Approved -$16K
Document: LAT-PR-01967
Section 6 SAS Subsystem
33
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Cost by Fiscal Year
• WBS 4.1.D without contingency
FY00 FY01 FY02 FY03 FY04 FY05 Total
in K$ in K$ in K$ in K$ in K$ in K$ in K$
85
215
494
923
914
965 3,595
Document: LAT-PR-01967
Section 6 SAS Subsystem
34
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Budget, Cost, Performance
Budget vs Actuals vs Performance
DOE + NASA Project Expenditures
4.1.D Science Analysis Software
$M, Then-Year Dollars
4
2
Actual Commitments
ACWP
BCWP
BCWS+ Planned Commitments
BCWS
0
. .
FY00
. .
. .
. .
. .
FY01
Document: LAT-PR-01967
. .
. .
. .
FY02
. .
. .
. .
. .
. .
FY03
. .
. .
. .
. .
FY04
Section 6 SAS Subsystem
. .
. .
. .
FY05
35
. .
. .
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Cost/Schedule Status
•
Status as of March 31, 2003:
Item
In k$
Budget at Complete
3,595
Budgeted Cost for Work Scheduled (a)
1,257
Budgeted Cost for Work Performed (b)
1,246
Actual Cost for Work Performed
1,093
Cost Variance
153
12% of (b)
Schedule Variance
-12
-1% of (a)
Document: LAT-PR-01967
Section 6 SAS Subsystem
36
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Concerns
•
Manpower is the major concern
– No technological risks
– “just” a matter of implementing and supporting the solutions we
have designed for
– Infrastructure group is thin, and hard to find people willing to do
it.
• SLAC, GSFC are providing much of that support
– Mitigation
• We concentrate on early starts to critical elements with
incremental improvements over time.
• Reuse appropriate software from other projects as much as
possible
• As much automation of repetitive tasks as possible
Document: LAT-PR-01967
Section 6 SAS Subsystem
37
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Summary
• SAS driven by Engineering Tests and LAT Integration
– EM support ready; CU looking good
– Sim/Recon ready
• Science Tools under development
– In concert with the SSC
– Drive schedule with Data Challenges
• Level 1 Pipeline early start
– Trying to have prototype in place for EM, FSW & DC1
support this year
– End-to-end tests scheduled with Mission Ground Systems
Internal validation, Beam tests and Data Challenges in place to
ensure successful end-to-end functionality well before launch.
Document: LAT-PR-01967
Section 6 SAS Subsystem
38
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Backups
Document: LAT-PR-01967
Section 6 SAS Subsystem
39
GLAST LAT Project
CDR/CD-3 Review May 12-16, 2003
Components of the Environment
Pulsar
ephem. (D4)
Level 0.5
Level 1 (D1)
Pointing/livetime
history (D2)
Alternative source
for testing highlevel analysis
Event
display (UI1)
Pulsar period
search (A4)
Ephemeris
extract (U11)
Arrival time
correction (U10)
Pulsar phase
assign (U12)
Data extract
(U1)
Pt.ing/livetime
extractor (U3)
Alternative for
making additional
cuts on alreadyretrieved event data
Observation
simulator (O2)
Data subselection (U2)
Pt.ing/livetime
simulator (O1)
Pt.ing/livetime
extractor (U3)
Exposure
calc. (U4)
Catalog
Access (U9)
Likelihood (A1)
Astron.
catalogs (D6)
Map gen
(U6)
IRFs (D3)
GRB unbinned
spectral analysis (A9)
IRF visualization (U8)
Src. ID (A2)
GRB event
binning (A5)
User Interface aspects of the standard
analysis environment, such as Image/plot
display (UI2), Command line interface &
scripting (UI4), and GUI & Web access
(UI5) are not shown explicitly.
GRB spectral-temporal
modeling (A10)
GRB spectral
analysis (A8)2
1
Document: LAT-PR-01967
Source model
def. tool (U7)
Interstellar em.
model (U5)
GRB LAT DRM
gen. (U14)
This tool also performs periodicity tests and the
results can be used to refine ephemerides
2 These tools can also take as input binned data
from other instruments, e.g., GBM; the
corresponding DRMs must also be available.
LAT Point source
catalog (D5)
Pulsar
profiles (A3)1
GRB rebinning
(A6)2
GRB visualization (U13)
GRB temporal
analysis (A7)2
Section 6 SAS Subsystem
40