LAT ISOC
Gamma-ray Large
Area Space
Telescope
DOE Status Review, 12 Oct 2004
GLAST Large Area Telescope
LAT
Instrument Science Operations Center
Rob Cameron
Stanford Linear Accelerator Center
rac@slac.stanford.edu
650-926-2989
1
LAT ISOC
DOE Status Review, 12 Oct 2004
Outline







Overview
ISOC Organization
ISOC Operations
Software Architecture
Operations Facility Planning
Development Schedule
Current Status
2
LAT ISOC
DOE Status Review, 12 Oct 2004
LAT ISOC in the GLAST GDS
Analysis Software
3
LAT ISOC
DOE Status Review, 12 Oct 2004
LAT ISOC Objectives



The LAT ISOC is organized to:
– safely operate the instrument
– produce the LAT science data products
Functions:
– Command generation; health and safety monitoring
– Maintain and modify flight software and testbed
– LAT performance verification and optimization
– Process and archive Level 1 and Level 2 data
– Maintain and optimize the software pipeline that produces
science data products
These functions are organized as teams that share personnel
4
LAT ISOC
DOE Status Review, 12 Oct 2004
ISOC Reviews



ISOC Peer Review: 2 March 2004
ISOC CDR: 4 August 2004
GLAST GSDR: 18-19 August 2004

CDR: very successful review
– 8 RFAs and 8 Recommendations received
– 6 RFA responses submitted for review, others in work
– All recommendations completed

GSDR: very successful review
– No ISOC-specific RFAs
– Ground-system RFAs will be applied to ISOC as needed
5
LAT ISOC
DOE Status Review, 12 Oct 2004
CDR RFA Status
Section
RFA Title
Requestor
Actionee
ECD/Comments
Sent to
Originator
for Review
1
ISOC Documentation
Tree
E. Andrews
L. Bator
Response
submitted
8/12/04
2
Requirements Traceability
Tool
E. Andrews
B. Craig
J. Martin
Response
submitted
8/27/04
3.1
Add ISOC Architecture
Diagram
E. Andrews
B. Craig
Response
submitted
8/16/04
3.4
ISOC Requirements &
Testing (Overall)
E. Andrews
J. Martin
Response
submitted
9/14/04
4.2
SAA Handling Approach
N. Johnson
L. Bator
Response
submitted
8/30/04
4.2
Launch Critical Support
Plan
M. Rackley
J. Leibee
S. Culp
Awaiting GSFC
input
4.2
Red/Yellow Limit
Philosophy
J. Leibee
L. Bator
Awaiting SAI
definitions
8
SAS Verification
Approach
M. Rackley
J. Martin
Response
submitted
Submitted to
Chairpersons
Closed
9/07/04
9/14/04
6
LAT ISOC
DOE Status Review, 12 Oct 2004
ISOC Organization
ISOC Manager
R Cameron
Database
Commanding,
Health & Safety
Flight Software
Documentation
Performance V&O
Science Products
Science Analysis
Tools
Timeline
Planning
Code
Maintenance
Calibration
Level1 CRs, gammas,
diagnostics
Simulation/Recon
Command
Generation
Code
V&V
Performance
Trending
Level 2
Transients, GRBs
Science Tools
Command V&V
Testbed
Maintenance
Standard Source
Monitoring
Pipeline
Operation
Infrastructure
Health & Safety
IRF Generation
Configuration
Tracking
7
LAT ISOC
DOE Status Review, 12 Oct 2004
Commanding, Health and Safety

The Commanding, Health and Safety (CHS) team is responsible
for:
– LAT mission planning support
– generation and validation of LAT commands
– passing LAT commands to the GSSC
– verifying commands were correctly executed
– receiving Level 0 data from the MOC
– logging and archiving of all commands and Level 0 data
– monitoring data to ascertain and track the health and safety
of the instrument
– continuous knowledge of the configuration of the LAT
8
LAT ISOC
DOE Status Review, 12 Oct 2004
CHS System



ITOS used for command, health and safety functions
– HK data limit checking
– Telemetry and command definition file validation
– Command load verification and validation
ITOS provides compatibility with MOC
Other tools
– Level 0 receipt and archiving
– HK trending
– Data transmission
– Mission planning and generation of file uploads
– Anomaly tracking and notification
– Relational database queries for trending and analysis
– Configuration management tools
9
LAT ISOC
DOE Status Review, 12 Oct 2004
Flight Software


The Flight Software (FSW) team is responsible for:
– Updating and validation of all flight software files
– Debug or problem fixes to the FSW
– Implementation, and validation on the instrument test bed
of authorized upgrades to FSW
– Continuing maintenance of the instrument testbed to
ensure it is available to validate code and command
sequences as well as to investigate any anomalies seen on
orbit
FSW team moves into ISOC after FSQ (April 2005)
– Continue code/test/CM cycle as part of the ISOC operations
– Maintenance of Test Bed
– Review commanding, HK, and performance on a frequent
and regular basis
10
LAT ISOC
DOE Status Review, 12 Oct 2004
Performance Verification and Optimization


The Performance Validation and Optimization (PVO) team is
responsible for:
– instrument calibration from low level through to IRFs
– continuous monitoring of the LAT science performance,
identification of instrument performance trends and
resolution of performance anomalies
– generation of performance reports
– generation and initial validation of algorithms that improve
on-orbit performance of the LAT
– management of pre-launch test and calibration data
– configuration and maintenance of the LAT reference
geometry and the LAT Monte Carlo Model
The basic structure for the PVO team is already in place for
LAT I&T activities
11
LAT ISOC
DOE Status Review, 12 Oct 2004
Science Products


The Science Products Team is responsible for
– Generation, archiving and distribution of the Level 1 data
– Generation, archiving and distribution of specific Level 2
data needed for transient source detection and GRB
parameter determination
– Configuration control of the pipeline and generated data
The functions (pipeline etc) used by the Science Products team
are developed by the Science Analysis Software team
12
LAT ISOC
DOE Status Review, 12 Oct 2004
Science Analysis Software


The Science Analysis Software Team is responsible for
– Development and maintenance of the pipeline machinery
– Development and maintenance of the simulation,
reconstruction and event classification software
– Development and maintenance of the calibration
algorithms, including low level cal and Instrument
Response Function generation
– Development and maintenance of the quicklook transient
analysis tools
– Development and maintenance of the high-level diagnostics
derived from reconstruction and classification
– Development and maintenance of the high-level analysis
tools
ISOC will have responsibility for tools used to produce its
deliverables
13
LAT ISOC
DOE Status Review, 12 Oct 2004
ISOC Staffing Plan
 ISOC staff will grow mostly by team transfer, plus some new hires
ISOC Staffing by Job Function
FTE
60
50
Support
40
Scientist
Technician
30
Software
20
Engineering
10
Management
0
1
5
9
13 17 21 25 29 33 37 41 45 49 53
Months since March 04
14
LAT ISOC
DOE Status Review, 12 Oct 2004
ISOC Operations
ISOC will be involved in all phases of LAT operation:
 LAT I&T
– Pre-FSW (EM)
– Post-FSW (Flight unit level)
– TVAC test at NRL
 Observatory I&T at Spectrum Astro
 Launch and Early Orbit (L&EO) – Phase 0
– LAT power-on and configuration
– Initial checkout and calibration
 First year – Phase 1
– Survey mode
 Second and subsequent years – Phase 2
– Pointed observations
– Survey mode
15
LAT ISOC
DOE Status Review, 12 Oct 2004
Initial Turn-On and Checkout (Phase 0)





Launch and Early Orbit (L&EO) phase currently scheduled for
60 days
Instrument turn-on will not be executed automatically
– Humans required to check environmental conditions prior
to significant steps
– Must establish correct LAT configuration at each step
Functional checkout of DAQ, ACD, CAL and TKR
Initial calibrations
ISOC will have staff at MOC during L&EO, but data will also
flow to west coast ISOC for processing
16
LAT ISOC
DOE Status Review, 12 Oct 2004
Science Operations (Phases 1 and 2)





Data taking
– Continuous
– Few commands needed
Calibration
– Weekly, biweekly and monthly
– A few commands to initiate
Load changes to tables and FSW
– Infrequent
– A few commands and/or file uploads which may be large
Load new tables and files
– Infrequent
– A few commands and/or file uploads which may be large
Perform Diagnostics
– Infrequent
– A few commands and/or file uploads which may be large
17
LAT ISOC
DOE Status Review, 12 Oct 2004
Operations Coverage



Weekday, day-time operations
– Mission planning, command building
– Operator shift coverage
• 5 am to 2 pm to cover MOC shift times on East Coast
• 9 am to 6 pm to cover ISOC internal coordination needs
Telemetry monitoring and science data processing
– Continuous automatic processing
On-call support available 24/7
– Support for targets of opportunity, anomalies
– Operators for real-time commanding or anomaly support
– Software engineer for emergency software support
– Computer technician for computer/network support
– Team Lead and Instrument Physicist for anomalies
18
LAT ISOC
DOE Status Review, 12 Oct 2004
LAT Mission Planning Timeline

Weekly mission planning cycle, with 3 week lead time
Monday
Tuesday
July 30
Wednesday
31
Thursday
Friday
August 1
2
GSSC Sends
Preliminary
Activity Timeline
to MOC
6
7
8
Sunday
4
5
11
12
18
19
MOC Requests
TDRSS Contacts
for the Week
9
10
ISOC Sends
Command Plan to
GSSC
ISOC Generates
Command Plan
13
Saturday
3
14
15
16
17
Deadline for
ISOC to Send
Changes* to
Week
20
21
GSSC Sends
Final Activity
Timeline to MOC
27
22
MOC Uploads
ATS
28
23
24
25
26
31
September 1
2
ATS Goes Active
29
30
*Minor changes that don’t affect observing
schedule or TDRS contacts
19
LAT ISOC
DOE Status Review, 12 Oct 2004
Software Architecture: Primary Data Flows
LAT ISOC
MOC
Level 0 Data,
LAT Alerts,
Integrated Observ. Timeline,
Spacecraft Timeline,
As-Flown Timeline,
Flight Dynamics Products,
SC Incident Reports,
PDB updates
Level 0
Sci Data
Receive
& Archive
Level 0
Hkpg Data,
LAT Alerts
Integrated Observ. Timeline,
Spacecraft Timeline,
As-Flown Timeline,
Flight Dynamics Products
GSSC
Anomaly
Tracking &
Notification
System
GSSC
Contingency:
LAT Instrument Commands,
LAT Instrument Loads,
LAT Timeline
Archive
& Send
Tested
OK
Commands
to Verify
Alerts, Logs,
Process status,
File status
Archive
& Send
Level 1&2 Sci Data Products,
LAT Timeline
CHS
monitoring
& trending
Mission
Planning
Receive
& Archive
Prelim. Science
Timeline, Long Term
Science Schedule,
Observ. Timeline Package
Science
Data Analysis
Pipeline
Level 1&2
Sci Data
Products
LAT Test Bed
MOC
LAT Incident Reports,
LAT T&C Database Updates
All data into ISOC,
and products out
Database
Web
Access
20
LAT ISOC
DOE Status Review, 12 Oct 2004
Automatic Telemetry Processing
LAT ISOC
Science Data Analysis Pipeline
Calib.
DB
Calibration Trending
Level 0
sci & hkpg
telemetry,
LAT Alerts
MOC
GINO
& FastCopy/DTS
GINO
& FastCopy/DTS
Level 1 processing
Receive
& Archive Level 0
science
telemetry
Apply
Calibration
Reconstruction
Archive
& Send
System-level
Monitor
Level 2 processing
Subsystemlevel
monitor
GCN
Level
1&2
data
products
GSSC
Transient Search/
GRB Refinement
Level 0 hkpg
telemetry,
LAT Alerts
Realtime Level 0
hkpg telemetry
(during contacts),
LAT Alerts
ITOS
Monitor hkpg
parameters
Text & Stripchart
displays (optional)
Monitor
Trends
ITOS
Monitor
Data
Alerts, Logs,
Process status,
File status
FASAT
Anomaly
Tracking &
Notification
System
Reports, Log entries,
Data Products
received and sent
Database
STOL Config monitor scripts
Beeper / email message
Beeper / email message
21
LAT ISOC
DOE Status Review, 12 Oct 2004
Requirements Drive ISOC Design


ISOC requirements tracing is complete
Each ISOC Level 3 requirement was identified as:
1. Procedural (non-software)
2. Existing software
3. Under development by SAS
4. To be developed by others (I&T, FSW, ITOS)
5. New development needed
42
277
39
21
88
Total


467
Some requirements map to multiple categories or SW tools, so
total exceeds total number of Level 3 requirements (379)
Top-down planning and scheduling of new software
development is underway
22
LAT ISOC
DOE Status Review, 12 Oct 2004
ISOC Operations Facility



Working with SLAC management and Facilities group to define
near-term ISOC operations facility. Coordinating ISOC space
requirements with LAT project management.
Near-term ISOC facility: a new temporary building at SLAC
Long-term ISOC facility: discussions are starting with SLAC
and KIPAC management on ISOC facility in new Kavli building
at SLAC, available 2006. Also ISOC presence on Stanford
campus.
23
LAT ISOC
DOE Status Review, 12 Oct 2004
Proposed ISOC Layout for 60΄x36΄ Building
59'-6"
15'-0"
25'-0"
15'-0"
11'-0"
Operations
Center
35'-6"
Viewing
wiindow
Glass doors or glass
windows in doors
20'-0"
Cipher
locked
Glass window
in door
14'-6"
Glass window
in door
No window (wall for
projection screen)
24
LAT ISOC
DOE Status Review, 12 Oct 2004
ISOC Trailer Location
25
LAT ISOC
DOE Status Review, 12 Oct 2004
ISOC Development Schedule
Phase / Milestone
A
u
g
0
4
S
e
p
0
4
O
c
t
0
4
N
o
v
0
4
D
e
c
0
4
J
a
n
0
5
F
e
b
0
5
M
a
r
0
5
A
p
r
0
5
M
a
y
0
5
J
u
n
0
5
J
u
l
0
5
A
u
g
0
5
S
e
p
0
5
O
c
t
0
5
N
o
v
0
5
D
e
c
0
5
J
a
n
0
6
F
e
b
0
6
M
a
r
0
6
A
p
r
0
6
M
a
y
0
6
J
u
n
0
6
J
u
l
0
6
A
u
g
0
6
S
e
p
0
6
O
c
t
0
6
N
o
v
0
6
D
e
c
0
6
J
a
n
0
7
F
e
b
0
7
ISOC CDR
GSDR
ITOS setup/configuration
ISOC Verif. with Test Bed
DB/Web/E-logbook Dev
Plotting/Trending Dev
Mission Planning Dev
Diagnostic Tool Dev
ISOC Demos
ISOC Simulation Testing
ISOC SW Releases
1
2
4
3
Ground Readiness Tests
2
3
4
5
7
6
End-to-End Tests
1
2
3
4
5
6
Environmental tests at NRL
ISOC testing at NRL
ISOC ready to take over LAT operation
ISOC used to I/F with LAT (front-door only)
Mission Simulations
Launch
SW Dev Slack Times
26
M
a
r
0
7
LAT ISOC
DOE Status Review, 12 Oct 2004
Current Status



ISOC team structure and development plan is in place
– System development underway
– Software releases, GRTs and ETE tests as milestones
ISOC activity is ramping up
– ISOC staffing plan is being refined
– Recent hires in July, August
– CHS software developer starting in November
– Test engineer to be hired in 2004
– ISOC facility and infrastructure being developed
Lots of coordination effort needed between various project
elements to bring together all the elements of the ISOC
– e.g. inter-team meetings, ICDs
27