GLAST Large Area Telescope
Gamma-ray Large
Area Space
Telescope
ISOC Review
15 February 2006
FSW Activities, Organization, and
Resource Needs
Jana Thayer
Stanford Linear Accelerator Center
jana@slac.stanford.edu
(650) 926-4596
Outline
•
•
•
•
•
Introduction to FSW
Main Responsibilities
Activities Over Time
Organization
Resource Needs
– People
– Space and facilities
– Things
• Summary
15 Feb 2006
ISOC Review - FSW
2
Introduction to Flight Software (FSW)
• The LAT flight software code base includes all on-board
software
• The flight software team –
– Develops, maintains, and tests the software that configures
the instrument and supports instrument housekeeping and
data acquisition by the LAT
– Provides the means for configuring the instrument for data
collection, retrieving event data from detectors on the
instrument, and storing that data for transmission to the
ground for subsequent analysis by investigators
– Develops and maintains the infrastructure that is used to
develop and test the on-board software
15 Feb 2006
ISOC Review - FSW
3
FSW – Major Responsibilities
•
•
•
•
•
•
Develop, test, and maintain flight software
– Develop new on-board filters
– Develop diagnostic code in response to problems
– Validate parameters and software for upload
– Perform regression testing/validate new code
Develop and maintain software tools for the development, testing, and
documentation of the on-board flight software
Maintain the tools that track LAT Configuration
– Software versions
– Uploaded data files, parameters
Maintain the Dataflow lab
– Maintain all flight spares, EGSE, and teststands including Testbed
– Develop this space into a facility for troubleshooting,
verification/validation, and running operational scenarios
Maintain and develop documentation
– Procedures, User’s Manuals, Technical Documents, Website, etc.
Monitor the dataflow system and help resolve instrument anomalies
15 Feb 2006
ISOC Review - FSW
4
FSW Activities - Breakdown
ISOC FSW
Operations
Code
Maintenance
LAT
Configuration
Filter
Development
Version
Control
GRB
Response
Development
Flight Code
Base
Build
Regression
Internal Tools
New Code
Networks
MCR
Dataflow Lab
CMX
Instrument
Monitoring
IT/
Infrastructure
Test
Hardware
RPM
Diagnostics
SLAC
FMX
Other
Operational
Scenarios
Other
Flight Build
Operational
Scenarios
(ATS)
Account
Management
Computer
Management
FES Dataset
Preparation/
Tracking
SLAC
Linux
Environment
SLAC
Windows
Environment
SNITZ
JIRA
FSW Web Site
Documentation
Construction
FSW Scientist/
SW Developer
SLAC/SCCS
MTech/ETech
Configuration
Manager
Release
Management
Dataflow Lab
Tools
Testbed/
Teststands
Tech Writer
15 Feb 2006
python
gcc
Hardware
perl
Doxygen
FSW Version
Control
VxWorks
Other
ISOC
ISOC Review - FSW
5
FSW Activities – Present
•
Current Activities
– Finish code development
• GRB response
• FMX (tool for tracking LAT Configuration)
– FQT (Flight Qualification Testing)
• Regression testing FSW builds
• Formal testing of build containing full functionality
– I&T support
• Adding functionality to existing code base
• Troubleshooting/debugging
•
Current staffing
– SLAC staff
• 1 FSW lead
• 6 SW developer/physicist + 1 FSW Test
• 1 Tech Writer/Web developer
– External Contributors
• 2 SW developers (NRL)
• 2 FES/filter (OSU)
• 1 Configuration Management (GSFC)
15 Feb 2006
ISOC Review - FSW
6
FSW Activities – Future
•
•
Future Activities
– Support I&T and system testing at NRL
– Maintain code base and retain the ability to grow the flight software
• Filter development
• GRB Response development
• Creation of diagnostic code to debug problems and spy on state
of LAT
– Maintain internal software tools
– Validation/Verification of new software uploads, parameters, and
procedures
– Interface with other ISOC groups to –
• Monitor dataflow system: trigger, filter, background rates, etc.
• Define new instrument configurations
• Troubleshoot/resolve instrument anomalies
Future steady-state staffing plan
– 4 – 5 SW developer/physicists
• Contribute to FSW and PVO
– Some SLAC staff will have moved from GLAST to other projects
• Some support available
15 Feb 2006
ISOC Review - FSW
7
Retaining Knowledge
•
•
•
•
On-board software functionality:
– Primary and secondary boot process/rad750 processor
– Collection and transmission of LAT housekeeping data
– Configuration of the instrument for data acquisition
– Calibration of the instrument
– Physics event filtering, characterization, and delivery
– Communication between the Spacecraft and the LAT
– Telemetry and telecommand services
– Spacecraft attitude/time management
– Thermal control
– Core processes and functions: multi-CPU communications, inter-task
communications, software watchdog, etc.
Test software:
– Regression testing of builds
– Front-End Simulators (FES)
Software tools:
– CMX
– FMX
Hardware:
– Testbed, 1x4 grid, EGSE
– FES
15 Feb 2006
ISOC Review - FSW
8
FTE Estimates
FSW FTEs per year
12
10
# FTEs
8
6
4
2
0
FY2006
FY2007
FY2008
FY2009
FY2010
FY2011
FY2012
FSW group is changing composition over the years as experienced people move on and less
experienced people come in. Information must be transferred from one set of people to
another, but time needs to be allotted to allow new people to learn.
15 Feb 2006
ISOC Review - FSW
9
FTE Justification
•
Skill set of people necessary for maintaining FSW in the future
– Real-time programming, VxWorks
– Experience with commissioning and operating a particle physics
detector
– C/C++, python, perl, unix/linux
– Configuration Management
– Ability to work within existing FSW framework
– Understanding of trigger and dataflow system
 FSW group contains many people with in-depth knowledge about the
operation of the T&DF system and the instrument
•
To fully utilize members of FSW over the length of the mission:
– Re-task code developers to monitoring of instrument
• Trigger and dataflow system
• Filter
• Background rates
– Contribute to calibrations and defining new instrument
configurations
15 Feb 2006
ISOC Review - FSW
10
Summary of staff needs
• People (included in previous FTE Estimate)
– 1.0 FTE: SW developer/physicist
• Now: Support I&T and system test
• Launch: Resolve instrument anomalies (quickly)
• Future: Retain core FSW knowledge
– Need overlap between the incoming and outgoing staff
– 1.0 FTE: SW developer
• Maintain existing software tools
• Develop new tools that help all ISOC groups interface to
LAT configuration information
• Act as “Configuration Manager” for FSW and HW
– 0.5 FTE: Lab Technician
• Custodian to dataflow lab
• Maintenance of hardware infrastructure
15 Feb 2006
ISOC Review - FSW
11
Existing Equipment/Lab Space
Uber
Test
Stand
Bench
Bldg 84
Room
B101
FES
Rack
Testbed
Multicrate
Test
Stand
15 Feb 2006
Pwr
Rack
Dalek
ISOC Review - FSW
GASU
Test
Stand
12
LAT Testbed
• Provides a full Trigger and Dataflow system with flight-like
interfaces and hardware
• Front-end Simulator (FES) ingests Monte Carlo data
– Dataflow integrity and throughput
– Filter
15 Feb 2006
ISOC Review - FSW
13
Hardware Infrastructure
• Existing hardware:
– LAT Testbed and FES (Front End Simulator)
– VSC (Virtual Spacecraft)
– SIIS (Spacecraft Simulator)
– Test stands (trigger and dataflow system)
– EGSE and flight spares in Building 33
• LAT Testbed and test stand usage
– Commissioning of LAT Dataflow System
– Code development/debugging by FSW, FSW test, I&T, ISOC
– Testing flight software functionality at the system level
– Running operational scenarios
– Regression testing of new FSW builds
15 Feb 2006
ISOC Review - FSW
14
Incoming Equipment
• ISOC FSW will maintain the existing equipment
• ISOC FSW will inherit flight spares and all useful EGSE from I&T
– 1x4 grid
• 2 Trackers (flight spare + non-flight)
• 4 CAL units (3 flight spares + 1 EM)
• 4 flight TEMs
• GASU and other support electronics
– LAT Power Rack/LAT Control Rack
– Mobile Computing Racks
– SIIS (Spacecraft to Instrument Interface Simulator)
– Mini-LAT with muon telescope
– Miscellaneous EGSE
• GASUs
• PDUs
• VME crates
• Spare parts and miscellaneous
15 Feb 2006
ISOC Review - FSW
15
New and Improved Dataflow Lab
B101
Table
Uber
teststand
B105
Table
VSC Power
Rack Rack
Debug
Bench
1x4 grid
TPG
Table
Power
Rack
VSC
Rack
FES
Rack
VSC
Rack
Power
Feed
Rack
Testbed
SIIS
B138
Table
Dalek
Multicrate
teststand
Test
Stand
Mini-LAT
Table
MCR MCR
Table
B136
15 Feb 2006
Existing Lab Space
New Lab Space
Existing Equipment
New Equipment
ISOC Review - FSW
16
Infrastructure Changes to B101
• To accommodate the flight units, install
– Install Nitrogen (keep 1x4 grid clean and dry)
– Install Oxygen sensor for safety
– Maintain temperature at 22C ± 5C
– Maintain humidity at 30% - 50%
– Monitor and archive humidity and temperature
• Changes to existing lab space
– Install double doors to new lab space in B105
• Additions to existing lab space
– Additional air conditioning
– Additional power feeds and outlets
– Bus bars installed (facility ground)
– UPS providing 10 min @ 7500W
– Storage solution for spares
15 Feb 2006
ISOC Review - FSW
17
New Lab Space
• B105/B136
– Today: cubicle/office area
– Future: lab space and
workstations to interface to
hardware test stands in
B101
B105
VSC Power
Rack Rack
1x4 grid
• B138
TPG
Table
– Today: storage area
(not GLAST)
– Future: GLAST storage
Power
Rack
VSC
Rack
• Both rooms brought up to
the same standard as B101
–
–
–
–
–
Tile
Paint
Air conditioning
Power feeds
~10 Ethernet drops
SIIS
B138
Table
MCR MCR
Table
B136
15 Feb 2006
ISOC Review - FSW
18
Other Resource Needs
•
•
•
•
Things
– Software licenses (i.e., VxWorks)
– Software
– Computers (Suns, laptops)
– DSL for core staff (on call)
– Travel
• GSFC
• NRL, I&T support
• NRL staff
– Conferences, meetings
Collaboration support
– Filter development
– Parameters for optimizing instrument performance
SLAC support
– Networking
– Windows environment
– Unix/linux environment
ISOC support
– Configuration Management
– Account management/computer maintenance
15 Feb 2006
ISOC Review - FSW
19
Summary
• FSW activities will evolve over time from primarily code
development to operations and instrument monitoring
• Resources needs
– 3 FTEs to maintain FSW activity level through FY2009 and
beyond
– Upgraded/additional lab space
– Collaboration support for filter development
– SLAC support for networking
• Concerns
– Retaining knowledge of core FSW for length of mission
– Maintenance of VxWorks license (by SLAC or FSW?)
15 Feb 2006
ISOC Review - FSW
20