DRAFT LAT End to End System Test Executive Summary LAT Systems Engineering recognizes and supports the need to perform End-to-End (E2E) system tests on the LAT prior to integration at the observatory level. These tests mitigate the risk of the LAT not meeting system performance requirements and support the on-going system integration efforts of the project. Systems Engineering has held reviews, discussions and meetings regarding the various forms the E2E tests may take. Options have been analyzed and reviewed with a goal of achieving the lowest impact and best value to the overall program. Benefit to the program of any test and its results are a primary consideration. Additionally, impact to schedule, cost and risk to one-of-akind hardware is of critical importance. Proposed methods of conducting the E2E test are an airborne test using actual background stimulation of the instrument at trigger rates greater than available on the ground, and groundtesting using simulated triggers. In summary of those meetings and discussions, Systems Engineering has determined that to allow the LAT to be operated on an aircraft is not compatible with the LAT design reference mission. Certifying that the LAT and its GSE, especially EGSE, may be operated without damage on an aircraft would require qualification efforts similar to that being executed for the primary mission. While the expected performance envelope of the LAT may include the proposed airborne environment, no proposed GSE environment includes this type of operation as a baseline. LAT subsystems, LAT support equipment and LAT test designs would need to be modified or developed to support this activity in parallel with the core objective. This effort would divert limited resources at critical times. Whatever risk is mitigated by the test would be more than offset in risk to the program in hardware, hardware development/qualification efforts, cost and schedule. An evaluation of what testing would be required to determine compliant End to End system performance is necessary. It is expected that ground testing of the LAT and Lat subsystems will naturally include portions of what would comprise the E2E testing. The goal is to establish a set of tests that would then provide the necessary confidence that the LAT performance as an instrument is as expected. Evaluation of the planned sub-system tests against the set of E2E comprehensive tests would identify the additional testing necessary to achieve this goal. DRAFT Background The LAT instrument carries no independent End to End instrument verification requirement. The GLAST observatory does carry an E2E requirement which involves the instrument, observatory, RF data links, MOC and IOC facilities. Since the LAT is a very complex particle detector instrument, it was advised that the instrument be tested in an E2E configuration. Early in the program, there was risk that the LAT front end (the DAQ) or the LAT Trigger &Data Flow system would not be adequate for the expected trigger and data rates for the mission. These needed to be tested prior to delivery for observatory integration. It has been expressed that the randomness of the events the LAT would be observing were not capable of being simulated on the ground. Options, which have been suggested prior to the orbital verification test of E2E performance, include a beam test and an airplane test. A beam test was determined to be not practical for effectively stimulating the instrument. An airplane test was proposed and accepted by the program, not as a functional test, but as Risk Mitigation. Such a test would provide the instrument an environment where stimulus using cosmic-ray induced background in the upper atmosphere would drive the instrument in a fashion that provides a good approximation to a realistic environment. This test was to be optional and performed as an opportunity test during airborne transport of the instrument. To perform the test requires research, preparation and some additional efforts so an analysis and trade of the Airplane Test was undertaken to assess the impacts to the program and determine the benefit of performing such a test. In separate activities, the Electronics and Flight Software Subsystem evaluated their requirements for system performance metrics testing, evaluation of electronics performance, LAT operational performance and subsystem integration support using the LAT Test Bed as a host for these efforts. The general perception by all associated with the LAT program is that an End to End test is a good idea. What has been at issue is how high a fidelity, what methodology , and to what specifications should such a test be conducted. DRAFT End to End Test Objective The objective of the E2E is to provide confidence in the operation of the LAT from the particle flux stimulus input to the final data product output sent to the solid-state data recorder. In addition to the specific instrument product throughput, there is also to consider the support operation of the instrument for power, thermal control and the mode related operational characteristics. The functional block diagram of the LAT System is shown in Figure 1. L1 DAQ SIU ACD EM ACD GLOBAL TRIGGER 16 CAL 16 CAL TKR/CAL EM (16) Event Data 1 Trigger TKR TKR Housekeeping Subsystem Power Control Thermal Control Trigger Control Logic 1553 Timing Reference GBM Alert Discretes L 3 Data L 3 Data (30 Hz) LAT COMM 5 Solid State Recorder L2-L3 TEM Buffer TEM TEM TEM TEM Buffer #1 TEM Buffer Buffer Buffer Buffer #16 Spacecraft Data Bus (1553) Front end Spacecraft Bus I/O Figure 1 LAT System Functional Diagram Monitor, Control & Distribution Level 1 Data Power Level 1 Data 10 Khz Reconfigurable Software Filters Processor Farm Survival Heaters Spacecraft Power Bus Event Builder LAT Heaters 2,3,4 Overall Goals The goal is risk mitigation and an E2E test should address and retire as best as possible the identified performance risks (see Figure 1) that are: Characterization and performance metrics for: o Trigger Rate (1) o Data Volume (2) o Event Distribution (3) Event Timing & Pile-up Effects (4) E2E Throughput Performance (5) DRAFT Planned Verifications As the program matured, tests were planned by the Electronics and Flight Software Subsystem that address the identified risk areas. Plans to perform tests to address the identified risk areas were planned and are documented in the Trigger and Data Flow Test Plan and the Flight Software Test Plan. There is a need to evaluate the planned verifications to evaluate how comprehensive the planned tests are and, to propose additional tests that will reduce system performance risk. Independently, the project must determine the adequacy of these tests and the extent to which they reduce the risks to the program. [AGAIN, HOW DO WE KNOW THESE ARE ADEQUATE?](Steve Ritz) Trigger Rate Trigger rate may be simulated in the test bed using event simulators programmed to trigger the Tower Electronics Modules in the Test Bed. These event simulators will drive each TEM, replacing the Tracker and Calorimeter instrument front ends. Event Simulators may be programmed to provide Tracker and Cal stimulus to each TEM in a manner that will allow determination of TEM saturation and trigger overload. Data Volume Data Volume may be simulated in the test bed using event simulators programmed to trigger and then provide simulated sensor data to the Tower Electronics Modules. The event simulators will provide to each TEM simulated data from then sensors. Event Simulators may be programmed to provide Tracker and Cal data products to each TEM in a manner that will allow determination of TEM data throughput capability at various volumes and trigger rates. Event Distribution The coordination of multiple event simulators to allow a distribution of events to the TEM modules thereby permitting natural triggering and equivalent data transfer volumes within the LAT DAQ front end is outside the present scope of the Test Bed effort. The test bed effort is presently bounded by those tests and verifications called out in the Flight Software Verification plan and the Trigger and Dataflow verification plan. Event Timing & Pile Up Effects Event timing may be simulated in the test bed using the event simulators programmed to scenarios which have been determined to provide the border conditions to the system which cause events to stress timing and create a “pile up” of data in the trigger and dataflow sub-system. End to End Performance The LAT T&DF sub-system is a scaleable design. It is the position of the system designers in the Electronics and Flight Software Subsystem that the results achieved in the test bed may be scaled to extrapolate system performance. Instrument throughput may be determined through the creation of scenarios that provide stimulus and data volumes that explore the system design boundary conditions. These scenarios will DRAFT permit system performance verification performance from the TEM level to the Solid State Recorder output interface. Additional Efforts Van de Graf Testing Testing of instrument front-end detectors in the Van de Graf test will provide important information regarding detector response to stimulus. By extrapolating the output of this test will provides support data to allow the test bed event simulators and their driver models to provide realistic stimulus to the TEM electronics. Beam Testing Testing of the instrument Calibration Unit will provide valuable insight into system throughput at a ¼ scale. Since the system is scalable, this data will support other laboratory and bench level testing in the determination and prediction of instrument performance. Use of LAT detector elements in the beam test allows for further refining of event stimulator models and allows highly accurate performance predictions. This information will also be used to adjust and refine the event filters in the initial load of flight software prior to launch. Further refinements will take place on orbit during operation to permit refining of the filter algorithms. OPTIONS Airplane Test Option for Risk Mitigation An airplane test, where the LAT is operated during airborne transport to characterize the performance of the fully operating LAT to natural background particle flux stimulus has been proposed. Background particle fluxes at an altitude easily reached by a commercial aircraft would be more than sufficient to stimulate the LAT (the background rate is actually somewhat lower on orbit). This proposed test would provide adequate event distribution for validation of the detector front end (DAQ) along with the data processing capability of the Trigger and Dataflow subsystem. It is, in effect, as close to an orbital functional test as the project could hope to achieve short of an actual orbital flight. Program Impact This test does not come without risks. Besides subjecting the LAT to an additional set of environmental requirements (which it may be qualified for, but that must be verified) it also subjects the support equipment to the additional environment. Equipment Qualification The airplane test certainly would require qualification of the GSE, both electrical and Mechanical. Qualification of the support equipment would be similar (if not identical) in scope to the requirements for equipment, which flew on the Kuiper Observatory or the SOFIA observatory that is presently being integrated. This effort on the GSE is additional scope to the program and there may not be enough schedule time to qualify all of the support equipment. DRAFT Qualification of support equipment for an airborne test is critical to a successful test. The equipment must be capable of operating for the requisite period of time under a shock/ vibration environment that typically is outside the normal operational envelope for laboratory equipment. The environment shown in the figure is Typical of an airborne shock/vibration environment for a piece of rack-mounted equipment. To successfully support an airplane test, the EGSE/MGSE required to support he LAT must be qualified to meet the test mission. Sample equipment Random Vibration Requirement (typical C-17) I. Random Vibration 1. Direction: Aircraft Vertical 2. Duration: 8 hrs 3. Configuration: As mounted in rack, un-isolated. DRAFT Risks associated with an Airplane Test The proposed “Airplane Test” would operate the LAT while it was being transported by air. LAT is not designed to “operate” during a dynamic vibration environment (Risk to LAT Flight Hardware) o Additional scope in analysis/qualification of designs to operate in this new environment (Risk to cost and schedule) o Physical risk of damage to unique flight hardware (Risk to LAT Flight Hardware) LAT GSE not qualified for airborne operation (Risk to cost and schedule) o Additional scope in GSE (Risk to cost and schedule) o Additional GSE to accommodate test (Risk to cost and schedule) Certify system for airborne operational dynamics and thermal environments (Risk to LAT Flight Hardware, cost and schedule) Development of unique GSE for the test configuration (Risk to cost and schedule) Ground Based Test Option for Risk Mitigation Any ground based testing would be unable to stimulate the LAT as well as the natural environment. While we may determine “static” system performance, we cannot provide natural event stimulation so the instruments dynamic performance may only be partially catalogued. LAT does not “operate” as expected (Risk to Mission Objectives) Summary The main elements listed below outline how the program may be impacted by the E2E test options. Scope Change LAT Airplane Test LAT Scope change: Multiple Mission instrument profiles To qualify for the airplane test, the LAT would now need to be qualified for two mission profiles. The on-orbit mission profile and the airplane mission profile. Ground Based Test No LAT scope change; Expanded ground test scope Expanding ground test scope would allow for inclusion of additional performance testing under E2E simulation conditions. Also required would be the development of techniques that would allow the LAT to be stimulated closer to natural particle event distributions as observed by the ballon flight. DRAFT Mission Impact The LAT would become a multi-mission system with the second mission only a test mission. It must now be qualified for the second operational mission. At a minimum, All LAT subsystems should be analyzed for operation in the Airplane Test environment. Ground based test requires no such re-qualification of flight hardware and, requires no special qualification of GSE. Schedule Qualification efforts for GSE and validation testing of the airborne LAT configuration with GSE would add to the existing schedule. Additional access to the aircraft of approximately 2 weeks (aggressive) for configuration of the test equipment and outfitting of power and environmental equipment would be required. This effort would need to match the Lat schedule perfectly to not impact the program schedule in some manner. Risk The Airplane test puts at risk one-of-a-kind hardware which if damaged, may be difficult to repair and, may require extensive regression testing to return to flight worthy condition. There is risk that the results of the test are open-ended and may not provide a determinate answer. This would either then require further testing or, result in all efforts expended on the test negated. Ground based test puts no flight hardware at risk in the proposed scenario. This testing is also more deterministic with quantifiable results. It does not however, provide insight to the environmental unknown. Recommendation Adopt “Expanded” Ground Based Test option This option provides the lowest exposure to the program and, the most benefit. Efforts on the ground would produce results, which contribute to the design and engineering verification of the instrument. Expand the ground based test option to include as much dynamic stimulus testing as possible. Items for further Investigation The LAT program should investigate potential modeling of a more “dynamic” particle environment to use as test bed scenarios, thereby providing insight of “Event Distribution” effects. SLAC should review the necessary effort to provide a higher definition stimulus to the TEM modules.