GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
GLAST Large Area Telescope:
Gamma-ray Large
Area Space
Telescope
I & T Input to Monthly
Technical/Cost/Schedule Review
05/24/05
Elliott Bloom
SU-SLAC
Subsystem Manager
Ken Fouts
SU-SLAC
Subsystem Engineering Manager
elliott@slac.stanford.edu, kfouts@slac.stanford.edu
650-926-2469
650-926-2553
4.1.9 - Integration and Test
1
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Last Month's Accomplishments (1 of 3)
• Management
– Supported Observatory I&T Planning Meeting at GD.
– Procedure Status
– 65 of 68 I&T documents in review/approval cycle.
– Tracking hardware shortages for LAT integration.
• Current shortages:
– GRID HTR PWR HARNESS ASSY, LAT-DS-05057
– GRID HTR TEMP SENSOR HARNESS ASSY, LAT-DS-03662
• IFCT
– Validated EGSE in the grid with EM TEM’s prior to connection to
flight HW.
– Completed Two-Tower Test
– Completed I&T Receiving Tests on TKR 1, 2 and 3.
– Completed Bay 5 Tower (TKR/TEM/CAL) installation
– Completed Bay 1 TKR Installation
– Installed and crimped Grid thermisters and thermocouples.
4.1.9 - Integration and Test
2
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Last Month's Accomplishments (2 of 3)
•
Online
– E-logbook
– Added ability to search shift logbook with Google Desktop
– Implementing requested improvements
– Beyond 2 towers
– Housekeeping multi-pen strip chart GUI
– Parallelizing INT scripts
– Attempting to move some LATTE 5 features into production (e.g.,
HSK, LATc)
– Continued supported of CAL, TKR, TRG, INT and test suites.
– Supported data taking, trigger and E2E script development.
– Problem assessment and fixes are a large part of online workload
– NCRs, support for clean room operations, support of subsystems
scripts.
– CCB activity and JIRA maintenance
4.1.9 - Integration and Test
3
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Last Month's Accomplishments (3 of 3)
• SVAC
– Successful turn around for calibration of two tower data
– ready before data taking started
– Joint effort TKR/CAL/SAS/SVAC
– Offline data processing system under CCB and working
well
– Single tower data analysis
– No show stoppers yet
– Generated several special test requests
– Good start with two tower data
– Data is being analyzed in details
– No major problem identified so far
4.1.9 - Integration and Test
4
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Power Outage Impact on I&T
•
•
•
•
•
Emergency Generator allowed work to continue in B33
TKR 3 was installed on Thursday
Room 103 circuit breaker/panel fault was resolved by turning off non-essential
power.
The current plan to resolve the B33 Room 103 power issue is as follows:
– SEM will procure several 150A breakers to replace the current main
breaker in the power box as time (and testing) permits.
– Several temporary power drops will be added to the office area on a
separate circuit so that computers can be powered up and desk lamps can
be used in the desk and board area outside of the clean room.
– SEM will verify the power panel arc flash calculations, write and execute
the procedures to open the panel for inspection and load measurements in
the current state (only 6 out of 42 breakers on).
– At the completion of TVAC on Wednesday, SEM would begin work to
replace the main breaker and determine the panel loads in more detail as
the remaining breakers are turned back on.
• 4 hour shutdown is planned for Wednesday.
Our goal is to safely bring the building back to 100% with minimal impact to
building users.
4.1.9 - Integration and Test
5
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T MGSE/Grid Activities
Tower A & B in the Grid
Mass simulators installed
EM PDU and GASU
EGSE shelf and cable
service loop
EGSE harness routing
Harness service loop
4.1.9 - Integration and Test
6
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T NCRs by Month Open/Closed Status
18
16
Count of Open/Closed Status
18
Count of Open/Closed Status
16
14
Previous
Month
14
12
10
Open/Closed Status
12
Current
Status
Open
Closed
8
6
10
Open/Closed Status
4
Open
Closed
2
8
0
Sep-04
Nov-04
Dec-04
Jan-05
Feb-05
Mar-05
Apr-05
Date by Month
As of May 23rd
6
4
2
0
Sep-04
Nov-04
Dec-04
Jan-05
Feb-05
Mar-05
Apr-05
May-05
Month
NCR
Date by
Month Opened
4.1.9 - Integration and Test
7
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Procedure Status
•
•
•
80
68 documents total
65 latest versions submitted for sign-off
59 are released
70
11
11
11
2
2
2
60
40
11
4
11
4
11
4
11
2
2
11
4
11
4
11
11
11
2
2
2
11
11
11
4
4
4
11
11
2
2
11
11
3
4
7
7
7
6
6
5
5
5
3
4
15
15
15
15
15
15
15
15
15
15
30
11
11
2
2
11
10
3
4
3
4
3
4
14
14
14
11
2
9
Depart.
Online
SVAC
Particle
MGSE
Manage.
IFCT-Mech
IFCT-Elec
18
18
18
18
17
17
17
17
17
17
16
17
16
Count of Draft
Started
Count of Draft
Complete
Count of Internal
Review Complete
Count of submitted
for signoff
Count of DCN written
Count of DCN and
Doc sent for signoff
Count of Signed by
Author
Count of Signed by
Elliott Bloom
Count of Signed by
QA
Count of Signed by
Others
Count of Released in
LATDocs
10
Count of Document
Number Assigned
20
Count of Document #
Documents
50
11
0
4.1.9 - Integration and Test
8
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Procedure Status (1 of 6) – Management
4.1.9 - Integration and Test
9
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Procedure Status (4 of 6) – IFCT Electrical
4.1.9 - Integration and Test
10
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Upcoming I&T Events
–
–
–
–
–
–
–
–
–
–
–
–
Two tower test (Complete)
Receiving tests for TKR 2 (Complete 5/13)
Receiving tests for TKR 1 (Complete 5/11)
Receiving tests for TKR 3 (Complete 5/16)
Receive TEM/TPS 1 and 2 ECD 5/9 (Complete 5/19)
TKR 2 in Bay 5 (Complete 5/17)
TKR 1 in Bay 1 (Complete 5/19)
CAL/TEM Installation in Bay 5 ECD 5/24
CAL/TEM Installation in Bay 1 ECD 5/25
4 Tower Tests ECD 6/7
Receive TEM/TPS 3 and 4 ECD 5/16 5/26
Receive TKR 4 ECD 6/9
4.1.9 - Integration and Test
11
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T Detail Schedule (1 of 4)
4.1.9 - Integration and Test
12
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T Detail Schedule (2 of 4)
4.1.9 - Integration and Test
13
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T Detail Schedule (3 of 4)
4.1.9 - Integration and Test
14
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T Detail Schedule (4 of 4)
4.1.9 - Integration and Test
15
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T 6 Month Look Ahead (1 of 2)
4.1.9 - Integration and Test
16
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
I&T 6 Month Look Ahead (2 of 2)
4.1.9 - Integration and Test
17
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Issues & Concerns
ISSUES
• Availability of flight assembly hardware.
– Shortages identified by I&T and being worked with
Subsystems.
• Flight Cables
CONCERNS
• Continued support for NCR closure.
• Resolution of Room 103 power issue.
• TPS rework and replacement, availability for future tower
installations.
• Delivery dates for third layer ELX boxes and TKR.
• LATTE 5.0 ready Oct. 1 for use in LAT Functional testing due to
start 10/14. E2E tests start 11/1.
4.1.9 - Integration and Test
18
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
“Issues” for single tower tests: offline analysis
Closed
•
•
•
•
•
•
•
•
•
•
•
Under investigation
CAL_HI and CAL_LO fired without any crystal hits
Investigation has not started
–
Most likely due to direct deposition in diodes
CAL_HI trigger fired without high energy deposition
–
Most likely due to direct deposition in diodes
Discrepancy between TKR occupancy on top GTRC’s between DATA and MC in electronics space (generated STRXX)
–
Most likely due to the differences in particle composition between DATA and MC
–
Maybe a small fraction could be from CC FIFO buffers filled up in data (this is a feature not a bug)
Discrepancy between Data and MC for TKR dead channel list
–
Partially disconnected channels are not simulated
Discrepancy calculated and measured event size (generated STR3)
–
Miscalculations based on wrong assumptions for event multiplicities
Discrepancy between GEM and TriggerAlg for trigger types
–
Analysis software (TriggeAlg) is not a realistic representation of GEM since it does not address threshold/calibration
differences
Discrepancy in trigger rates for 1,5,10 and 20 KHz runs, underestimated for low rates and overestimated for high rates at a few
% level
–
Rates from generator were not known to better than a few %, so it is consistent with measurements
–
overestimation had to do with deadtime since some events did not make into the stream
Compared events that triggered BOTH the TKR and EXT pulse generator running at 20KHz with events that trigger TKR only,
former had lower hit multiplicity (generated STR8)
–
Reason was that when a trigger window was opened (on raising edge) by EXT the TKR signal was already on for some
time which means some of the hits already died away by the time we looked at. Expect this to be more frequent a higher
rates
Distribution for TKR trigger arrival times was not flat
–
Needed to select EXT trigger to open the window and then make the same plot and naturally the spike disappears
There is consistently 10-15% more triggers seen during the 5 and 10kHz runs than expected as measured by the exponential fit
to the delta event time register of the GEM.
TKR only triggers is around 85% (TBR) of the time when the muon telescope triggers
4.1.9 - Integration and Test
19
GLAST LAT Project
4.1.9 - Integration and Test
Technical/Cost/Schedule Review 05/24/05
The Oscar goes to…
20
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Two Tower Tests Baseline Run : Flight Configuration
Reconstructed
positions
79% single track events
14% two-track events
4.1.9 - Integration and Test
Calibrated Energy Spectrum
Expected to peak at 100 MeV
There are 234664 triggers
There are 0 events with Trigger Parity errors
There are 0 events with Packet errors
There are 20 events with TEM errors
Time of first trigger: Thu May 5 20:48:41 2005 (GMT)
Time of last trigger: Thu May 5 21:48:42 2005 (GMT)
Duration: 3601 seconds
Rate: 82 hz
21
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Time Between Events: Baseline Run
As expected 26.5 us !
4.1.9 - Integration and Test
22
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Two Tower Tests Baseline Run : Reconstructed Z Direction
From Anders…
Tracks passing through
1 tower only
Maximum trigger acceptance is reduced
and is consistent with expectations based on
geometrical calculations
Tracks passing through
both towers
Tracks coming from top of the tower
Cos q = -1
Selection cuts:
•Single track events
•Ratio between predicted and
measured MIP > 0.7 and < 1.3
4.1.9 - Integration and Test
23
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Two Tower Tests Baseline Run : Sum of Crystal Energies in the CAL
Tracks passing through
1 tower only
Energy in MeV
Energy calibrations work within a
tower and across towers
Tracks passing through
both towers
Selection cuts:
•Single track events
•Ratio between predicted and
measured MIP > 0.7 and < 1.3
Energy in MeV
4.1.9 - Integration and Test
24
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Two Tower Tests Baseline Run : MC vs DATA
From Dave…
Tower 0
4.1.9 - Integration and Test
Tower 4
25
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Science Data vs. Housekeeping Data
Tower 0, bottom layer
Temperature
Hit multiplicity
Hit multiplicity
Temperature
From Xin…
Tower 0, top layer
Tower 4, bottom layer
Hit multiplicity
Temperature
Time
Temperature
Hit multiplicity
Time
Tower 4, top layer
Time
Time
4.1.9 - Integration and Test
26
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Priorities for Data Analysis for 2 towers
•
We proposed to the Collaboration to analyze these runs first
– We are identifying names to be attached to the list below
–
1/1 – baseline
– Check trigger types and rates, event sizes , raw and recon distributions for tracks within one
tower and across towers
– Monte Carlo comparison
–
2/1,2/2 – baseline and change PDU PS values
– First time we have a Power Distribution Unit. Could it be a source of noise? If so, can the
raw distributions tells us?
–
2/6, 2/7 – read TKR from Left or Right only
– Cable lengths are different between odd and even numbered towers, but the DAQ takes that
into account. Let’s check it by comparing raw and recon distributions from both towers.
Select events that triggered at the center and at the edges
–
4/1 to 4/4 – baseline with pulse generator @ 1,5,10,20 kHz
– Muon distributions should not be affected by high rates from pulse generator
– Do we understand the deadtime?
–
B2 – nominal settings TEM diagnostics enabled
– It is just like 1/1 but with TEM diagnostics ON
–
B10 – CAL HE muon gain, 4 range readout, TEM diagnostics enabled
– Calibrate TOT
– measure dead strips
– Calibrate edges of crystals
–
B13 – CAL HE muon gain, 4 range readout, Zero Suppression disabled
– Check pedestals in the CAL
– Trend pedestals, muon peaks, log ratios, what else?
4.1.9 - Integration and Test
27
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Wrapping up…
• Good start with two tower data
– Runs finished this morning
– Will distribute data to Collaboration today
• More to come…
– Join us in the Instrument Analysis Weekly Meetings
– Friday 8 am PDT (VRVS)
– Still looking for more people…
– Our web site
http://wwwglast.slac.stanford.edu/IntegrationTest/SVAC/Instrument_
Analysis/Instrument_Analysis.html
4.1.9 - Integration and Test
28
GLAST LAT Project
Gamma-ray Large
Area Space
Telescope
4.1.9 - Integration and Test
Technical/Cost/Schedule Review 05/24/05
Cost/Schedule Reports for
Presentation
April 2005 Month End
4.1.9 Integration & Test
29
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Level 3 Milestone Count
4.1.9 Level 3 Milestones
14
14
Baseline
Forecast
12
12
10
May-06
Apr-06
Mar-06
Feb-06
Jan-06
Dec-05
0
Nov-05
0
Oct-05
2
Sep-05
2
Aug-05
4
Jul-05
4
Jun-05
6
May-05
6
Apr-05
8
Mar-05
8
Feb-05
Cum # of milestones due
10
Months
4.1.9 - Integration and Test
30
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Level 3 Milestone List
AV
A cti vity
ID
NW
DP NO
A cti vity
D escr i ption
B aseline
Finish
-2m
V ar
-1m D eltaB sln
V ar
V ar
E ar ly
Finish
Total
Float
FY 0 5
FY 0 6
FE B M A R A P R M A Y J U N J U L A U GS E P O C T N O V D E C J A N FE B M A R A P R M A Y J U N J U L
Instr um ent Pr oject Office (Level 3)
4.1.9 I&T
9
1 M 9 9 01 0
9
S t ar t 2 T ow er C o m p r eh e n s iv e P e rf o rm a n c e T es t
0 4 /2 0 /0 5
6
4
-3
0 4 /2 5 /0 5 A
9
1 M 9 9 02 0
9
S t ar t 4 T ow er C o m p r eh e n s iv e P e rf o rm a n c e T es t
0 5 /1 2 /0 5
6
1
-7
0 5 /2 3 /0 5
9
1 M 9 9 03 0
9
S t ar t 8 T ow er C o m p r eh e n s iv e P e rf o rm a n c e T es t
0 6 /2 0 /0 5
0
-4
-5
0 6 /2 7 /0 5
39
9
1 M 1 0 01 7 4 0
B
O n li n e F U S / W F in a l R el e as e -I & T t o IO C
0 7 /1 4 /0 5
0
0
0
0 7 /1 4 /0 5
572
9
1 M 9 9 04 0
9
S t ar t 16 To w e r C o m p re h e n s i v e P e rf or m a n c e Te s t
0 9 /0 7 /0 5
0
-4
-4
0 9 /1 3 /0 5
25
9
1 M 1 0 00 1 3 0
9
L A T R e a d y to S h ip t o N R L f o r E nv Te s t
1 2 /2 0 /0 5
0
-9
-4
0 1 /0 3 /0 6
458
9
1 M 1 9 01 0
9
S h ip LA T to N R L fo r E n v T e s t
0 1 /0 3 /0 6
0
-1 3
-6
0 1 /0 9 /0 6
30
9
1 M 1 9 02 0
9
L A T E M I/E M C T e s t
0 2 /0 1 /0 6
1
-1 4
-8
0 2 /0 9 /0 6
29
9
1 M 1 9 03 0
9
L A T S i n e V ib e
0 2 /1 4 /0 6
1
-1 4
-8
0 2 /2 2 /0 6
29
9
1 M 1 9 04 0
9
L A T A c o u s tic Te s t
0 2 /2 4 /0 6
0
-1 6
-1 0
0 3 /0 6 /0 6
29
9
1 M 1 9 05 0
9
L A T TV A C
0 4 /1 4 /0 6
0
-1 5
-9
0 4 /2 3 /0 6
29
9
1 M 1 9 06 0
9
L A T W e ig h t & C G
0 4 /1 7 /0 6
1
-1 4
-8
0 4 /2 5 /0 6
29
9
1 M 1 9 07 0
9
S h ip LA T to S pe c tr u m A s t ro
0 4 /2 1 /0 6
0
-1 6
-1 0
0 5 /0 1 /0 6
29
Run Date
05/23/05 12:41
Data Date
05/01/05
© P ri mavera S ystems, Inc.
4.1.9 - Integration and Test
GLAST LAT PROJECT
Level 3 Milestones
Baseline V ariance (Organized by Subsystem)
43
LT-DA : B aseli ne V ariance
FL-D4: A V : Level 3 Mi lestones
Report #10
S heet 1
31
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Milestone Variance Explanation
• Schedule Impact
– None
• Cost Impact
– None
• Corrective Action
– Not required
4.1.9 - Integration and Test
32
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Cost Report
Unfilled
Estimated Final
Orders
Cost
During Month
Cum. to Date
Detail
Balance of Contractor Contract Outstanding
Actual Planned Actual Planned MAY05 JUN05 Contract Estimate Value
Reporting
Category
Cost Incurred
Estimated Cost
4.1.9 INTEGRATION & TEST
4.1.9.1 I&T MANAGEMENT
45
47 1,062 1,116
4.1.9.3 INSTR OPS COORDINATOR
4.1.9.4 MECH GROUND SUPT EQUIP
31
84
4.1.9.6 INTEG, FACILITIES, CONFIG & TEST
4.1.9.7 PARTICLE TEST
4.1.9.8 ENVIRONMENTAL TEST
654
137
25
4.1.9.9 SCIENCE VERIFICATION, ANALYSIS & CALIBRATION
CAPW[3]Totals:
290
4.1.9 - Integration and Test
167
29
195
1,354
1,354
0
31
31
14
2,098
2,098
654
654
1,093
4,219
4,219
11
5
48
48
4
288
747
747
1,596
300
9,451
300
9,451
654
252 2,762 3,116
0
49
31
20 2,055 1,883
4.1.9.5 ONLINE SOFTWARE
47
43
48
307
383
300 300
486 7,214 7,531
178
74
327
186
78
314
33
0
4
19
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
Cost Variance Explanation
• Why overrun/underrun?
– Currently on plan
• What will be done to correct?
– Stay the course
4.1.9 - Integration and Test
34
GLAST LAT Project
FTE Report Technical/Cost/Schedule Review 05/24/05
(DOE/NASA-funded only)
PLANNED
ACTUALS
4.1.9 Integration & Test
FTEs
25.0
20.0
15.0
10.0
5.0
0.0
P0
SE
5
05
G
5
L0
AU
JU
5
5
N0
0
AY
JU
M
05
05
AR
R
AP
M
5
B0
FE
4.1.9 - Integration and Test
35
GLAST LAT Project
Technical/Cost/Schedule Review 05/24/05
FTE Variance Explanation
• Why overrun/underrun?
– Small overrun due to use of additional SLAC services for
machining
• What is the impact?
– None
• What will be done to correct?
– N/A
4.1.9 - Integration and Test
36