GLAST LAT Project
October 23, 2003
GLAST Large Area Telescope:
Gamma-ray Large
Area Space
Telescope
Tracker Subsystem
WBS 4.1.4
GTRC Review
November 14, 2003
Robert Johnson
Santa Cruz Institute for Particle Physics
University of California at Santa Cruz
Tracker Subsystem Manager
rjohnson@scipp.ucsc.edu
GTRC Mini-Review
Tracker, WBS 4.1.4
1
GLAST LAT Project
October 23, 2003
GTRC Specifications
The GTRC serves as the interface between a front-end electronics
module (MCM) and other MCMs and the data acquisition (TEM).
24 64-channel amplifier-discriminator chips for each detector layer
Data flow
GT FE
GT FE
GTRC
Control signal flow
GTRC
GTRC
Control signal flow
Control signal flow
GTRC
Nine detector layers are read out on each side of each tower.
GTRC
GTRC
9-99
8509A22
GTRC Mini-Review
Data flow to FPGA
on DAQ TEM board.
2 readout
controller chips
for each layer
Tracker, WBS 4.1.4
Data flow to FPGA
on DAQ TEM board.
2
GLAST LAT Project
October 23, 2003
GTRC Specifications
• The detailed specification is LAT-TD-00170.
• An FPGA could not be used for reasons of space, power, and the need for
differential I/O to avoid disturbing the amplifier chips.
• Some notable features:
– 20 MHz clock; 2.5 V operation
– All I/O is LVDS and serial
– Configuration register that can be set and read back nondestructively
– Zero-suppression of the GTFE data; memory for 64 hits
– Buffering of all signals and clock to and from the GTFE chips, including
the trigger signals, commands, data, etc.
– Calculation and quadruple buffering of the TOT of the layer-OR trigger
primitive
– Double buffering of the data: allows collection of data from GTFE chips
while sending the previous event to the TEM
– Token-driven daisy-chain readout
– Parity checking on all command and data transfers
– Trigger number checking to flag mixed events
GTRC Mini-Review
Tracker, WBS 4.1.4
3
GLAST LAT Project
October 23, 2003
GTRC Description
• All digital
• Tanner standard-cells, except for
• LVDS I/O cells.
• SEU hardened configuration
register.
• RAM (64 hits, 2 buffers)
• Design in VHDL; synthesis, auto place
and route.
• Agilent 0.5 micron 3-metal process
NTOKEN_OUT
NSDATA_IN
I/O
EVENT
MEMORY
0
EVENT
READOUT
CONTROL
EVENT
MEMORY
1
GTFE
READOUT
CONTROL
RD_IN
TOT
NTREQ
FASTOR
I/O
TREQ_IN
I/O
NTACK
TRIGGER
TACKB
REGISTER
READBACK
CONTROL
CTRLREG
NSCMD
SCMDOUT
CLK
CLKB
CMD
DECODER
NRESET
CONTROL
REGISTER
RESETB
I/O
NSDATA
TOKEN
GTRC Mini-Review
Tracker, WBS 4.1.4
4
GLAST LAT Project
October 23, 2003
GTRC History; 3 Generations
• BTEM version (1999), designed by Gerrit Meddeler in the HP 0.8
micron process. New LAT DAQ requirements and interface
necessitated a major redesign. None of the old logic code or layout
was retained.
• GTRC V1: functional, with flip-flop memory, but several bugs
• GTRC V2: nonfunctional due mainly to clock routing problems
• GTRC V3 and V4: simultaneous submissions. V3 had RAM and V4
used flip flops (only 32 hits). The V3 is the one reviewed last
December at SLAC.
– Lacking 4 buffering of the TOT (won’t align with events)
– Failed to report some parity errors
– Slow LVDS receivers
• GTRC V5: interim prototype submission, parasitic on another run,
before complete testing of the V3 was done, to fix the TOT and parity
• GTRC V6: “flight submission” dedicated run; included the fixes in V5
plus speeding up of the drivers and receivers.
GTRC Mini-Review
Tracker, WBS 4.1.4
5
GLAST LAT Project
October 23, 2003
Tracker Electronics Test Systems
• Wafer Testing (GTFE and GTRC)
– Procedure documented in LAT-PS-1250
– Test vectors documented in LAT-TD-247 and LAT-TD-248
– Verifies LVDS levels and power consumption as well as function
– The GTRC vectors (LAT-TD-248) have been augmented with a test used
to verify the GTRC V6 TOT bug
• MCM Testing
– Procedure documented in LAT-PS-1971
– Test vectors documented in LAT-TD-249
– Verifies LVDS levels, power consumption and leakage, all functionality,
and limited performance testing (no SSDs)
– Tests different frequencies and different VDD levels
• Burn-in System (documentation in progress, LAT-TD-2367)
– Tests a complete tower of MCMs together with TEM
– Includes thermal cycling for environmental acceptance tests
– Repeatedly executes a set of test vectors while at elevated temperature
GTRC Mini-Review
Tracker, WBS 4.1.4
6
GLAST LAT Project
October 23, 2003
GTRC V6 Design Problems
• GTRC Time Over Threshold:
– Logic in existing chip is flawed (causes frequent DAQ time-outs)
and can only be repaired by repeating the production. The
existing V6 chip can only be used with the time-over-threshold
disabled.
– The problem occurs when a second trigger is received in
coincidence with the falling edge of the time-over-threshold.
– This was verified both in VHDL and on bare die driven by the
wafer probing system.
– The fault was not caught until a complete system was put
together and run on cosmic rays. It would be unlikely to be
noticed without a random event source and lots of events.
– The TOT algorithm can be done in the TEM ASIC (GTCC). This
was tested in the FPGAs of an EM TEM. There are even some
data throughput advantages to doing it in the TEM.
GTRC Mini-Review
Tracker, WBS 4.1.4
7
GLAST LAT Project
October 23, 2003
GTRC V6 Design Problems
• GTRC to GTRC data transfer and timing margins:
– The data are output from one GTRC on the falling edge of the
clock (a clear mistake in retrospect) and were supposed to be
captured in the next GTRC on the rising edge of the next clock
25ns later. Simulations during the design phase indicated no
problem with this, but they did not include enough of the parasitic
capacitance.
– At 20 MHz the system was running well, but only by consistently
missing the data on the first clock edge and capturing it 50ns
later. The problem was seen when the frequency was lowered.
– Our test plan was deficient in not looking much more closely at
this data transfer early on. Even with the V3 we looked at some
different frequencies but did not stumble upon the problem.
– The observed internal delays are now understood in simulation.
– The design is easy to fix. Big margins at 20 MHz can be achieved
by simply removing the clock inversion, but schedule…
GTRC Mini-Review
Tracker, WBS 4.1.4
8
GLAST LAT Project
October 23, 2003
GTRC Data Transmission
To GTFEs
A
CLK
C
GTRC-1
GTRC-0
B
D
Q
D
Q
D
Q
D
Q
This inverter drives several flip flops in
widely separated core locations, resulting in
a significant delay of data output.
• Measured ~28 ns delay from A to B is well understood now in
terms of agreement between simulation and measurements.
• The ~19 ns delay from A to C also agrees well with simulations.
GTRC Mini-Review
Tracker, WBS 4.1.4
9
GLAST LAT Project
October 23, 2003
Impact of GTRC Timing Problem
• Using the existing chips:
– At 20 MHz and 2.5V they skip a clock on each transfer.
• Not a problem in itself (same timing as would be achieved by
output on the rising edge), BUT
– Fails if the frequency is lowered to 19 MHz.
– Fails at 20 MHz if the voltage is raised.
– It works properly up to about 15 MHz at 2.44V, or to higher
frequency if the voltage is raised (but insufficient power to fix the
problem).
– The frequency limit of correct operation can be raised a little by
lowering the termination resistance, with 200-ohm external
resistors placed in parallel with the existing internal 700 ohms.
• Requires 16 resistors on each flex-circuit cable (not difficult or
expensive).
• Still cannot achieve 20 MHz with good margins.
GTRC Mini-Review
Tracker, WBS 4.1.4
10
GLAST LAT Project
October 23, 2003
Timing Margins for GTRC V6
fr equencymar ginsatVDD2.29V( onthechip)
fr equencymar ginsatVDD2.56V( onthechip)
2.29 V
t wo clock cycles delay
t wo clock cycles delay
20
not wor king
15
Proper operation
10
2.56 V
26
Improper operation
fr equ ency in M H z
fr equ ency in M H z
25
21
not wor king
16
11
one clock cycle delay
one clock cycle delay
6
5
-23
-15
-5
5
15
25
temper atur eindegC
35
45
55 60
-23
-15
-5
5
15
25
temper atur eindegC
35
45
55 60
• Measurements made on a string of 9 MCMs (i.e. one tower side) connected
to a TEM via burn-in flex-circuit cables, using connectors savers.
• No additional termination resistors have been added to the system.
• The voltages listed here are measured at the chips. The voltage at the TEM
is higher.
GTRC Mini-Review
Tracker, WBS 4.1.4
11
GLAST LAT Project
October 23, 2003
Timing Margins for GTRC V6
fr equencymar ginsat VDD2.84V( onthechip)
26
t wo clock cycles delay
fr equ ency i nM H z
2.84 V
21
not wor king
16
11
one clock cycle delay
6
-23
-15
-5
5
15
25
temper atur eindegC
35
45
55 60
The Agilent process is designed for maximum 3.3V operation. This
slide shows the margins with the maximum voltage we can achieve
with our existing TEM/PS. Probably the margin will continue to
increase if we push up to 3.3V.
GTRC Mini-Review
Tracker, WBS 4.1.4
12
GLAST LAT Project
October 23, 2003
GTRC V6 with 200-ohm Termination
• Adding 200 ohms in parallel with the existing 700 ohm termination
speeds up the rise time of the signal on the cable between GTRC
chips and raises a little the maximum operational frequency.
• We did not finish the complete survey of this condition because
the TEM broke during the measurements and the unencapsulated
test MCMs were losing their wire bonds.
• Safe operation could be achieved at about 16 MHz with the
voltage at the TEM raised from 2.5 V to 2.75 V.
Measurements at 25 °C
Maximum frequency of proper operation.
Temperature
2.45 V at TEM
25°C
Voltage at TEM
Maximum freq.
15.8 MHz
2.46
15.5 MHz
30°C
15.8 MHz
2.61
17.0 MHz
35°C
15.6 MHz
2.76
18.5 MHz
40°C
15.4 MHz
2.93
19.6 MHz
45°C
15.3 MHz
3.00
20.2 MHz
50°C
15.1 MHz
3.08
20.6 MHz
GTRC Mini-Review
2.61 V at TEM
17.2 MHz
16.0 MHz
Tracker, WBS 4.1.4
13
GLAST LAT Project
October 23, 2003
Tracker Power
• Measurements made this week at 20 MHz with the latest MCMs.
• Assuming 4 A/SSD at 120 V bias (total of 4.4 W of bias power).
• CDR allocation: 155 W.
Total Tracker Power
300
Power (W)
250
200
150
100
50
0
2.25V
2.50V
2.75V
3.00V
3.25V
VDD (V)
GTRC Mini-Review
Tracker, WBS 4.1.4
14
GLAST LAT Project
October 23, 2003
Mini-Tower Hit Efficiency
From Hiro Tajima, measured using cosmic rays under 3 different trigger
conditions. 5 layers are used for tracking (exactly 5 clusters required, with
straight track in the view with 3 hits); the 6th layer is used to find the hit
efficiency, including cutting away from dead areas between wafers.
The peak amplifier pulse height occurs at about 1.0
to 1.5 s TACK delay.
For the EXT trigger (scintillator), zero on this scale
is about 0.9s after passage of the particle.
GTRC Mini-Review
Tracker, WBS 4.1.4
15
GLAST LAT Project
October 23, 2003
Impact of Loss of TOT
• While not originally thought to be necessary in order to meet our
science requirements, the TOT has recently been found to be
important in greatly reducing a troublesome background source:
– Cosmic ray hits the calorimeter from below or from the side.
– A proton or heavy ion exits the calorimeter, enters the Tracker,
and stops several layers up in the Tracker.
– This topology can look very much like a photon conversion, but
the stopping ion produces a very large ionization that can be
readily distinguished from relativistic electrons.
GTRC Mini-Review
Tracker, WBS 4.1.4
16
GLAST LAT Project
October 23, 2003
TOT Analysis (Simulation)
TOT
Asymmetry
between 1st
3 and last 3
planes.
100 MeV Gammas
Albedo Protons
TOT
Average
GTRC Mini-Review
Tracker, WBS 4.1.4
TOT here is
truncated to
250 counts by
GTRC
17
GLAST LAT Project
October 23, 2003
TOT Analysis (Simulation)
The asymmetry is not useful with
the 50 microsecond truncation
imposed in the GTRC design.
But the average TOT still gives
good separation, only about 10%
worse than with no truncation.
GTRC Mini-Review
Tracker, WBS 4.1.4
18
GLAST LAT Project
October 23, 2003
Background Analysis (Simulation)
Remaining Background
Analysis of 25M MC Mixed BKG Events
(Bill Atwood)
Generated cos
3 Classes of Background Events Remain:
• Range-outs from below (.04 Hz)
• Horizontal Events (.004 Hz)
• ACD Leakage and inefficiency (.04 Hz)
Elimination Strategy
Measured cos
Aeff & Background Rate:
Aeff = 8400 cm2 on Axis (E > 3 GeV)
Aeff x DW = 2.0 m2-str
BUT....
Background Rate 4-5 times too high
GTRC Mini-Review
1) Range-outs
- ToT Identification in Tracker - kills > 90%
- MIP Identification in CAL - should kill > 50%
2) Horizontal Events - should kill > 50%
- Edge CAL hits
3) ACD Leakage
- Events found accurately;
- Cover cracks with Tapes - should kill > 95%
Estimated Rate after this to be < .006 Hz
(about 3% residual background in EGD signal)
Tracker, WBS 4.1.4
19
GLAST LAT Project
October 23, 2003
New GTRC Designs
• GTRC V7: the new flight design
– TOT algorithm rewritten to fix the bug
– Clock inversion removed for the data output, token output, and
trigger-request output
– Recompiled logic core; no changes outside the logic core
• GTRC V6b: backup design in case V7 fails
– Clock inversion removed by hand edit of the layout and
schematic; verified by LVS
– Essentially identical to the V6 chips modified by FIB (see below)
GTRC Mini-Review
Tracker, WBS 4.1.4
20
GLAST LAT Project
October 23, 2003
GTFE V7 Design Verification
• VHDL simulation of the logic core. Note that there are no changes
between V6 and V7 outside of the logic core.
• Recompiled logic core is grafted into the old V6 layout, replacing only
the old logic core.
• Usual LVS and DRC of the new layout.
• Nanosim simulation of the complete extracted netlist.
• Verification of the logic core in FPGAs
• FIB removal of the culprit inverter in 6 GTRC V6 chips and tests on
MCMs
GTRC Mini-Review
Tracker, WBS 4.1.4
21
GLAST LAT Project
October 23, 2003
Test of FIB Patched GTRC
• Layout was modified
by ion beam to send
data and token out on
the rising clock edge.
GTRC-1 Data Out
• These scope traces
show transmission of
register readback data.
1 MHz
GTRC-0 Data Out
Data output from
GTRC 1 on this edge.
Data output from
GTRC 0 on this edge.
Clock In
Data captured by
GTRC 0 on this edge.
GTRC Mini-Review
Tracker, WBS 4.1.4
22
GLAST LAT Project
October 23, 2003
Test of FIB Patched GTRC
Note the long time constant from the cable
capacitance times the 700 ohm impedance.
20 MHz
GTRC-1 Data Out
GTRC-0 Data Out
Data output from
GTRC 1 on this edge.
GTRC Mini-Review
Data output from
GTRC 0 on this edge.
Clock In
Tracker, WBS 4.1.4
23
GLAST LAT Project
October 23, 2003
Test of FIB Patched GTRC
• Three of the patched GTRC chips were operated on separate MCMs
and read out in a chain using burn-in cables with connector savers.
• All frequencies above 5 MHz were tested (in 0.1 MHz steps).
• Both register readback and data (charge injection) are tested.
• The results below are at room temperature, but the test was also
carried out from –20°C to +60°C with very similar results.
• The upper frequency limit is consistent with the internal limitations of
the MCMs.
Voltage on MCM at Chips
Maximum Frequency of
Proper Operation.
2.29 V
25 MHz
2.56 V
27 MHz
2.84 V
29 MHz
GTRC Mini-Review
Tracker, WBS 4.1.4
24
GLAST LAT Project
October 23, 2003
Test of FIB Patched GTRC
• The string of 9 MCMs, including those with FIB patched GTRC chips,
was taken to SLAC and operated with the new TEM that has ASIC
cable controller chips (GTCC).
• This system also operated at all frequencies from 1 MHz to about 29
MHz.
• Again, the limit is the communication within the MCM, not the
GTRC/GTCC communication.
• We also repeated frequency margin tests at UCSC on individual
MCMs to verify that they operate at all frequencies from 1 MHz to
close to about 29 MHz.
GTRC Mini-Review
Tracker, WBS 4.1.4
25
GLAST LAT Project
October 23, 2003
Test of V7 VHDL Code
• We have installed the updated VHDL code into 3 FPGA test boards.
Each test board has 2 FPGAs, which play the GTRC role, and 2
GTFE amplifier chips.
• Two FPGA test boards are being operated with the TEM system and
the GTFE chips, and no problems have been seen so far with
reading back registers and charge-injection “data”.
• The third FPGA test board was modified to be operated directly from
a COM card (VME I/O card), with the GTFE chips disconnected.
This allows us to input fake “GTFE data” into the GTRC FPGA
directly from the VME. In this way we can execute the complete set
of GTRC wafer test vectors (see LAT-TD-00248) through the new
VHDL code.
– No problems seen in execution of the test vectors.
– The TOT problem seen in V6 chips does not show up in this test.
GTRC Mini-Review
Tracker, WBS 4.1.4
26
GLAST LAT Project
October 23, 2003
GTRC V7 Test Plan
• Wafer testing at UCSC (start immediately upon receipt of wafers)
– Complete system exists and is flight-production qualified (and will
still be within the 1-year calibration time frame).
– We will get it running smoothly again before arrival of the new
wafers, using an old wafer and the old wafer map.
– However, a new wafer map must be carefully prepared in advance.
– All of the test vectors (LAT-TD-00248 plus the new TOT test) have
already been run through the V7 logic code in the FPGA, so this test
is very unlikely to turn up any logic bugs.
– Test and ink enough wafers for all of GLAST in a day or two.
– We will test both V7 and V6b dice, just in case.
• Wafer dicing and inspection at GDSI
– Get PO in place and grease the skids in advance.
– No change with respect to the previous procedure used on V6,
except that the reticle layout is new.
– It should be possible to complete this within a week.
GTRC Mini-Review
Tracker, WBS 4.1.4
27
GLAST LAT Project
October 23, 2003
GTRC V7 Test Plan
• Dice 1 wafer immediately (by MOSIS?).
• Borrow Mike Huffer’s MCMs, which are not encapsulated.
• Put V7 chips on 1 MCM and test it.
– One of two test system, identical to the test system presently at
Teledyne for MCM production testing.
– Test vectors are documented in LAT-TD-00249.
– This should take less than a day.
• Put V7 chips on 8 more MCMs and install a full readout string of 9
MCMs into the burn-in system.
– Uses the standard TEM-based EGSE system.
– All of the hardware exists now, but work is still in progress to
complete the set of test vectors.
– Test over the full frequency, voltage, and temperature ranges.
• The burn-in system is already set up for thermal cycling.
GTRC Mini-Review
Tracker, WBS 4.1.4
28
GLAST LAT Project
October 23, 2003
GTRC V7 Test Plan
• Replace the GTRCs on the rest of Huffer’s 36 MCMs with V7 chips.
– Test each one with the MCM test system.
– Install all 36 back into the test setup of the electronics group for
further testing.
• Meanwhile, MCM production at Teledyne is proceeding with V7
chips, just as soon as the wafers are diced.
– Rush the first lot of completed, tested, burned-in flight MCMs to
Italy for integration onto trays.
– The first n trays assembled go into the stacked-tray test system,
where they are coupled to the EGSE by flex-circuit cables.
• Here the new chips see cosmic-ray data for the first time.
• This test system can execute practically the complete set of
tower test scripts before tower assembly takes place and even
without a complete tower of trays.
GTRC Mini-Review
Tracker, WBS 4.1.4
29
GLAST LAT Project
October 23, 2003
GTRC V7 Test Plan
• Radiation Testing
– Repeat TID testing using the new V7 chips.
– Repeating the heavy-ion testing should not be necessary, but if
required, we could do it again in Italy (for SEU testing).
• Qualification Testing
– We will move forward with qualification testing using MCMs from
the preproduction run presently in progress (GTRC V6 chips).
• Thermal cycling
• Extended burn-in
• Vibration testing
• DPA
– This will have to be repeated with MCMs from the flight lots.
GTRC Mini-Review
Tracker, WBS 4.1.4
30