Status of the CSC Trigger
Darin Acosta
University of Florida
Outline
Status of the CSC Track-Finder
Status of the CSC Local Trigger
PHOS4 Experience
TTC Jitter Measurements
CPT Week, April 2002
2
Darin Acosta
The CSC Track-Finder Crate
Track-Finder crate (1.6 Gbits/s optical links)
Muon Sorter
CCB
SR SR SR SR SR SR
/ / / / / /
SP SP SP SP SP SP
MS
BIT3 Controller
Clock and Control
Board
SP 2002 Card
(3 Sector Receivers +
Sector Processor)
(60° sector)
SR SR SR SR SR SR
/ / / / / /
SP SP SP SP SP SP
From MPC
(chamber 4)
From MPC
(chamber 3)
From MPC
(chamber 2)
From
Trigger Timing
Control
From MPC
(chamber 1B)
From MPC
(chamber 1A)
To
Global Trigger
•
•
•
To DAQ
Power consumption : ~ 1000W per crate
16 optical connections per SP
Custom backplane for SP ↔ CCB and MS connections
CPT Week, April 2002
3
Darin Acosta
CSC Track Finder Backplane
Standard VME 64x J1/P1 backplane
A24/D16 (but D32 possible using address lines)
SRSP 12
SRSP 11
Darin Acosta
SRSP 10
4
SRSP 9
Custom GTLP 6U backplane
SRSP 8
SRSP 7
Muon sorter
Clock and control
SRSP 6
SRSP 5
SRSP 4
SRSP 3
CPT Week, April 2002
SRSP 2
Signals
specified,
routing to
commence
SRSP 1
Standard VME
J2/P2 backplane
Sector Processor 2002 Board Layout
DC-DC Converter
Phi Global LUT
Eta Global LUT
Phi Local LUT
EEPROM
EEPROM
EEPROM
Indicators
VME/CCB
FPGA
TLK2501
Transceiver
Front FPGA
From CCB
PT LUT
DDU FPGA
To MS
Main
FPGA
Mezzanine
Card
Optical
Transceiver
CPT Week, April 2002
5
Darin Acosta
TRANSITION
BOARD WITH
LVDS
TRANSCEIVERS
TO/ FROM
BARREL
ME1 SR LUT Triad
FRONT
FRONT
FPGA
FPGA
A18
CLCT PAT# - 4
Q-4
CLCT_ID - 8
L/R -1
C3
A11
PHIL LUT
256K x 18
Flow
Through
SRAM
Phi_L -10
PhiB_L - 6
PhiB_L - 6
Phi_L - 2
CSC_ID - 4
WG_ID - 7
ETAG LUT
512K x 18
Flow
Through
SRAM
PhiB_G - 5
Eta_G - 7
MAIN
MAIN
FPGA
FPGA
CSC_ID – 4
WG_ID – 7
CSC ID - 4
CLK40P1
C3
D16
16 Bit
C2
Trans
ceiver
Phi_L - 10
WG_ID - 5
CSC_ID - 4
PHIG LUT
512K x 36
Flow
Through
SRAM
Phi_G -12
Phi_DT - 12
To DT
C4
CLK40P2
D12
16 Bit
Trans
ceiver
C2
CLK40
Legend: A – Address Lines
D - Data Lines
C – Control Lines
CLK – Clock
CLK40
45 synchronous memories for conversion of 15 track segments
Î >64 MB per board ⇒ Need high VME bandwidth, broadcast
capability to identical chips, and crate broadcast capability to SPs
Î
CPT Week, April 2002
6
Darin Acosta
Main Sector Processor FPGA
9xD24
ME2/
ME2/
ME4
ME4
STUBS
STUBS
9xD5
3xD1
6xD24
ME1
ME1
STUBS
STUBS
6xD9
2xD1
MAIN
MAIN
FPGA
FPGA
C3
MUX
MUX
3xD12 + 4xD1
3xA22
3xC4
PT
PT
LUT
LUT
3xD8
3xD8
DT
DT
STUBS
STUBS
2xD25
D8
TRAN
TRAN
3xD8
3xC1
DDU
DDU
INT
INT
CCB
CCB
&
&
VME
VME
INT
INT
C2
C4
C1
D32
C2
C9
CLK40
A8
C3
CFG
CFG
ROM
ROM
D16
Legend:
G – Number of Signal Groups
GxAn – G Groups of n Address Lines
GxCn – G Groups of n Control Lines
GxDn - G Groups of n Data Lines
TRAN - Transceiver
CCB&VME Int – Combined CCB and VME Interface
CFG ROM – Configuration ROM
CLK40 – Clock 40 MHz
DDU- INT – Readout Interface
Placed on mezzanine card
Î Firmware written in “Verilog++” (see March TSWG talk)
and implemented in ORCA as well
Î Latency only 4 BX
Î
CPT Week, April 2002
7
Darin Acosta
PT LUT & MUX
MAIN
MAIN
FPGA
FPGA
D32
LVTT
LVTT
/GTLP
/GTLP
D8
A22
FRAME
FRAME
22
PT
PT
LUT
LUT
D32
Back
Back
Plane
Plane
C4
A22
D32
LVTT
LVTT
/GTLP
/GTLP
PT
PT
LUT
LUT
FRAME
FRAME
11
C4
A22
PT
PT
LUT
LUT
D32
WIRED MUX
Legend:
C4
Ax – x Address Lines
Dx – x Data Lines
Cx - x Control Lines
/OEAB=CLK40-90
/OEAB=CLK40-270
CLKAB=CLK80
Each Pt LUT is actually two 4M × 4 SRAMs
Î Data multiplexed at 80 MHz onto GTLP backplane to Sorter
Î
CPT Week, April 2002
8
Darin Acosta
DDU FPGA
FRONT
FRONT
FPGA
FPGA55
D32
FRONT
FRONT
FPGA
FPGA44
D32
FRONT
FRONT
FPGA
FPGA33
D32
S1
C2
S1
C2
FRONT
FRONT
FPGA
FPGA22
D32
FRONT
FRONT
FPGA
FPGA11
D32
DDU
DDU
FPGA
FPGA
S1
C2
D16
S1
C5
CLK80
C2
TLK
TLK
2501
2501
S2
S1
C2
MAIN
MAIN
FPGA
FPGA
D32
S4
C2
CCB&
CCB&
VME
VME
FPGA
FPGA
D32
S1
C2
C3
CONFIG.
CONFIG.
ROM
ROM
CLK4
0
DDU FPGA collects data from input and output buffers for
transmission to a single CSC DDU (aka FED)
Î Optical link is bi-directional (if needed) for asynchronous xfer
Î CSC TF can exist in a separate partition from CSC chambers if
we are allowed to send one SLINK connection to DAQ
Î
CPT Week, April 2002
9
Darin Acosta
Current DDU Prototype
J. Gilmore
CMS-EMU Meeting, Gainesville
4/5/02
4
SP2002 Interfaces
●
VME Interface – updated for chip-level broadcasts
(and crate broadcasts envisioned as well)
●
CCB Interface – using same interface as CSC Local Trigger
●
MPC Interface – updated (BC0 flag is sent through all TF
cards now: MPC->SP02->MS, To/From DT)
Î
MPC Data Validation - updated
●
DDU Interface - updated (optional bi-directional)
●
FM Interface – updated (RJ-45, 4 diff. signals)
Î
Fast Monitoring Signals – updated (pinout)
●
MS Interface – updated (BC0)
●
DT Interface – updated (Synch/Calib -> BC0 ?)
Detailed accounting of all bits and protocols for these
interfaces are specified in documents available from
http://www.phys.ufl.edu/~acosta/cms/trigger.html
CPT Week, April 2002
10
Darin Acosta
DT Interface
Data delivered from the Sector Receiver to the DT Track-Finder
@ 40 MHz using LVDS.
Signal
Bits / stub
φ
η
Quality
BXN
Clock
BC0
Total:
12
1
3
–
–
–
16
Bits / 3 stubs
(ME1: 30°)
36
3
9
2
1
1
52
Bits / 6 stubs
(ME1: 60°)
72
6
18
4
2
2
104
Description
Azimuth coordinate
DT/CSC region flag
Derived from 4 bit Quality
2 LSB of BXN
Clock for data
Bunch Crossing 0
Data delivered from the DT Track-Finder to the Sector Receiver
@ 40 MHz using LVDS.
Signal
φ
φb
Quality
Muon Flag
BXN
Clock
BC0
Total:
Bits / stub
12
5
3
1
2
1
1
25
Bits / 2 stubs
(MB1: 60°)
24
10
6
2
4
2
2
50
Description
Azimuth coordinate
φ bend angle
2nd muon of previous BX
2 LSB of BXN
Clock for data
Bunch Crossing 0
CMS Note on DT/CSC interface ~ready to be released
CPT Week, April 2002
11
Darin Acosta
SP2002 FPGA Choices
Front FPGAs (3 Muons per FPGA, 5 FPGAs total)
Interfaces require at least 365 I/Os
‡ Choice: XC2V1000-?FF896C with 432 user I/Os
Î May be socketed (BGA soldered to high-density pin array)
Î
Main FPGA
Interfaces require at least 716 I/Os
‡ Choice: XC2V4000-?FF1152C with 824 user I/Os
Î Placed on mezzanine board
Î
VME & CCB FPGA
Î
Interfaces require at least 150 I/Os
‡ Choice: XC2V250-?FG456C with 200 user I/Os
DDU FPGA
Î
Interfaces require at least 110 I/Os
‡ Choice: XC2V250-?FG256C with 172 user I/Os
Additionally, there are 51 SRAM chips
Î
Board will be dense! (Merger of 4 boards)
CPT Week, April 2002
12
Darin Acosta
CSC TF Milestones for 2002
CSC Track-Finder:
Dec. 2001: Specify backplane connections 9
9
Î Apr. 2002: Conceptual design complete
‡ Reviewed by US trigger group during 2 day workshop in
March
Î May 2002: Schematics complete
Î Sep. 2002: Layout complete
Î Nov. 2002: Finish fabrication of pre-production prototypes
‡ 2 month delay from original Sep. 2002 milestone
Î
CPT Week, April 2002
13
Darin Acosta
CSC Prototype Test Schedule
MPC logic tests:
10/1/02 – 12/31/02
Î SP logic tests:
12/1/02 – 4/30/03
Î MPC → TMB:
7/1/02 – 12/31/02
Î MPC → SP:
3/1/03 – 4/30/03
Î FAST site chain test:
5/1/03 – 6/30/03
‡ Cosmic ray test with chambers and full chain of prototypes
Î Structured beam test:
7/1/03 – 9/30/03 ?
‡ Chain test with detectors in beam line, sometime in ’03
Î DT TF → CSC TF:
7/1/03 – 9/30/03 ?
‡ Crate test sometime before or after beam test
‡ Transition boards need to be designed
Î
Note: we still have a lot of software to design and write to perform
these system tests. Must interface previous trigger test code to
FAST site test code, XDAQ, etc. Fortunately, we have several
students and postdocs identified to help in this area. But we could
use some guidance and examples on how to get started.
CPT Week, April 2002
14
Darin Acosta
CSC Production and Testing
Currently scheduled to complete CSC trigger
production (CCB, MPC, SP) by Sep. 2004
Slice test scheduled for ~Oct. 2004
Î
If production not yet complete, will use current prototypes
Installation to begin Nov. 2004
Start integration with DAQ Mar. 2005
CPT Week, April 2002
15
Darin Acosta
Special Triggers
After commissioning, we’ll still want to monitor trigger
efficiencies and alignment during data taking
Some ideas:
Prescaled low pT thresholds
‡ Useful to monitor efficiency, alignment, etc.
Î Prescaled loose 2-station triggers or even single station
triggers
‡ Useful to monitor local trigger efficiency (BTI and LCT)
since this is dependant on analog chamber performance
(i.e. can’t just run digital simulation)
Î Accelerator muon triggers
‡ CSC Track-Finder (and GMT) will have the ability to trigger
on muons traveling parallel to the beam axis during normal
running
‡ Should be useful for in-situ alignment studies of chambers
Î
CPT Week, April 2002
16
Darin Acosta
CSC Trigger Primitive
Electronics Status
• Wire boards: ALCT (Anode Local
Charged Track)
• Strip/coincidence/readout boards: TMB
(Trigger MotherBoard)
Hauser, CERN, April 2002
1
ALCT2001 Boards
Power, computer
connectors
80 MHz SCSI outputs
(to Trigger Motherboard)
Xilinx mezzanine card
Main
board
for 384ch type
Delay/ buffer ASICs,
2:1 bus multiplexors
(other side)
Input signal
connectors
Analog section:
test pulse generator,
AFEB power,
ADCs, DACs
(other side)
Hauser, CERN, April 2002
2
ALCT Functions
1.
Inputs discriminated signals from AFEB frontend boards, provides AFEB support:
•
Distributes power, shut-down, test pulse signals.
•
Sets and reads back discriminator thresholds.
•
Monitors board currents, voltages, and temperature.
2.
Delay/translator ASIC on input does time
alignment with bunch crossings.
3.
Searches for muon patterns in anode signals. If
found, sends information to trigger motherboard.
4.
Records input and output signals at 40 MHz in
case of level 1 trigger.
Hauser, CERN, April 2002
3
ALCT Status
• Board has been thoroughly debugged
• Have extensive suite of testing software and an
external test fixture.
• 3 versions:
• ALCT2001-384 is in pre-production (30 boards, 40 mezzanine
cards)
• ALCT2001-672 have 6 prototype boards being assembled
• ALCT2001-288 have 6 bare prototype boards, assembly soon.
• Now integrating with software for chamber tests at U
Florida and UCLA
Hauser, CERN, April 2002
4
TMB - Trigger MotherBoard
• TMB2001 prototypes for use at
FAST sites and for system
tests
• Produces cathode patterns
from comparator outputs
• Correlates cathode and anode
(from ALCT) patterns
• Sends chamber-level trigger
decision to MPC
• Raw hits data “spooled” to
DMB
• Interfaces to “everything”
•
•
•
•
•
•
•
•
CFEBs
DMB
CCB
ALCT
MPC
VME
RPC (later)
JTAG
Hauser, CERN, April 2002
CFEBs
ALCT
(Future: will be via transition
module)
RPC via transition module
5
TMB Hardware Status
• 17 TMB boards were produced (all FAST sites plus
development uses)
• Bad job done by assembly company:
• Connectors soldered in crooked, boards could not be inserted
• Connectors were removed and reinstalled properly by UCLA
• Misc. errors (about seven per board)
• 2 boards have been fully corrected and debugged,
including CFEB, ALCT, DMB, DDU, CCB interfaces
• 1 TMB was shipped to OSU for CFEB/DMB/DDU tests
• The other TMB is for continued firmware development
at UCLA
Hauser, CERN, April 2002
6
PHOS4 Problem
• PHOS4 delay chips are convenient, so were used in
TMB and CCB (Clock&Control Board)
• But…
• They can only be reset once. This can be a big problem.
• We have had a lot of difficulty getting them initialized properly
(months of work). Sometimes power cycling a crate is the only
solution (ugh).
• They produce an asymmetric output clock, especially if one
PHOS4 used in series with another.
• Delay = 0 setting gives about 3 ns variance between chips, not
very tight.
• PHOS4 power-on sequence work-around allows board to operate
• Wait 1 second, send a reset, wait 1 second, program via I2C bus
• (Previously, PHOS4 chips did not power up in clocking state (dead
board!), 5ns delay sometimes became 10ns etc.)
• Still have very asymmetric (58/42%) output clock
Hauser, CERN, April 2002
7
TMB-CFEB-LVDB Test
Hauser, CERN, April 2002
8
ALCT-TMB Testing
Hauser, CERN, April 2002
9
Virtex-II For Future Use in TMB
• Allows faster
performance and some
clocking and I/O
advantages
• Initial layout done
• Designed for “medium”
(896 ball) FPGAs for
TMB (or ALCT)
• XC2V1000, 1500, 2000
chips
• Ball locations
compatible with “large”
(1152 ball) FPGAs for
e.g. Sector Processor
• XC2V3000, 4000, 6000,
8000, 10000 chips
Hauser, CERN, April 2002
10
PHOS4 features
•CERN designed 4-channel delay ASIC with 1 ns
delay precision
•Radiation Hard
•Programmable (write-only) over I2C bus
Issues with PHOS4
„
„
The PHOS4 is intended for use with non-interruptable clock
source (since the chip utilized DLL circuitry)
If clock is interrupted, then the behavior is unpredictable
{
{
{
{
wrong delays
“dead-looking” channels
no direct way to identify what is wrong with the chip
Only power cycling can help
„
Lack of dedicated “reset” pin
Delay settings are not readable back via I2C bus
„
We have suggested that the chip be redesigned
„
Clock Interruptions?
„
„
„
In our opinion, the clock should never
be interrupted (except on power
cycling)
TTC tree is designed so clock won’t be
interrupted
A question to CMS/LHC is “How often
can we expect the clock to be
interrupted?”
Interruptions
„
What to do if interruptions are a
problem?
{
PHOS4 can be simply removed from the
CCB boards (all 4 socketed) and clocks
wired directly to LVDS transmitters
„
„
„
no individual slot-to-slot and module-tomodule clock adjustments
no extra effort to program I2C
same PCB layout, minimal on-board
changes
Another Question
„
„
All clock lines in the custom peripheral
backplane are point-to-point LVDS of the
same length.
Do we really need to adjust the 40Mhz
clock from slot to slot and from
module to module?
Another Alternative
„
If fine clock adjustments on the
backplane are still needed
{
PHOS4 can be replaced with another device,
for example, 3D7408 proposed by OSU
„
„
„
„
1-channel commercial CMOS programmable delay
chip
This device has 45/55 output duty cycle instead of
50/50 at 40Mhz (tested at Rice). Is it acceptable for
DMB/TMB/ALCT?
layout changes on a CCB board required
Radiation tolerance?
TTC Issues
Trouble Reported
„
„
„
At the last CMS week (and
continuing) reports from many
groups that they can not drive optical
links with a TTC derived clock
(Sorry, but nobody [but us] is
documenting what they are doing, so
I can’t point you to anything written
on the topic)
Our report is at
http://bonner-ntserver.rice.edu/cms/jitter.pdf
The Issue
„
„
„
„
TTC group has specified that they
will deliver clock good to 50ps rms.
Groups using CERN GOL (not us)
claim TTCrx can’t drive their links
Reports of jitter measurements
~500ps
At last CMS week microelectronics
group agreed to improve TTCrx
z
( time scale?)
Our Measurement
Mezzanine Card
ECP680-1102-630C
TTCrx ASIC operating voltage
+5.0V
+3.3V
+5.0V
Clock40Des1 jitter, ps (no BC, no L1A)
153
170
330
Clock40Des1 jitter, ps (BC commands + L1A)
183
215
360
New TTCrx
Old TTCrx
ECP 6801102-610B
Our Test
B
I
T
3
PC
T
T
C
V
I
C
C
B
T
T
C
V
X
•
•• ••
• •
ERROR
• TTCrx
•
O
P
T
O
100 m
40 Mhz
VME 9U
•
•
O
P
T
O
Clock
multiplier
•
•
1m
COPPER CABLE
OPTICAL CABLE
VME 6U
100 m
• OLD AND NEW TTCrx BOARDS WERE TESTED WITH
40.00 Mhz CLOCK SOURCE FROM TTCvx MODULE
• 40.00 Mhz CLOCK WAS MULTIPLIED BY 2 BY AV9170 CHIP
• NO ERRORS OBSERVED IN PRBS TEST FROM ONE OPTOBOARD
TO ANOTHER AT 80.00 Mhz (BER < 10-13 c-1)
Our Result
„
„
„
„
„
Clock jitter is lower for the newest TTCrx
ASIC (Version 3.1, 12/01)
Jitter increases if broadcast commands
and L1A are transmitted from the TTC
system
Jitter distribution for ASIC Ver.3.1 looks
“normal” (unlike previous version)
Clock jitter is lower if the new ASIC is
powered from +5V
Jitter introduced by any of two TTCrx
ASICs and other components in the clock
distribution circuitry at our testing setup is
tolerable for TLK2501 transceivers
operating at 80.00 Mhz using prototype
MPC-SR link