SLHC Design Considerations for the
CSC Track-Finder
&
An Asynchronous Trigger Proposal
Darin Acosta and Alex Madorsky
University of Florida
Outline

CSC Muon Trigger





Brief review of the CSC Track-Finder for CMS
CSC BX assignment issues
CSC occupancy estimates
Some “obvious” changes for SLHC operation
Interesting R&D Directions


Xilinx Rocket IO “X” technology
Proposal to go asynchronous at Level-1
 After BX assignment, of course
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
2
CSC Muon Trigger Scheme
EMU
Trigger
On-Chamber
Trigger Primitives
Trigger
Motherboard
(UCLA)
Strip FE cards
Muon
Port
Card
(Rice)
LCT
3-D Track-Finding
and Measurement
Sector Receiver/ Processor
(U. Florida)
OPTICAL
FE
SR/SP
MPC
SP
LCT
3 / port card
TMB
FE
2 / chamber
Wire LCT card
Wire FE cards
RIM
RPC Interface
Module
Combination of all
3 Muon Systems
13 Feb 2004
SLHC Workshop, Madison WI
3 / sector
In
counting
house
RPC
4
CSC Muon Sorter
(Rice)
DT
4
4
Global  Trigger
Global L1
4
Darin Acosta, University of Florida
3
CSC Track-Finder Crate
Muon Sorter
SR SR SR SR SR SR
/ / / / / /
SP SP SP SP SP SP
MS
CCB
Clock & Control
Board
SBS 620 Controller
Single crate solution, 2nd generation prototypes under test
SR SR SR SR SR SR
/ / / / / /
SP SP SP SP SP SP
Sector Processor
From MPC
(chamber 4)
From MPC
(chamber 3)
From MPC
(chamber 2)
From MPC
(chamber 1B)
From MPC
(chamber 1A)
To DAQ
180  1.6 Gbit/s optical links:
Data clocked in parallel at 80 MHz in 2 frames (effective 40 MHz)
Custom 6U GTLP backplane for interconnections (mostly 80 MHz)
Rear transition cards with 40 MHz LVDS SCSI cables to/from DT
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
4
SP2002 Main Board (SR Logic)
Phi Global
LUT
PLL
patch
Eta Global
LUT
Phi Local
LUT
TLK2501
Transceiver
To/from
custom
GTLP
backplane
Front
FPGA
Optical
Transceivers
SR Logic
• 15 x 1.6
Gbit/s Links
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
5
SP Trigger Logic
SP2002 mezzanine card



Xilinx Virtex-2 XC2V4000
~800 user I/O
Same mezzanine card is
used for Muon Sorter
Track-Finding logic
operates at 40 MHz


13 Feb 2004
SLHC Workshop, Madison WI
Frequency of track stub data
from optical links
Easily upgradeable path
Darin Acosta, University of Florida
6
Track-Finding Latency
11  25 ns, or 275 ns
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
7
CSC BX Assignment

CSC Time resolution



60 ns maximum drift-time per plane, 6 planes per chamber
 5 ns chamber resolution (ALCT takes time from 2nd hit)
Not likely to change for SLHC…
Centered peak:

For time distribution centered in BX interval:
LHC
0.6%
98.8%
BX1
BX2

13 Feb 2004
SLHC
0.6%
BX3
10%
80%
BX1
BX2
10%
BX3
Probability to get LCT in correct BX
 LHC:
98.8%
 SLHC:
80%
(will clearly need to consider multi-BX)
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
8
From Andrey: TB’99 Data
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
9
Other CSC Time Scenarios for SLHC

Bifurcated peak:

0.6%
49.4% 49.4% 0.6%
BX1
BX2

Offset peak by 3 ns:

3%
70%
BX1
BX2
13 Feb 2004
Fit 98.8% of LCT’s in 2 BX
window, but ambiguity on which
BX muon belongs
27%
BX3
SLHC Workshop, Madison WI


Puts 97% of LCT’s in 2 BX
window
70% in central BX
27% or 3% in following BX
(choice depends on BX
algorithm for Track-Finder)
Darin Acosta, University of Florida
10
Track-Finder BX Assignment


Current CSC Track-Finder takes earliest arriving LCT as
definition of track BX over a multi-BX window
LHC:


SLHC:



2-station tracks:
 Centered peak: 98.8% correct BX assignment
2-station tracks:
 Offset peak:
87%
 Centered peak: 80%
3-station tracks:
Offset peak gives
 Offset peak:
89%
best performance
 Centered peak: 73%
 2 BX window (25 ns) for these efficiencies
88% BX i.d. efficiency with offset peak and taking
earliest arriving LCT, two or more stations
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
11
Alternative Track-Finder BX Assignment


Take ALCT approach, and consider second arriving
segment to define track BX
SLHC:

2-station tracks:
 Offset peak:
87% correct BX assignment
 Centered peak: 80%
Same as before

3-station tracks:
 Offset peak:
78% (82%)
 Centered peak: 90% (95%)
Offset peak gives worse
performance for 3 stations
2BX (3BX) windows

Can improve BX i.d. efficiency to 95% with centered
peak, taking second LCT, requiring 3 or more stations
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
12
Andrey’s Conclusion:
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
13
Improved CSC Performance?

It’s worth considering what can be done to slightly
improve CSC timing (change gas, increase high-voltage)


Suppose 5 ns resolution  4 ns
6%
88%
BX1
BX2
6%
1%
BX1
BX3
BX i.d. from first hit:


2-station tracks:
 Offset peak:
 Centered peak:
3-station tracks:
 Offset peak:
 Centered peak:
13 Feb 2004
BX i.d. from second hit:

93%
88%

96%
83%
SLHC Workshop, Madison WI
77% 22%
BX2
BX3
2-station tracks:
 Offset peak:
 Centered peak:
3-station tracks:
 Offset peak:
 Centered peak:
61%
88%
87% (87%)
96% (98%)
Darin Acosta, University of Florida
14
CSC Occupancy (LHC)

Start from CMSIM trigger study



R.Cousins, J.Mumford, and V.Valuev: CMS Note 2002/007
Dedicated ORCA trigger simulation, albeit with LCT logic that
does not exactly match final production hardware & firmware
Correlated LCT Occupancy (ALCT+CLCT match)



Entire CSC system: 0.05 / pp collision
or 0.9 LCT’s per BX @ L = 1034
MPC occupancies @ L = 1034
 ME1:
0.025 / BX *rescaled to 30° subsectors
 ME24: 0.008 / BX
 Recall that MPC can accept up to 3 LCT’s / BX
Adding neutrons leads to 30% increase in ME1ME3,
3X higher in ME4
 Ignore neutrons for now. Correlated LCT rate from neutrons
probably doesn’t scale linearly with luminosity since it is
composed mostly of random hits. Hard to make projections.
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
15
Projected CSC Occupancy (SLHC)

SLHC


Assume L = 1035, BX interval decreases to 12.5 ns, chambers
stay the same
Correlated LCT Occupancy


Entire CSC system: 0.9 * 10 / 2 = 4.5 LCT’s / BX @ 80 MHz
MPC occupancies @ L = 1035, assuming 80 MHz operation
 ME1:
0.125 / BX (ME24 is 3X smaller)


(spoils di- measurement in 1 MPC)
MPC occupancies @ L = 1035, assuming 40 MHz operation
 ME1:
0.25 / 25 ns
(ME24 is 3X smaller)


P (2) = 0.7%
P (2) = 2.6%
(spoils di- measurement in 1 MPC)
Occupancies are not huge, but we neglected neutrons,
and LCT ghost probability might be higher.
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
16
Projected Occupancy in CSC Track-Finder

For optimum efficiency and decreased sensitivity to
exact CSC timing, we plan to use a 2 BX (50 ns) window
at LHC to accept LCT’s from the MPC’s




SR occupancies for SLHC @ L = 1035, 4 BX window




e.g. See A.Drozdetski’s testbeam analysis talk from Oct.’03
EMU meeting
LCT efficiency goes from 98% to 99.5% with 2 BX window,
Track-Finding efficiency goes with square or cube of this
Earliest arriving LCT defines BX (but this may not be best choice…)
ME1:
0.5 / 50 ns (every other trigger BX!)
ME24 is 3X smaller
BX i.d. study suggests only 23 BX will be required…
Need to perform detailed rate studies to see if we pick
up fake tracks that trigger
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
17
Inclusion of Muon Data with Tracker


It would be extremely desirable to include tracker data
at Level-1
How it is planned to be used at LHC for HLT:



Attach tracker hits to improve PT assignment precision from
15% standalone muon measurement to 1.5% with the tracker
 Will improve sign determination as well and offers vertex
constraints
Find pixel tracks within cone around muon track and compute
sum PT as an isolation criterion
 Less sensitive to pile-up than calorimetric information if
primary vertex of hard-scattering can be determined
(~100 vertices total at SLHC!)
To do this requires  information on the muons finer
than the currently reported 0.052.5°

No problem, since both are already available at 0.0125 and 0.015°
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
18
Muon Rate at L = 1034
From DAQ TDR
Note limited
rejection power
(slope) without
tracker information
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
19
Preliminary Conclusions on CSC for SLHC

Will probably want to upgrade front-end trigger boards
and optical links to send LCT data @ 80 MHz



BX identification is about 80% correct at LCT level
BX identification is about 90% correct at Track-Finder
using current algorithm


2 BX acceptance window, 2 or more stations, offset timing peak
Can be increased to 95% correct BX i.d. at Track-Finder



Simulated muon occupancy may be low enough to avoid this,
but with strong caveats. Real data may be worse.
3 BX acceptance window, 3 or more stations (need ME4!)
Requires new logic in Track-Finder to take second LCT time
Trivial to add more finer eta and phi information to
reported muon candidate for use by a muon-tracker
“match box”
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
20
More Generic R&D
Xilinx RocketIO X
Maximum speed: 10.3125 Gbit/sec
 Latency is not yet published
 “Will be better than RocketIO” according to tech support
 Minimal latency calculation (in clock cycles) based on Rocket IO
documents:

13 Feb 2004
Input register
2.5
TX CRC (bypass)
1
8B/10B encoder
1
TX FIFO
4.5
TX SERDES
1.5
RX SERDES
1.5
Realingment
3.5
8B/10B decoder
1
RX FIFO (reduced latency)
2
Output register
2.5
Total
21
SLHC Workshop, Madison WI
A lot, but…
Darin Acosta, University of Florida
22
RocketIO X


Clock cycle at 10 Gbit serial speed: ~250 MHz
Total latency: ~84 ns (should be better for RocketIO X)
Thus, probably an interesting avenue to explore to bring
huge data volumes into an FPGA

13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
23
An Asynchronous Level-1 Trigger?

The high-speed data links that will be available for use
at SLHC (10100 Gbit/s), and the challenges of timing in
a fully synchronous system and distributing a jitter-free
clock at the sub-ps level, started us thinking about
going asynchronous after the front-end BX assignment



We DO still need a clock synchronous with the machine
frequency distributed to the FE boards for BX assignment
But unlike early LHC electronics, there will now be several
orders of magnitude separating the machine frequency from
the data communication frequency
We KNOW this has to work because HLT is already
asynchronous

Moreover, we know that CSC trigger data collected
synchronously at the CSC Track-Finder exactly matches the
DAQ data collected asynchronously
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
24
Advantages of an Asynchronous Design
1. De-couple clocking requirements of high-speed data
links from synchronous BX assignment
 Data instead is sent with a BX label when it is available,
and trigger logic assembles event for processing
 Can use on-board xtal oscillators for serdes reference clock,
rather than multiplying an 80 MHz clock to multi-GHz
2. Maximize the utility of the available bandwidth
 A synchronous system must keep a low occupancy on the data
links, otherwise Poisson fluctuations in one BX will overflow
the link  wasted bandwidth by sending lots of 0’s
 Some trigger subsystems recognized this and already serialize
data over multiple BX
 RPC, DT
 A step toward an asynchronous design already!
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
25
Advantages II
3. Respond more robustly to bursts
 Could allow more segments/clusters in an unusually busy
event by allowing transmission time to vary
(Imagine a black-hole event…)
4. Detector technology may not keep pace with shorter
SLHC BX
 e.g. The CSC FE may not have the ability to accurately
determine the 12.5 ns bunch. Track-Finder might need a 50 ns
(4 BX) window to trigger efficiently.
 DT system may be in even worse shape with BX assignment
 Less compelling why data must be sent synchronously when
it is naturally distributed over several BX
 Might it be possible to not have a machine BX at all, but one
long train? (BX i.d. then becomes a time-stamp) Big question
will be detector occupancies…
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
26
Advantages III
5. Decouples clock frequency of algorithm from BX
frequency
 A shorter BX does not mean faster logic. Logic works at
transistor switching speed. Too short of a clock means
wasted overhead to allow signals to settle before latch
 Might allow continued use of legacy 40 MHz boards
6. Might allow incorporation of DSP’s and CPU’s into
trigger architecture
 Is a high-performance DSP useful?
 What about embedded PowerPC chip in FPGA?
 If not for triggering, very useful for slow control, DAQ,…
 Perhaps track-finding, jet-clustering, b-tagging could benefit
 Might think of this as merging traditional L2 into L1, rather
than L2 into L3 as CMS now does. Given that we want tracker
at L1, maybe this is a way we need to go.
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
27
Constraints

Proper BX assignment is still required at front-end



Level-1 decision must be reached by a maximum latency or sooner



Must have a time-out mechanism on data transmission and algorithms
Still must keep latency short!
Event building circuitry needed in FPGA’s


Data is stamped with 12.5 ns bunch crossing time
Distribution of synchronous 80 MHz clock much less challenging (it’s
done already) than one used to drive data links
 TTC system may be overly complicated for what we need
Should be small
Escape clause:

It’s always possible to wrap up the asynchronous logic into a
synchronous “black box” (re-align data at trigger output)
 We considered this as an option for the MPCSP optical link
transmission in the current system, for example, to solve clock
jitter issues
 A matter of determining how small or large an asynchronous block
can be built
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
28
Advanced Detector Research Proposal


Submitted a proposal to the
DOE Advanced Detector
Research program for FY04 to
prototype such an
asynchronous system
Based on existing CSC
detector, with upgrade to
ALCT logic and redesigned
Processor board
13 Feb 2004
SLHC Workshop, Madison WI
Darin Acosta, University of Florida
29