ATLAS
Thoughts on Data Transmission
US-ATLAS Upgrade R&D Meeting
UCSC
10-Nov-2005
A. A. Grillo
SCIPP – UCSC
10-Nov-2005
1
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Old Guidelines for the Present SCT DAT
ATLAS
• Avoid shared parallel write busses where one failed IC tends to take
out the whole bus, i.e. use serial data paths.
• Provide redundant data transmission paths onto and off of the
detector.
• Employ optical transmission for all high speed communication
between the detector unit and the off detector electronics.
• Saves some services volume (fibre vs. wire).
• Minimizes interference between different detector units making
scale up based upon test results of a small subset more viable.
• Provide a mechanism to bypass a dead chip.
• All electrical communication between ICs or between the detector
units must be via low amplitude fully differential signals.
• Minimizes electrical noise.
10-Nov-2005
2
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Present SCT Data Transmission System
ATLAS
Apologies for the extremely old picture.
VCSELs not LEDs
Ignore patch panel and fibre lengths
Two data links,
1 clk, 1 com
per module
data 1
SCT
data 2
LDC
LED
hybrid
clock
n
com m and
DORIC3
PIN's
PATCH
PANAL
LED
O
PIN
LED
radiation hard
fibre (8m )
non-radiation hard
fibre (80m )
SCT
hybrid
n+1
10-Nov-2005
3
Redundan
t
clk &
comThoughts on Data Transmission
R
A.A. Grillo
SCIPP-UCSC
D
Interconnects on Each Module
ATLAS
One Detector Module with Twelve ICs
serial
22 serial
outputs
outputs
“N”
normal
“B” bypass
10-Nov-2005
4
Clk & Com not shown but
bussed in parallel to all ICs.
Likewise for redundant Clk & Com.
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Transmission Data Rates & Multiplexing
ATLAS
The natural clock frequency of the readout ICs is the beam crossing
frequency or perhaps x2 the beam crossing frequency.
• Now at BC Frequency = 40 MHz
• Could become 67 MHz, 80 MHz or even 100 MHz
Optical transmission now typically operates at GHz rates
• If such rates are attainable with rad-hard components, fibre
count will be reduced.
Operating the optical transmission at GHz rates implies multiplexing
• Difficult to operate electrical transmission at those rates
Multiplex several serial chains with several ICs in each chain (figure).
The number of chains and the number of ICs in each chain will depend
upon the details of the detector unit (e.g. stave).
• Higher rates for each chain will add flexibility for layout options.
• Each chain may require phase adjustment.
10-Nov-2005
5
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Possible Multiplexed Architecture
ATLAS
10-Nov-2005
6
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
The Present Bypassing Scheme
ATLAS
The present bypass scheme has worked well but some difficulties:
• The layout of the hybrid circuit has added complexity with the
addition of four extra traces for each IC. The differences in
capacitive load of the normal and bypass traces required slightly
different designs for the two drivers/receivers in order to keep
power to a minimum.
• The connections to bypass an IC at the edge of the hybrid
required very long traces. These long traces required a much
more powerful differential driver (i.e. more power consumed) and
these signals became the function limiting the operational speed
of the IC.
• Some of the redundant paths so complicated the topology of the
hybrid layout that one was not implemented precluding the
bypassing of one IC.
10-Nov-2005
7
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
A Simpler Bypassing Scheme
ATLAS
Define two directions for the data flow (e.g. left and right)
If an IC fails, the data flow direction of ICs are arranged such that all to
the left of the dead IC send data to the left and all to the right of the
dead IC send data to the right (figure).
Both left and right drivers drive only short path to adjacent IC
Add third driver (powered only when needed) to drive long line to
multiplexer or displaced section of chain. (Equivalent of master driver
on present IC.)
Number of drivers and hybrid traces are not reduced but trace routing
is simpler and drivers are more efficiently matched to loads.
10-Nov-2005
8
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Bypassing a Dead IC
ATLAS
10-Nov-2005
9
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Configuration Read Back
ATLAS
The present SCT readout IC, the ABCD, does not allow for read-back of
its configuration registers.
• Tests for single event upset in a particle beam indicate that we
should not have a problem if we periodically reload the registers.
• The higher fluences of the upgraded ATLAS and the smaller
geometries of the IC structures could create a worse problem.
• Cannot tell in situ what the real error rate is; the re-load rate
must be set to the worst case estimate.
Upgrade IC should provide for read back of configuration registers.
• To minimize fibres, must use same output as for data.
• Protocol developed for the SCT readout provided for such
configuration read back but not implemented because of space
required for extra circuitry. Need to re-examine the protocol to
see if it can be improved, then implement in the new IC design.
10-Nov-2005
10
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Command Protocol
ATLAS
Present command protocol developed to allow triggers, short commands
and long configuration commands using the same input stream.
A primary 3-bit identifying sequence was chosen in order to allow
consecutive triggers to appear with only a gap of two beam crossing
periods. (An ATLAS trigger requirement)
Later relaxed to a longer gap. The Pixel IC made use of a longer bit
stream making it more immune to bit flips in the command sequence.
Keep longer pattern as Pixels (if still allowed by trigger) or increase data
rate to allow longer sequence as long as SEU rate acceptable.
• If beam crossing rate increases, the command data rate must
increase proportionally.
10-Nov-2005
11
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Encoding of Clock and Command Signals
ATLAS
Clock and Commands carried on one fibre with biphase-mark encoding
• Signal decoded on module into separate LVDS signals
• Decoding could be in same IC as multiplexer
• Would allow path for configuring multiplexer
Possible to multiplex more than one command stream on the same fibre.
• Reduce time for frequent re-configuration (if necessary)
• But … higher data rates will aggravate SEU problems
Need separate electrical drivers for each chain of ICs
• Isolate failures
• Possible need for phase adjustments for each chain
Could be advantage to maintain encoding in electrical chains and
decode at each IC (figure).
• Reduce number of drivers and dual traces
10-Nov-2005
12
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Keeping Clk/Com Encoded in E-Streams
ATLAS
10-Nov-2005
13
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC
Redundant Clock & Command Signals
ATLAS
Important to maintain redundant clock and command signals.
Present scheme of sharing clk/com fibre of adjacent module as
redundant stream creates some problems.
• Electrically couples adjacent modules destroying isolation
• Electrical connection between modules is mechanically awkward.
Preferable to have two clk/com fibres per detector unit to maintain
electrical isolation.
10-Nov-2005
14
Thoughts on Data Transmission
A.A. Grillo
SCIPP-UCSC