Atlas SCT ROD FDR
Implementation Model
Atlas Wisconsin Group
Khang Dao, Damon Fasching, Douglas Ferguson, Owen Hayes,
Richard Jared, John Joseph, Krista Marks, Mark Nagel
Sriram Sivasubramaniyan (Oklahoma), Alden Stradling and
Lukas Tomasek (Institute of Physics AV CR, Prague)
August 20, 2002
ROD Overview Notes
• FPGA real time data path with DSPs for back-end processing and operation
control.
– Different firmware and part loaded for SCT or Pixel RODs
• 9U VME card, custom back plane.
• Back-of-Crate Card for optical interface to modules.
• Clock and Trigger.
– Clock and trigger signals are available on the back plane from the Trigger Interface
Module in the crate.
•
•
•
•
•
•
•
•
Supports 100 kHz trigger rate without an upgrade.
SCT: 96 input links at 40 MHz
Diagnostic FIFO Memories on board for testing and data monitoring
Register based interface with DSP list processor.
Fiber Optic S-Link to Readout Buffers.
12 FPGAs, 4.7 Million gates of programmable logic.
5 DSPs, One for control and 4 for calculations.
Cypress Semiconductor VME Slave Interface chipset.
– Block Transfers - 2.2 Mbytes/second Slave DSP read access.
– A32/D32 data transfers
– Simple interface and supports standard VME transactions w/ interrupts.
2
Serial
Output
Link
48
8/20/2002
ROD Controller:
Operation,
Command, Trigger
Slink on BOC
S-link Clk
S-link Test
DMA
Cont.
DSP Event
Trapping,
Histograms
(Four DSP Chips)
Host Port
Interface
Program
Reset
Manager
S-link WR
Data Bus
32
Cont / Addr
ROD Busy
Trig Data (TTC Bus)
VME Bus
BOC
Setup
Bus
R/W
Bus
...
S-link Data
VME Slave
Interface
Timing Interface
Module
R/W
Bus
Cnfg, Reset
R/W Bus
Control & Status for Block
Xfer/ DMA / S-Link
DMA 4
R/W
Bus
Router Halt Output
Data Valid
S-link Data 43
Xon/Xoff
Router
FPGA
S-Link
& DSP
DMA
DMA 1
Input
Memory
& XCVR
Control
96
Cnfg, Reset
Serial
Input
Link
Data
RCVR
Dec. Trig Count
Control, Status, Mask
Busy
96
Formatter
and EFB
FPGA
Token and Dynamic Mask
Event Data/Tirg Type
Back of Crate Card / Front End Electronics
ROD Implementation Model: FPGA Front End / DSP Back End
3
Data Receiver Implementation
48
Data
Input
4k words
FIFO Out
Register
FIFO
FIFO In
Register
Link Input
from BOC
Back plane Connector
48
ADV OE RT
48
Link or
Diagnostic
Data to
Formatter
48
Data
FIFO Data
Register
4k words
FIFO Data
Register
Input
FIFO
ADV OE
Input
Memory
& XCVR
Control
8/20/2002
Control
FIFO
Operation
Advance FIFO and latch new input
Enable FIFO output data
Retransmit FIFO
Read or write data
Select 16 or 32 bit part of word
R/W
Bus Data
48 bits are read as
32 bit word bits 0:31
16 bit word bits 32:47
4
Data Receiver Notes
• Input Diagnostic Memories.
– Play or record data to perform diagnostics and tests.
– Start of record operation is triggered by the ROD Controller.
– Trapping (record) of Link Data can be synchronized to triggers or configuration
commands.
– The controller can retransmit the list in a loop. The retransmit can be issued using the
serial inputs (SP0 or SP1) from the DSP, or by a preset counter used in one of the Test
Bench modes. Trigger information (from the ROD TIM FIFO) can also be
retransmitted so that the ROD can have trigger rates up to 100kHz.
– The SCT ROD implementation of the Input Diagnostic Memories is two groups of 4k
x 48 bit FIFOs. The Pixel ROD will use 64k deep FIFOs.
• Drivers and Registers
– The input drivers allow the input link data to be routed to the Formatters and FIFOS
simultaneously. The FIFO can not be read by the DSP while the Input Links are
connected to the Formatters.
– The Register ICs are used to latch the data from the ROD bus or the FIFO. The
FIFOs receive data from the ROD Bus from the DSP through the Controller. The
ROD Bus is not setup to handle a burst from a FIFO.
8/20/2002
5
FPGA Formatter and EFB
Token30
Input 12
Links
Input 12
Links
Input 12
Links
Input 96
Links
Input 12
Links
Serial to Parallel Conversion
Flag Header and Trailer Errors
Data Modes:
Raw, Condensed, Expanded
Header Trailer Limit
Busy Limit
Derandomizing FIFO
4k Deep Debug Memories
Token01
Formatter 1
Input 12
Links
Input 12
Links
Input 12
Links
8/20/2002
Gatherer Halt Output
Data
32
Halt
Output
Almost Full
Dual EFB
Design
Halt
Output
Formatter 4
Token45
Data
32
Formatter 5
Data Valid
46
Control
Data
Valid
Token74
Event
Formatting
Error
Counters
16Kx46
FIFO
46
46
Control
Almost Full
32
16Kx46
FIFO
EV Fragment
Header/Trailer
Cont header,trailer or data
Format #
Formatter 6
Formatter 7
Busy
Event ID
Checking,
Format #
Formatter 3
Mode
Bits
Event Fragment Builder FPGA
Data
Valid
Formatter 2
TokenCmd
Input
Links 12
Debug
FIFO
4k x 39B
Dec.
Trig
Count
R/W
Bus
To/From Controller FPGA
Debug
FIFO
4k x 39B
Event ID
Data /
Trig Type
Dynamic
Mask
Control,
Status,
Mask
R/W
Bus
To/From Controller FPGA
*Proposed for Pixel ROD: Formatters 0,1,4,5 only
6
To/From Router FPGA
TokenCmd
Formatter 0
Formatter 0: Top Level
MasterNSlave
1b
L1TriggerPulse
1b
TOKENin
1b
from previous chip
TOKENout
to next chip
Link Formatter FormatterWritesTrailer[11..0]
And
Link Output
FIFOs
1b
Link[11..0]DATA 12b
ModeBitsIN[11..0]
12b
Configuration(11:0)
12b
Formatter Writes Trailer
Encoder:
StoreTrailerHit
1ea
2 state
state machine
FormatterWritesTrailer
D
FF
ShowTrailerFLAGS
OverflowError[11..0]
12b
TimeOutError[11..0]
CondensedMode[11..0]
DataValid[11..0]
12b
12b
12b
Link[11..0]hasTOKEN
12b
HeaderTrailerLimit[11..0]
12b
HeaderTrailerLimit[11..0] 12b
HeaderTrailerLimit
RODBusyLimit[11..0]
12b
RODBusyLimit[11..0]
RODBusyLimit
ClearCurrentFlag
12b
1b
RodController
latch Trailers and store new internal state
on ShowTrailerFLAGS
12b
FIFO Word Counts(12 x 8b)
HeaderTrailerLimit
RodBusyLimit
TokenToReadOutTimeout
DataOverflowLimit
FormatterBUS
26b
from RodContoller
32b
FifoDATA
32b
HOLDoutput
1b
HOLDoutput
1b
ModeFifoEmpty
1b
ModeFifoEmpty[11..0] 12b
FormatterBUSStrobe 1b
RegTransferACK 1b
FifoDATA
Operations
Controller:
Register Block
Status (12 x 16b)
Configuration (12 x 4b)
HeaderTrailerLimit
RodBusyLimit
TokenToReadOutTimeout
DataOverflowLimit
8b
8b
7b
8b
OverflowError[11..0]
12b
OverflowError
1b
TimeOutError[11..0]
12b
TimeOutError
1b
CondensedMode[11..0]
12b
CondensedMode 1b
DataValid[11..0]
12b
DataValid
Link[11..0]hasTOKEN 12b
8/20/2002
Page 7
Link Number:
12:4 Encoder
1b
Link# hasTOKEN
ThisChipHasToken
EFB
Formatter 0: 12 Links Shown
Token from Controller
Link 1
12 ea.
FIFOs
and
Control
...
Serial to Parallel
Conversion and
correction of 1
bit errors in header
and trailer
Record error in
data words
Condensed Mode
and Raw Mode
Selects
Static Link
Mask 1 bit
per link
8/20/2002
7b
Dynamic
Mode Bits
32 b
Cont
Trailer & Word in
R/W
Bus
Valid
Trailer
Busy
Token to Next Link
Readout controller provides:
Readout of data on token
Link time out error detection,
header generation
Data over flow detection
Header trailer limit
Busy function and limit
Dynamic mode bits
Value Meaning
00
Normal
01
Skip Readout
10
Mask
11
Dump 1st read 2nd
Trailers to Controller
FIFO 256 W
by 32 bits
Timeout
data_to_Large
R/W
Bus
Readout
Control
Token from Previous Link
Config, Raw ABCD format per link bases
Condensed 2 hits per 16 bits & not Config, Raw
Not condensed and not Config, Raw
Expanded 32 bits for 1 or 2 hits
Edge Mode
Link on/off
39 b
FIFO 256 W
by 32 bits
Cont
Trailer & Word in
...
Link 11
32 b
12 ea.
converters
Link 10
EFB Halt Output
32 b
7b
Readout
Control
39 b
Valid
Trailer
Busy
Data to EFB
Data Valid to EFB
Mux
Link 0
Serial to Parallel
Conversion and
correction of 1
bit errors in header
and trailer
Record error in
data words
Trailers to
Controller
Decrement
Trigger
Counters
Busy to
Controller
2b
Header
Trailer
Limit to
Controller
2b
Fatal
Error to
Controller
Token to Next Formatter
2 bits per link
8
Formatter
..
.
..
.
Formatter FPGAs have separate versions of logic for SCT and Pixel Modules.
Link output FIFOs are implemented with FPGA internal memory. The XCV400E has 40
4Kbit RAM Blocks that can be implemented as 256x16 Dual Port RAM. The SCT
Formatter must accept 12 input links, and each Link FIFO must be 32 bits wide. The
maximum size of the Link FIFO with 12 links of input data is 256 words deep.
The next version of the Formatter will provide dynamic memory allocation to provide a
512 words deep FIFO if one link from a module is masked off. This modification will
combine 256x32 FIFOs on adjacent links in the case that the Static Mask Bit is set.
Formatter can keep up with any input data rate (within memory availability).
Only headers and trailers shall be written to the Link FIFO when the occupancy reaches
almost full (Header/Trailer Limit). The Header Trailer Limit value is set by writing to
the HT Limit Register in each Formatter. The value a global setting for each group of 12
links.
Individual links can be put in raw, expanded or condensed hit data mode (SCT) by
writing the appropriate value to the Formatter Configuration Register.
8/20/2002
9
.
.
Formatter
Sorts data by sequentially transferring one event of data from each link to the Event
Fragment Builder when the Controller issues a FE Command Pulse (Token
Command).
When FIFO Dual Port Memory is almost full:
– The Decoder forces the links that are almost full (Header/Trailer Limit)into write
header/trailer only mode to reduce bandwidth.
– If Memory continues to fill, and the Link FIFO reaches it set value of ROD Busy
Limit, full flag, the ROD Busy signal is issued to to stop triggers. At this point, an
error flag is issued to interrupt the ROD controller - fatal error condition(ROD
busy)
This should not be possible unless flow control has been asserted by the SLink for an extended period of time, Header/Trailer Limit is set to the same
value as ROD Busy Limit, or the front end electronics have malfunctioned.
Support flow control - EFB can assert pause signal. The Link FIFO will continue to
fill and can become full if back pressure is asserted for an extended period of time. In
this condition, the ROD will lose any event data that is on the FE links, but will retain
all events that are in the Link FIFOs.
.
.
8/20/2002
10
.
.
.
.
Formatter
If the token has been sent to a specific link, and there is no data in the FIFO, the link will
timeout after an internal counter reaches the value set in the Token to Readout Timeout
Register. In this case the Link Formatter will output a header and a trailer with the timeout
error bit flagged, and pass the token to the next link.
Events that exceed the length limit defined for a specific subset of links will have data
deleted and a header trailer generated with an error flag.
Individual links can be masked off by writing a 1 to the Static Mask bit in the Formatter
Configuration Register or dynamically using the Mode Bits. This allows the user to mask
off known malfunctioning links from a list or table or on the fly.
Correct for loss of synchronization (module data is ahead or behind other modules) in a
module (not required but is in VHDL). The dynamic mode bits correct for a link by
advancing or withholding the link data with respect to the other links. The values are shown
below:
Dynamic mode bits:
Value Meaning
00
Normal Readout Mode
01
Skip Readout of FIFO for this Trigger
10
Mask Link for this Trigger (full contents of the FIFO will be flushed)
11
Dump current event, read second for this trigger
The L1 ID number is corrected in the EFB. BCID numbers are ignored.
8/20/2002
11
Formatter
.
Serial-to-Parallel conversion with error checking.
Errors Detected: Header 1 bit error
Trailer 1 bit error
Condensed mode data error detection
Link time out error in readout
Header trailer limit
Data too large limit
Header Errors Accepted:
111010
111010
1111010
011010
101010
110010
111110
111000
111011
1110100
Head Det. Head Error
Yes
No
Y
Yes
Y
y
Y
y
Y
y
Y
y
Y
y
Y
y
Y
y
Trailer Errors : Trailer
1000000000000000
A changed bit in any position is accepted
but trailer error is set.
If in state “new_cluster” and trailer_det=1
set trailer error. Example below:
<11101><0><nnn><bbbb,bbbb><1
>
<01><aaaa><ccc,cccc><1><ddd>
chip
channel
hit
^
Trailer Detect error
8/20/2002
Synchronization error detection:
If one of the field separator bits is incorrect. The decoding is
confused. The link will be changed to raw data till a trailer is
detected.
Condensed mode error detection:
The ROD variable “edge_mode” is used to determine if the
input hit data is consistent with the state of this variable. If not
a condensed mode error bit is set.
Link time out error detection:
If a link has not received data in the time set in the time
timeout register, a header and trailer will be written with error
code.
Header trailer limit error detection:
If the Formatter header trailer limit is passed. The formatter
will not place new data in the FIFO. When a trailer is detected
the trailer is written with the header trailer limit error code.
Data too large error detection:
This is a historic state that is not longer needed. If the event is
to large (based on a register) a status bit is set.
12
.
Formatter Data Format
Formatter
SCT all data is in 16 bit packets packed in 32 bit words
bit 31:0 Data
bit 35;32 Link Channel address (for all 32 bit words)
bit 36 Time Out Error (for all 32 bit words)
bit 37 Condensed mode bit (for all 32 bit words)
Formatter Output (bits 31:0 bits 37:32 for all words)
Name
Bits 15:0 or 31:16
header
001pLLLLBBBBBBBB
trailer
010zhvxxxxxxxxxx
1 hit cond
1FFFFCCCCCCCxfx0
2 hits cond
1FFFFCCCCCCCsfx1
1st hit clus exp
1FFFFCCCCCCC0DDD
1 hit clus exp
1xxxxxxx0xxx1DDD
2 hit clus exp
1xxxxxxx1DDD1DDD
flagged error
000xxxxxxFFFFEEE
raw data
011nnnxxWWWWWWWW
cond=condensed, clus=cluster
exp=expanded
Key :
x= do not care (The ROD fills these with 0’s)
B=BCID
W=raw data
C=cluster base address
D=3 bit hit data
E=ABC error code
F=FE number
h=header trailer limit error
L=L1ID
n=count of raw data
f=error in condensed mode data, 1st hit
s=error in condensed mode data, 2nd hit
p=preamble error
z=trailer bit error
v=data overflow error
8/20/2002
Condensed example
W1 001pLLLLBBBBBBBB
W2 010zhvxxxxxxxxxx
W1 001pLLLLBBBBBBBB
W2 1FFFFCCCCCCCxUx0
1FFFFCCCCCCCxfx0
0000000000000000
1FFFFCCCCCCCsfx1
010zhvxxxxxxxxxx
Header+1 hit
trailer
Header+2 hit
1 hit+trailer
Raw data example
W1 001pLLLLBBBBBBBB 011NNNxxBBBBBBBB
W2 011NNNxxBBBBBBBB 010zhvxxxxxxxxxx
Header+8bit
8 bit+trailer
Expanded example
W1 001pLLLLBBBBBBBB
W2 1xxxxxxx1DDD1DDD
W3 010zhvxxxxxxxxxx
Header+1 hit
2 hit +1 hit
trailer
1FFFFCCCCCCC0DDD
1xxxxxxx0xxx1DDD
0000000000000000
Condensed to raw on error
Correct
W1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hit
W2 1FFFFCCCCCCCxfx0 010zhvxxxxxxxxxx 1 hit+trailer
Error on 1 before 3rd hit (could have done it before
2nd hit
W1 001pLLLLBBBBBBBB 1FFFFCCCCCCCsfx1 Header+2 hit
W2 011nnnxxWWWWWWWW 010zhvxxxxxxxxxx raw+trailer
Format will switch to raw on the
following errors for the rest of the
event.
0 after beam crossing number
first bit of hit data leader is 1
bit before hit pattern is 0
13
Debug FIFO Implementation
39
39
Data
Input
OE
FIFO In
Register
Data Input
from
Formatter
4k words
FIFO Out
Register
FIFO
ADV OE RT
39
Link or
Diagnostic
Data to
EFB
39
Data
FIFO Data
Register
4k words
OE
FIFO Data
Register
Input
FIFO
ADV OE
Input
Memory
& XCVR
Control
8/20/2002
Control
FIFO
Operation
Advance FIFO and latch new input
Enable FIFO output data
Retransmit FIFO
Read or write data
Select 16 or 32 bit part of word
R/W
Bus Data
48 bits are read as
32 bit word bits 0:31
16 bit word bits 32:47
14
Debug FIFO Notes
• Debug Diagnostic Memories.
– Play or record data to perform diagnostics and tests.
– Start of record operation is triggered by the ROD Controller.
– Trapping (record) of Formatter Output Data can be
synchronized to triggers.
– The SCT and Pixel ROD implementation of the Debug
Diagnostic Memories is two groups of 4k x 39 bit FIFOs.
8/20/2002
15
Event Fragment Builder (EFB)
Halt
Output
Data
Valid
Format #
Data
39
Halt
Output
Data
Valid
To/From Controller FPGA
Format #
EFB Data Engine 1
43
16Kx46
Output Control Output
Almost Full
EFB Data Engine 2
43
16Kx46
Output Control Output
Almost Full
Error
Summary
Dyn
Mask &
L1-BCO
Engine FIFO
32w x 107b
2 ea.
Derandomizing
FIFO
4k x 16b
FIFO
43
Event Data
Event ID checking
Error Counters
Format event
R/W
Bus
8/20/2002
Event Data
Event ID checking
Error Counters
Format event
Control,
Status,
Event
Data /
Trig Type
Router Halt Output
Header FIFO
64w x 32b
Word
counts
FIFO
EV Fragment
Header/Trailer
32
Header/trailer Event
Fragment Generation
Data Type
Event Data
Valid
Data Output Control
Read Event
16
To/From Router FPGA
Data
39
.
.
.
.
.
..
EFB: Overview
Provides two parallel streams with data processing engine from half of the input
links to mask the Formatters link-to-link latency.
– Each engine has a 16k word output FIFO for derandomizing of data and to hold large
events.
– The two data processing engines helps achieve 40 MWords/sec bandwidth to the S-Link.
If “Router Halt Output” is issued, data can still flow through the EFB and fill the
output memory.
If the output memory is full back pressure will be applied to the Formatter.
EFB checks Event ID (BCID and L1ID) and generates errors. The BCID error
checking can be disabled by setting a bit in an EFB Internal Register.
Performs summary error counts with mask Counting of errors can be masked on a
link by link basis by writing the appropriate value to the Error Mask Registers in
the EFB.
The EFB formats input data event for the S-Link.
Generates S-Link compatible headers and trailers. The information for this
step is sent to the EFB from the Controller over the Dynamic Mask Data bus.
The Event ID and Dynamic Mask for a specific Event must be written to the
EFB before a Token can be issued to the Formatter to read out the link FIFOs.
8/20/2002
17
EFB: Event ID and Dynamic Mask
• The Event ID,Trigger Type, ECR, and Dynamic Mask are sent to the EFB from the
Controller. This data arrives after TIM has transmitted an L1 trigger, Serial ID and Trigger
Type data to the Controller and before a FE Command Pulse can be sent to the Formatters.
• The Event ID and Dynamic Mask is data set of 16 half words. The data format is shown
on the next slide.
• The EFB stores the 16 16bit words per event in a 4kword x 16 FIFO, transfers this group
of data to 3 FIFOs used for Event Header generation, L1ID/BCID checking, and
synchronization correction.
• The Event Header FIFO is 16 words by 64 bits, stores the L1ID, BCID, ECR, and Trigger
Type, and is used to build the Event Headers when the EFB is ready to send an event to
the Router
• Each EFB engine require a Dynamic Mask FIFO. This FIFO is 16 words by 64 bits, stores
the L1ID, BCID, and the Dynamic Mask bits required by each link for correction.
• The Event Header and Dynamic Mask FIFOs are ready for data processing 20 clocks after
the EFB receives the full data set from the Controller.
8/20/2002
18
EFB: Event ID and Dynamic Mask Data Format
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Word
Name
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
Engine FIFO
L1 ID [15:0]
[3:0]
ECR ID [7:0] & L1 ID [23:16}
no
BC ID [11:0] & 0000
[7:0]
TIM EvtType [1:0] & TrigType [7:0}
no
ROD Event Type [7:0]/(spare bits) [15:8] no
Dyn Mask [7:0]
yes
Dyn Mask [15:8]
yes
Dyn Mask [23:16]
yes
Dyn Mask [31:24]
yes
Dyn Mask [39:32]
yes
Dyn Mask [47:40]
yes
Dyn Mask [55:48]
yes
Dyn Mask [63:56]
yes
Dyn Mask [71:64]
yes
Dyn Mask [79:72]
yes
Dyn Mask [87:80]
yes
Dyn Mask [95:88]
yes
Dynamic mask 95:0
DM Bits [1:0]
DM Bits [1:0]
DM Bits [1:0]
is 2
“00”
“01”
“10”
Header FIFO
yes
yes
yes
yes
yes
no
no
no
no
no
no
no
no
no
no
no
no
bits per link, used to correct module synchronization.
no change to L1 ID for this event
Increment L1 ID for this event
Decrement L1 ID for this event
ECR = Event Counter Reset, TrigType = ATLAS Trigger Type
8/20/2002
19
EFB: Data Engine - 1 Engine Shown
Output FIFO Almost Full
Halt Output
Data
Valid
39
Data Link 1-48
Format #
Add or
subtract from
L1 number
Mode
bits add
or sub 1
from
L1ID
BCID /
L1ID
Check
L1ID
and
BCO
Error
Detect,
Count
Format
Data &
Count
Output FIFO Data
43
WR Output Data
Mask
BCID
Check
Error Summary Data
lnput bits to error summary
Dynamic
Mask &
L1-BCO
R/W
Bus
8/20/2002
Mask (96 ea.)
Header bit error (0)
Trailer bit error (1)
Flagged error (2)
Sync error (3)
Hit pattern error (4)
L1ID error (5)
BCID error (6)
Timeout error (7)
Almost full (8)
Data overflow (9)
If summary AND Mask =True
Count = Count +1
Summary Error =
Summary Error | current Error
Word count to
WC FIFO
Summary and Count
to Error FIFO
20
EFB: Data Engine
• Correct L1 ID Number
Based on the Dynamic Mask bits, the L1 ID increments or decrements. This is
used when a link is resynchronized on the ROD without stopping ATLAS
triggers.
• BC ID / L1 ID Check:
For Link Header, compare expected BCID and L1ID with the decoded data
and flag any errors. All other data types are ignored.
Separate Event FIFOs provide L1/BCID data so each EFB data path can work
independently on different events.
BCID check can be globally disabled if required.
• Error Detect:
See EFB Data Engine above for error bit definition.
Looks at data bit fields to identify errors.
Generates the error summary block for the event.
One maskable error counter per link.
One global error counter
BCID errors can be ignored for a link that has been resynchronized
8/20/2002
21
EFB: Data Engine
•
Error Bits Summary in the Link Error Mask Register:
Bit(0)
Bit(1)
Bit(2)
Bit(3)
Bit(4)
Bit(5)
Bit(6)
Bit(7)
Bit(8)
Bit(9)
•
=>
=>
=>
=>
=>
=>
=>
=>
=>
=>
Header bit error
Trailer bit error
Flagged error
Sync error
Hit pattern error
L1ID error
BCID error
Timeout error
Almost full
Data overflow
Format Data and Count:
1.
2.
3.
4.
Removes front end trailer words unless the trailer error flag or the almost
full flag was set.
Removes header from links with no hit data and no errors.
After unnecessary half words are removed, it packs all the data into 32 bit
words so there are no gaps in the data.
Counts number of words sent to the output FIFOs over an event and
writes the count into a FIFO.
8/20/2002
22
EFB: Header/Trailer Generation
Engine 1
FIFO:Error
Summary
Data
Engine 1
FIFO:Error
Summary
Data
Engine 1
Engine 2
FIFO:Word
Count
FIFO:Word
Count
Data FIFOs
Output
Control
Gather Error
Summary, Word
Counts and Event ID
data and raise Event
Data Valid
FIFO: Header
Event ID Data
Then Control Event
Readout
3
32
Event Header Data
32
Event Trailer Data
Data Type
Event Data Valid
Router Halt Output
Halt output suspends
readout
8/20/2002
23
To\from Router
Header/Trailer Event
Fragment Generation
EFB: Header Generation
• Header/Trailer Block Generation, Data Output Control:
– After the data counts are available from both EFB engines, an event is
delivered to the Router then to the S-LINK (data is pushed).
• A complete event must processed and written to the output FIFOs
before Event Data Valid is true.
• An Event Header block is first transmitted to the S-Link.
• Data is then transmitted from the output FIFOs.
• Finally, the summary and event fragment trailers are transmitted.
• Pause:
– If The DSP asserts Stop (calibration mode only) or the S-Link asserts Xoff,
data will not be transmitted, but the EFB can still process data and write to
the output memories.
– If any of the internal EFB FPGA FIFOs are almost full or the output FIFOs
are almost full, the FIFO controller will be told to pause and back pressure
will be applied to the Formatters.
– Output FIFOs are big enough to hold largest SCT/Pixel Event.
8/20/2002
24
EFB: Output Format
Event Header
Word
0
1
2
3
4
5
6
7
8
Contents
0xB0F00000
0xEE1234EE
0x9
TBD
TBD
L1 ID
BC ID
L1 TT
DET
Comment
(beginning of fragment)
(start of header)
(header size)
(format version number)
(source identifier)
(level 1 ID)
(bunch crossing ID)
(ATLAS Level 1 trigger type)
(detector event type ROD or TIM)
Event Trailer
Word
Contents
0
1
2
3
4
Error count
Error flags
0x2
nData
0x1
5
0xE0F00000
8/20/2002
Comment
(Status 1: count of words with error)
(Status 2: Bit error type occurrence)
(Number of status words)
(word count of data words)
(status block position: 0/1 =
before/after data)
(end of fragment)
25
.
EFB: Output Link Format
Link Data Format
SCT all data is in 16 bit
packets packed in 32 bit
words bit 31:0
Data bit 42-32 are valid for
header word only
bits 42:32
blktMMMMMMM
Key :
x= do not care (The ROD fills these with 0’s)
B=BCID
W=raw data
C=cluster base address
D=3 bit hit data
E=ABC error code
F=FE number
h=header trailer limit error
L=L1ID
n=count of raw data
f=error in condensed mode data, 1st hit
s=error in condensed mode data, 2nd hit
p=preamble error
z=trailer bit error
v=data overflow error
t=time out error
K=condensed mode
l=L1 error
b=BCID error
M=link number
8/20/2002
EFB Output (bits 31:0 bits for all words)
Name
Bits 15:0 or 31:16
header
001pLLLLBBBBBBBB
trailer
010zhvxxxxxxxxxx
1 hit condensed
1FFFFCCCCCCCxfx0
2 hits condensed
1FFFFCCCCCCCsfx1
1st hit clust expan 1FFFFCCCCCCC0DDD
1 hit clust exp
1xxxxxxx0xxx1DDD
2 hit clust exp
1xxxxxxx1DDD1DDD
flagged error
000xxxxxxFFFFEEE
raw data
011nnnxxWWWWWWWW
clust=cluster, exp=expanded
EFB Output examples for two events
Bits 42:32
Bits 31:16
Bits 15:0
blKtMMMMMMM
blKtMMMMMMM
blKtMMMMMMM
blKtMMMMMMM
blKtMMMMMMM
blKtMMMMMMM
Event header 9 words (Normal event)
link header
Condensed
Condensed
link header Note:link trailers
Condensed
condensed
are removed on no
link header
condensed
trailer error, link
condensed
condensed
by link
event trailer 6 words
Event header 9 words (a trailer error)
link header
Condensed
Condensed
link header
Condensed
condensed
link trailer
link header Note:error in trailer
condensed
condensed
condensed
link trailer Note:special trailer
fill
event trailer 6 words
If there are no errors in trailer and header and no data the
link header is not written.
26
Router Implementation: Top Level
event_info_in(45:0)
tristate_n_in(2:0)
data_valid_in(1:0)
46b
slink_data_out(31:0)
Write link and error info into Link Headers,
Format data for S-Link and Error Checking
slink_uctrl_out
slink_wen_out
S-Link Write Control
slink_cmd_reg(15:0)
slink_xoff_in
slink_bad_in
Stop Output
stop_output_out
stop_out_override
dsp0_sbsram_clk_in
dsp0_send_data_in
dsp0_sbsram_me_in
dsp0_trap_enable_in
dsp0_mask_dbr_in
Error Format Data
slink_data_out(31:0)
dsp2_cntrl_in(3:0)
dsp3_ cntrl_in(3:0)
router_bus_data_inout(15:0)
router_bus_addr_in(7:0)
router_bus_rnw_in
router_bus_strb_n_in
clk40_in
rst_n_in
8/20/2002
dsp0_dataframeready_out
header detect
Trap Event Data
trap_event_type_reg(31:0)
dsp1_cntrl_in(3:0)
dsp0_event_data_out(31:0)
DSP FIFO Buffer
DSP 0
dsp1_event_data_out(31:0)
dsp1_ dataframeready _out
dsp2_event_data_out(31:0)
dsp2_ dataframeready _out
dsp3_event_data_out(31:0)
dsp3_ dataframeready _out
DSP 1
DSP 2
DSP 3
Register Block
router_bus_registers
router_bus_ack_out
slink_test_out
slink_rst_out
stop_out_override
27
Event Header Data 32
Event Trailer Data 32
Output FIFO Data 43
Event Data Valid
Data Type
Slink on BOC
Router: Data Format and Trap
S-link Data
Read event data and format for
S-Link and DSP error data.
Detect Errors in the Link
Headers, and create error code
words for the Error Format.
Detect Event Headers to enable
trapping algorithms
DSP Error Data
Header Detect
4 event trapping filters
Filters can trap on event types ATLAS, TIM , ROD or error data
Each trap has a divided by factor that will reduce data to DSPs
Control & Status
for Block Xfer/
DMA / S-Link
4 DMA transfer engines. Each engine has a 1k 32b word derandomizing
DMA 1
8/20/2002
...
DMA 4
buffer. Each buffer is broken into 256 word blocks.
DMA moves full blocks of data. The blocks are filled as shown below:
small event
word count & continue bit in last word
large event
word count & continue bit in last word
exactly one block
word count & continue bit in block 2 last word
R/W
Bus
DMA
Cont.
28
. .
.
.
Router: Overview
Write link and error information into the Link Headers for the S-Link Data Format.
When a link header is detected in the data, the Router moves the link number and error bits
information from bits 45 down to 32 and overwrites the L1ID and BCID fields. (Shown on a
later slide)
Write to S-Link if Data Valid is true, and the User has not set the S-Link Masking bit or
setup to mask specific Events.
Detect errors in the Link Headers, and generate an error code for each link in Error Data
Format. (Shown on a later slide)
.
.
The error codes are stored in a block of registers for the duration of the Event. When the data
from EFB Engine 2 is done in the Router, the Error codes are written into the Event Trailer
Two bits error codes for each link
00 = no data
01 = good data
10 = BCID or L1ID error
11 = time out error
Router FPGA routes data to the S-Link and the Slave DSPs.
- Internal FPGA dual port memory are used to asynchronously connect to DSPs.
- During runtime monitoring, data is DMA’d into the internal SRAM of each DSP at
80 Mwords/sec. This channel must have the highest DMA priority or errors will be
seen in the transfers.
• DSP captures the data it needs, throws away the rest.
8/20/2002
29
Router: Overview
• The Register Block allows VME access to the Router over the ROD Bus. The registers in
the Router include control and status for the DSP trapping algorithms and the S-Link.
• Because the Router needs to monitor header words 8 and 9 to sense an Event to trap, the
Router uses a 9 clock pipeline for the Event Data path.
• If the S-Link asserts Xon/Xoff or S-Link Bad, the Router will mask the S-Link Write
Enable to stop data transmission to the S-Link.
• In Calibration Mode only, if the DSP FIFO is almost full, and the StopOutputOverride bit
is set, the Router can assert the StopOutput signal and temporarily apply back pressure to
the EFB and Formatters. The Router can not apply back pressure from the DSPs to the
ROD data path in Normal Data Taking Mode.
8/20/2002
30
Router: DSP Trapping Algorithm
• The Router supports four sets of trapping filters, one set per DSP. Each filter set is
comprised of two filters per DSP.
• The first filter per DSP takes priority over second filter. Each filter has a mask bit to select a
specific type of event (ATLAS, TIM, ROD).
• The first filter can be set to trap all events which disables the second filter.
• Each filter has a Trap Match (8 bits per filter) value used to match the event type.
• Each filter has a divide by factor (8 bits per filter) which allows pre-scalable trapping.
• The trap variables must be loaded by setting the Load Trap Bit in the Reset/Load Trap Register.
This is done to protect from changing the trap type while the Router is in the middle of trapping
an event.
• Each DSP DMA port has a bit to select between S-Link data format or error data format.
• Each DSP must set the Trap Enable bit to enable the Event trapping. Any the time that
the Trap Enable bit is cycled from 0 to 1, the trap is reset and the FIFO counters are reset.
• The Mask Data Frame Ready input to the Router masks the Data Frame Ready signal
until the DSP ISR is complete. If this bit stays high for an extended period, the FIFO will
become full, and DSP data will be lost. This does not affect the Real Time Data Path
signals
• When the an event has been trapped, a special event completion word is written to the
last location of the FIFO block (256 words per block) that the event occupies. This word
is made up of a pattern that does not appear in the data, and the total word count of the
event. If an event end exactly on a block boundary, the word is written to the end of the
next boundary.
8/20/2002
31
Router: S-link Module Output Data Format
The format for the module link header is changed. The L1ID and BCID is replaced with
error codes and the link number as shown below:
Key :
x= do not care (The ROD fills these with 0’s)
B=BCID
W=raw data
C=cluster base address
D=3 bit hit data
E=ABC error code
F=FE number
h=header trailer limit error
L=L1ID
n=count of raw data
f=error in condensed mode data, 1st hit
s=error in condensed mode data, 2nd hit
p=preamble error
z=trailer bit error
v=data overflow error
t=time out error
K=condensed mode
l=L1 error
b=BCID error
M=link number
Router Output (bits 31:0 bits for all words)
Name
Bits 15:0 or 31:16
header
001ptlbKxMMMMMMM
trailer
010zhvxxxxxxxxxx
1 hit condensed
1FFFFCCCCCCCxfx0
2 hits condensed
1FFFFCCCCCCCsfx1
1st hit clust expan 1FFFFCCCCCCC0DDD
1 hit clust exp
1xxxxxxx0xxx1DDD
2 hit clust exp
1xxxxxxx1DDD1DDD
flagged error
000xxxxxxFFFFEEE
raw data
011nnnxxWWWWWWWW
clust=cluster, exp=expanded
EFB Output (bits 31:0 bits for all words)
Name
Bits 15:0 or 31:16
header
001pLLLLBBBBBBBB
trailer
010zhvxxxxxxxxxx
1 hit condensed
1FFFFCCCCCCCxfx0
2 hits condensed
1FFFFCCCCCCCsfx1
1st hit clust expan 1FFFFCCCCCCC0DDD
1 hit clust exp
1xxxxxxx0xxx1DDD
2 hit clust exp
1xxxxxxx1DDD1DDD
flagged error
000xxxxxxFFFFEEE
raw data
011nnnxxWWWWWWWW
clust=cluster, exp=expanded
Replaced
L1ID and
BCID in
Event
Header
Bits 42:32
blKtMMMMMMM
8/20/2002
32
Router: S-link Event Header Output Format
The format for the header is not changed except for the addition of UCTL from the
EFB input. The format is shown below:
Event Header
Word
0
1
2
3
4
5
6
7
8
Contents
0xB0F00000 + UCTL
0xEE1234EE
0x9
TBD
TBD
L1 ID
BC ID
L1 TT
DET
Comment
(beginning of fragment)
(start of header)
(header size)
(format version number)
(source identifier)
(level 1 ID)
(bunch crossing ID)
(ATLAS Level 1 trigger type)
(detector event type ROD or TIM)
Event Trailer
Word
0
1
2
3
4
5
8/20/2002
Contents
Error count
Error flags
0x2
nData
0x1
Comment
(Status 1: count of words with error)
(Status 2: Bit error type occurrence)
(Number of status words)
(word count of data words)
(status block position: 0/1 =
before/after data)
0xE0F00000 + UCTL (end of fragment)
33
Router: Error Data Output Format for the DSP
One of the slave DSPs is concerned with error diagnostics. It will need all error flags,
module numbers and L1ID and BCID fields. The event header format is changes as
shown below:
Two bits error codes for each link
00 no data
01 good data
10 BCID or L1ID error
11 time out error
EFB Event Header
Word
Contents
DSP Error Header
Word Contents
0
1
2
3
4
5
6
7
8
0
1
0xB0F00000
0xEE1234EE
0x9
TBD
TBD
L1 ID
BC ID
L1 TT
DET
EFB Event Trailer
Word
Contents
0
Error count
1
Error flags
2
0x2
3
nData
4
0x1
5
0xE0F00000
8/20/2002
2
3
4
5
same
same
deleted
deleted
deleted
same
same
same
same
DSP Error Trailer
Word
Contents
0
same
1
same
2
value
3
value
4
value
5
value
6
value
7
value
8
same
Router error data output same as
EFB Output (bits 31:0 bits for all words)
Name
Bits 15:0 or 31:16
header
001pLLLLBBBBBBBB
All other bits are also the same
Comment
(beginning of fragment)
(start of header)
(header size)
(format version number)
(source identifier)
(level I ID)
(bunch crossing ID)
(ATLAS Level trigger type)
(detector event type ROD or TIM)
Comment
(Status 1: count of words with error)
(Status 2: Bitter error type occurrence)
(error status for links 1-16)
(error status for links 17-32)
(error status for links 33-48)
(error status for links 49-64)
(error status for links 65-80)
(error status for links 81-96)
(end of fragment)
34
Back End DSPs
• The Back End DSPs are TI TMS320C6701, floating point processors. The CPU clock is
160MHz, and each DSP has 32MBytes of SDRAM.
• The DSPs are removed from the data path allowing them to work independently without
affecting the ROD data taking
• The VME host can access the Slave DSPs through a 16 bit Host Port Interface.
• SDSPs are hardwired for HPI Boot and Little Endian byte order.
• The interface to the Slave DSP is through the Master DSP to the ROD bus.
• The Slave DSPs respond to D32 block transfers from the VME master. (2.2Mbytes/S)
• The back end DSP performs histogramming, spying, and event trapping tasks.
– Calibrations: raw data, histograms, summaries to reduce VME traffic and load on
crate processor, or comparison to reference histograms.
– Can trap specific events, i.e. embedded calibrations.
• Uses Trigger Type Info.
8/20/2002
35
ROD Controller: Implementation
48
BOC
FE Command
Streams
Busy to TIM
TIM
TIM Data
Event ID Error
EFB
Formatter
ROD
Controller
FPGA
EMIF
Bus
ASYNC
Event ID/ Dynamic Mask
DSP Serial
Ports(1:0)
Dec. Trig Count
Interrupts
Master DSP
TMS320C6201
HPI Bus
(VME)
Token and Mode Bits
FIFO Occupancy Status
Input
Memory
&
XCVR
Control
R/W Bus
to
FPGAs,
BOC,
FIFOs,
SDSPs
8/20/2002
GPI/O
PRM
FPGA
36
ROD Controller: Implementation
FormatterTrailerInformation
12b
Front Panel Status/Indicator LEDs
RodResourcesInterfaceProgramBits
8b
1b
FormatterReadoutModeBitsEncoder:
FrontEndOccupancyCounters:
TIM
EvntFrgHeaderAndControlEncoder:
AddressDecoder:
FrontEndCommandEncoder:
8b
ControlREG,
DebugMEM
Interface:
Formatter Readout Mode Bits
16b
EFB Dynamic Mask
12b
FE Command Streams
48b
En, Ack
R/NW
busBuffer:
busBuffer:
busBuffer:
Test Bench
Interface
BOCsetupBUS
8b
FormatterABUS
16b
FormatterBBUS
16b
EvntFrgBldrBUS
16b
RouterBUS
8b
busBuffer:
32b
busBuffer:
Rod Controller FPGA:
busBuffer:
busBuffer:
DSP CE0 Mtype=010b async
32b
BootMode=01101x
BootProcess=8bitROM defualt timings
MemoryMap=Map1
Interrupt 1b
RodController
DSPmodule:
TMS320C6201
busBuffer:
2b
SPI
busBuffer:
4b
busBuffer:
TCU
busBuffer:
VMEbusIInterface:
32b
32b
16b
busBuffer:
HPI
Interrupt 1b
FPGA Program Manager:
Reset Manager:
8/20/2002
EMIF
5b
5b
5b
5b
5b
5b
EVNTMEM R/W & Control 32b
BackEndDSP0
16b
BackEndDSP1
16b
BackEndDSP2
16b
BackEndDSP3
16b
BOOT Strap:
PROM
32b
DSP CE2 Mtype=011b sdram
DSP CE3 Mtype=011b sdram
32b
32b
DEBUGMEM R/W & Control 32b
DSP CE1 Mtype=000b rom
8b
RodResourcesInterfaceProgramBits, rst
AFormattersProgramBits, rst
BFormattersProgramBits, rst
EventFragmentBuilderProgramBits, rst
RouterProgramBits, rst
dspReset[4..0]
INMEM R/W & Control
DSP MEMORY:
16Meg Bytes
SDRAM
37
ROD Controller: Master DSP
•
The Master DSP is a TI TMS320C6201B, fixed point processor. The CPU clock is
160MHz, and the DSP has 16MBytes of SDRAM (upgradeable to 32MBytes).
•
The VME host can access the Master DSP through a 16 bit Host Port Interface.
• The Master DSP is hardwired for 8bit ROM Boot and Little Endian byte order.
• The Boot ROM can be accessed on the VME Bus so that new versions of code can be
downloaded and started when required by the DAQ system.
• The Master DSP responds to D32 block transfers from the VME master.
(2.2Mbytes/S)
•
The Master DSP provides the main path of communication to the ROD from the VME
through the Host Port Interface.
•
The ROD Bus, controlled by the ROD Controller, is connected to the Master DSP EMIF
asynchronous port, and utilizes CEn memory space 0. This is the path for access to all of the
data path FPGAs, the Slave DSPs, the diagnostic memories, and the BOC.
•
The Host Port Interrupt out from the DSP is wired to a VME interrupt input.
•
The Master DSP has 2 serial communication ports (SP0 and SP1) that are used in the
ROD to send serial commands to the FE Modules
8/20/2002
38
ROD Controller: Master DSP
•
The Master DSP allows high level control operations to be performed on the ROD.
– Because the control path of the ROD is not required to operate in real-time,
performance is less critical.
– Using Primitive Lists, the MSDP performs control of ROD functional setup (the
specific mode of the ROD), histogramming, spying, and event trapping.
Periodic Reset
• When the periodic reset is issued, the ROD raises busy and interrupts the master DSP. The
master DSP then waits for all events to be played out of the front end chips by monitoring
that the counts of all events pending is zero. Configuration as needed is then performed
within approximately 1 thousandths of a second. Near the end of the cycle, the reset is
issued to the modules and busy is removed.
• It is also possible to continue triggers while the ROD is correcting the state of the front
end modules. In this mode busy would not be raised while the modules are configuring.
Triggers would just return empty event with an error flag.
• The exact details of the periodic reset if needed will have to be worked out
with ATLAS Trigger/DAQ.
8/20/2002
39
ROD Controller FPGA
Token and Mode Bits
To
To EFB
Formatter
Input
Memory
&
XCVR
Control
Back-of-Crate Card
r/w
48
Control 48
links
48
2 input
or
Circuits 48
r/w
Generate
Dynamic
Mask
Trigger
Operation
Control
48 bit
Mask
& Input
bit by bit
48 bit
Mask
& Input
bit by bit
r/w
Event ID
Data
Trig Type
Dynamic
Mask
FIFO
Serial
Expansion
Event Data
ROD BCID, L1ID
and Trigger Type
in ROD based
triggers
From
Formatter
From
Formatter
Dec. Trig Count
To data
RCVR
Decr
96
4
bit
Incr
Counters
All
Zero
r/w
Busy to TIM
TIM Data
TIM
FIFO
4k W 8b
r/w
R/W bus
to
FPGAs
and BOC
8/20/2002
R/W
Control
Host Port Interface (HPI) Bus
Registers
Master
DSP r/w
DSP DSP
Serial Serial
Port 1 Port 0
Back End
DSPs
40
ROD Controller: FPGA
• The main function of the ROD Controller FPGA is to control the operation of the ROD and
coordinate the flow of data on the board in real time.
• Listing of the real time functions:
1.
2.
3.
4.
5.
Process commands from the TIM: Trigger, fast and slow commands and calibration signals
Process serial data from the TIM: Event ID and Trigger Type.
Count and store ECR commands, and include the count in the Event Header in the L1ID field.
Send FE Commands to the Modules.
Supply the Formatter Mode Bits and the EFB Event ID and Dynamic Mask Bits to the Data Path when an
L1 trigger has been received and the appropriate data has arrived from the TIM.
6. Count and store triggers, and issue them to the Data Path when ROD is ready to process an Event.
• Listing of quasi real time functions:
1.
2.
3.
4.
5.
6.
Issue and count corrective triggers sent to the data path.
Issue Corrective Mode Bits for an Event.
Issue a corrective Dynamic Mask and transfer it to the Default Mask.
Change the masking of FE Command Streams.
Interrupt the Master DSP to indicate the type of trigger sent to the data path.
Trap Configuration or Event data in the Input Diagnostic FIFOs.
• Listing of non real time functions:
1. Read/Write to the internal memory mapped register set.
2. Read/Write to the BOC, diagnostic FIFOs, and the Slave DSPs.
8/20/2002
41
ROD Controller: FPGA
• The controller has an internal test bench block and a set of diagnostic registers to allow the
ROD to be tested in a stand alone mode. This is required for initial debugging and some
system testing.
• Listing of the Test Bench functions:
1. Internal FIFO to mimic the TIM inputs
2. Separate states to run stand alone test on the ROD.
A. Trap all Link inputs to Inmem FIFOs
B. Trap all Link inputs to Inmem, send to Formatters, trap Formatter Output to Debug FIFOs, play data through the ROD
to the SDSPs (Event ID data must be provided by a TIM or internal registers)
C. Load Inmem and TIM FIFOs with simulation data files and play through the ROD
D. Load Inmem and TIM FIFOs with simulation data files and play through the ROD with a set number of retransmits (1
to 65536).
E. Load Inmem FIFOs with simulation data files and trap in the Debug FIFOs
F. Load Debug and TIM FIFOs with simulation data and trap in the Event FIFOs
G. Load Event FIFOs with 1 Event of simulation data and trap in the Slave DSPs
H. Load Inmem FIFOs with 1 Event of simulation data, set the internal Event ID, Mode Bits and Dynamic Mask registers
to the required values, send L1 Triggers over SP0 or SP1, trap in the SDSPs
• Event ID FIFO stores a list of expected trigger IDs that the EFB can check against.
• Process the DSP Serial ports to provide for module configuration and ROD based trigger
information.
8/20/2002
42
ROD Controller: FPGA
• The controller has a memory mapped set of internal registers required for all ROD
operations.
• Listing of the internal registers by functional group:
1. Main control and status
2. FE Command Mask settings
3. Calibration Commands
4. Formatter Mode Bits/EFB Event ID and Dynamic Mask control and status
5. Formatter Mode Bits /EFB Event ID and Dynamic Mask Look-up Tables
6. Diagnostic control, status and counters
7. DSP interrupts
8. FE Occupancy Counter Control and Status
9. Diagnostic Event ID, L1ID and BCID values
10.Corrective Events FIFO access
• Can execute a long command sequence without tying up the ROD Crate Controller.
• Keeps count of the number of pending triggers to sense all events processed and for
diagnostics. All events processed is used in a periodic reset to sense that the data is not in
the front end chips.
• The ECR Counter can only be reset by writing a 1 to the ECR reset bit in the RCF Spare
Control Register
8/20/2002
43
ROD Controller: FPGA
• Normal Data Taking
• The ROD receives L1A, Event ID, and Trigger Type signals from the TIM.
• With a latency of 2 clocks after receiving the L1A, the ROD sends the L1 Trigger
command to the FE modules on the output links that are active.
• The FE Trigger Counter and the FE Occupancy Counters are incremented by 1.
• The Formatter Mode Bits Encoder sends the default Mode Bits to the Formatters.
• The EFB Event ID Dynamic Mask Encoder receives the deserialized ID and Trigger
Type data, processes and sends the Event ID and default Dynamic Mask data to the
EFB.
• The Token is issued to the Formatters, the FE Trigger Counter is decremented by 1,
and an interrupt is sent to the Master DSP to indicate that a default trigger has been
processed by the ROD Controller.
• The FE Occupancy Counters are decremented when a Link Formatter writes a
trailer into the Link FIFO.
8/20/2002
44
ROD Controller: FPGA
• Resynchronization of modules during data taking
• Slave DSP determines what link requires L1 ID adjustment. The information is
transferred to the Master DSP.
• The Master DSP writes the correction information to the Corrective Mode Bits
and Corrective Dynamic Mask Registers in the ROD Controller FPGA then sets
the “New DM Mask Ready” bit in the RCF Main Control Register.
• When the next L1A is detected from the TIM, the ROD performs the steps listed
in the Normal Data Taking description with the following exceptions:
• The Corrective Trigger is trapped in the Corrective Events FIFO.
• The Mode Bits and Dynamic Mask Encoders issue the corrective Mode Bits and
Dynamic Mask, and send an interrupt to the Master DSP to indicate that a Corrective
Mask has been sent. This mask is valid for 1 L1 Trigger only for the Formatter. It is
permanent for the EFB. (This feature is in the VHDL, but not tested)
8/20/2002
45
ROD Controller: FPGA
• FE triggers from Master DSP Serial Streams
• The ROD Controller FPGA contains a BC counter, L1 counter, and Event Type
Register per serial stream. When the Calibration Trigger Decoder block detects
a L1 Trigger from either serial stream, the L1 counter is incremented, the value
of the BC counter is latched into a register, and the Event Type Register value is
written to the Event ID FIFO.
• The Default Mode Bits and Dynamic Mask Look-up tables are used when a
trigger is detected on SP0
• The Corrective Mode Bits and Dynamic Mask Look-up tables are used when a
trigger is detected on SP1
• Fast Reset Commands will reset the counters as specified. A BCR command
will reset the ROD BC counter and the FE module BC counter simultaneously,
an ECR will not produce this result because the BC reset happens after other
resets are performed by the chip controller.
• The BC ID register can be set to a static value.
• This mode is used for triggers during histogramming and diagnostics.
8/20/2002
46
ROD Program Reset Manager: Implementation
To
VME
A(31:0)
A(31:0)
D(31:0)
Control
AckN
Control
AckN
HRDYN
Control
D(31:0)
To
MDSP
HPI
D(31:0)
HD(15:0)
HHWIL
HCNTRL
HDS1
Master
DSP
Control
HRDYN
ID(7:0)
Internal
IA(20:0)
ID(7:0)
Flash
Interface
IA(18:0)
R/W
Registers
FPGA
Configuration
Data
Control
FLASH
12MBytes
ID(7:0)
VME
Address
Decoder
and
Data
Bridge
Control
Status
FPGA
Cnfg/RST
Controller
90ns
FPGA Config Control
FPGA Config Status
FPGA Reset Control
DSP Reset Control
GP I/O
To
FPGAs
To ROD
Controller
8/20/2002
47
ROD Program Reset Manager: Overview
• Controls power on sequence of the ROD by configuring the data path
FPGAs in a specific order, and controlling resets to the DSPs.
• Allows VME host to configure and reset Data Path FPGAs.
• Allows VME Host to reset all DSPs.
• Allows VME Host to monitor status of the configuration status of Data
Path FPGAs
• Acts as a bus-bridge between the ROD and the VME Bus.
• Enables D32 transfers between the VME and the Master DSP (16 bit
HPI)
• Controls the read and write access to the 12 Mbytes of Flash memory
that stores the FPGA configuration data.
• Stores the specific Serial Number of each board
8/20/2002
48
ROD Program Reset Manager
• Power on sequence of the ROD:
• Configure ROD Controller FPGA
• Activate Master DSP
• Configure Formatters, EFB and Router
• Active Slave DSPs
• VME to Master DSP D32 access:
• Write Access: The PRM receives a D32 data word from the VME bus, splits it
into 2 half words and sends the first half word to the HPI data port. After
HRDY_N indicates that the HPI is ready for the second half word, the PRM
toggles the half word ordering bit, moves the upper word to the lower half of
the bus and writes it to the HPI data port. At this point, the PRM asserts
LACKN to the VME Chipset, and is ready for the next transfer.
• Read Access: Same as above functionally.
8/20/2002
49
ROD Program Reset Manager
• Controls the configuration and reset of Data Path FPGAs, resets of all DSPs, and status of
configuration and reset signals
• Writes to the Configuration and Reset registers on the VME bus allow the
configuration and reset of all of the Data Path FPGAs and DSPs
• Reads to the Status registers show the state of configuration and reset signals
• Controls the read and write access to the 12 Mbytes of Flash memory that stores the
FPGA configuration data.
• Internal registers and algorithms allow for simple access to the Flash memory.
• The Address and Write Data register is used to set the memory address for writes and
reads and the data to write if required
• The Read Data register stores the data from the last read to the Flash
• The control register allows reads, host algorithm writes and PRM algorithm writes
• The host algorithm write gives complete control to the host to program the part. This is
useful during chip or sector erase functions.
• The PRM algorithm write keeps control of the programming operation in the PRM. It is
most useful for programming the part because it takes care of the repetitive commands
required to program a Flash, and only requires address and data from the host.
8/20/2002
50
ROD Clock Distribution: Implementation
BOC CLK+
From
BOC
BOC CLK-
PECL
=>
LVTTL
PLL
SEL CLK
Clock
Distribution
IC
To
ROD
36
1 In / 36 Out
40MHz
XTAL
OSC
8/20/2002
To
S-Link
S-LINK CLK
51
ROD Clock Distribution: Overview
• Accepts the BOC clock PECL input, and converts to
LVTTL to match the input of the Phase Locked Loop used
on the ROD.
• Internal crystal oscillator can be used in stand alone mode
• Distributes separate clock lines for all parts that require
clocking
8/20/2002
52