Timing and Event System
S. Allison, M. Browne, B. Dalesio, J. Dusatko,
R. Fuller, A. Gromme, D. Kotturi, P. Krejcik, S.
Norum, D. Rogind, H. Shoaee, M. Zelazny
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Outline
Overview – HW Block Diagram and Reqts
Progress since Oct, 2006
Outstanding Issues
Task List
HW Status & Long-Haul Fiber Issues
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Timing Architecture
~
Linac main drive line
FIDO
119 MHz
SLC
MPG
P
I
N SLC
O
E pattern
C
T
Events
& Data
360 Hz
E
V
G
E Events I
V & Data O
R
C
P S
N T m
E B P
T
Apr 16, 2007
Facility Advisory Committee Meeting
TTL-NIM
convert.
SLC Trigs
TTL
LCLS Digitizer
LLRF
BPMs
Toroids
Cameras
GADCs
Old klystrons
Stephanie Allison
saa@slac.stanford.edu
Hardware Block Diagram
2007 Commissioning
Modulator
Triggers
Existing
Control
System
RF
Timing
Future
MPS
Beam Path
Acq and BPM FEE
Calibration
Triggers
Timing
Crate
F F P
E
360Hz
A A N
Fiducial V
119MHz
Clock
I
O
N N E
G
1 2 T C
Beam Rate,
Beam Path
E
V
... R
3 1 1 1
8
…
Acc and
Standby
Triggers
E
V
LLRF R
Crate 1
E
V
R
2
E
I
V
O
R
C
3
Fiber Distribution:
Timing Pattern, Timestamp, Event Codes
Trigger
F E C I
Laser
Steering A V A O
N R M C
Crate
E I
BPM V O
Crates R C
PADs and PACs
C
A
M
8
Apr 16, 2007
Facility Advisory Committee Meeting
I
O
C
8
TORO FEE
F
E I
Toro
A
Farc
V O
Crate N R C
4
Triggers
Profile E C C I
Monitor V A A O
R MM C
Crate
E
V
… R
1 1 2 1
4
…
Triggers
C
A
M
7
Stephanie Allison
saa@slac.stanford.edu
C
A
M
8
I
O
C
4
Event System Requirements
Event Generator IOC:
Send out proper event codes at 360Hz based on:
PNET pattern input (beam code and bits that define beam path and other
conditions)
Add LCLS conditions such as BPM calibration on off-beam pulses , diagnostic
pulse etc.
Future – event codes also based on new MPS and user input
Send out system timestamp with encoded pulse ID from PNET
Send out PNET pattern to be used by SLC-aware IOCs
Manage user-defined beam-synchronous acquisition measurement
definitions
Check for match between user meas definitions and input PNET pattern at
360Hz and tag matches in outgoing pattern
MPS Algorithm Processors and Master Pattern Generator (MPG):
Rate-liming logic including beam “burst” and “single-shot” modes
Send out PNET pattern to EVG and CAMAC controls (for modulator
triggers)
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Event System Requirements, cont
Event Receiver IOC:
Receive event pattern 8.3 msec before corresponding
pulse
Set trigger delays, pulse widths, and enable/disable via
user requests (not yet done on a pulse-by-pulse basis)
Perform beam-synchronous acquisition based on tags
set by EVG in the pattern
Perform beam-synchronous acquisition for the SLCaware IOC based on the PNET part of the event pattern
Set event code per trigger (triggering done in HW when
event code received)
Process pre-defined records when specific event codes
are received (not yet available)
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
EVG Event Time Line – 4 Fiducials
360Hz
Fiducial
F0 (n=0)
Time (msec) 0 1.0
HW starts sending
R0
event codes, starting
with fiducial event code
Receive Fn+3 PNET,
determine Fn+3 LCLS P0
pattern, and advance
pipeline (n-2->n-1->n)
Send LCLS pattern
Set Event Codes in Alt
RAM based on the last
patterns for Fn+1
120Hz BEAM
L0
E1
F3 (n=3)
F1 (n=1)
2.8
R1
F2 (n=2)
5.6
R2
8.3 9.3
R3
P1
P2
P3
L1
E2
L2
E3
B-3
Apr 16, 2007
Facility Advisory Committee Meeting
L3
E4
B0
Stephanie Allison
saa@slac.stanford.edu
Trigger Event Time Line – 1 Beam Pulse (B0)
Record processing (event, interrupt)
Hardware Triggers
Triggering
Event Codes
Event Timestamp, Start
pattern records,
and BSA ready
Receive pattern for
3 pulses ahead
Kly Standby Beam
Acq
Kly Accel Trigger
Fiducial Event
Received
Fiducial
B0
F3
Time (usec)
0
18
0.3 100
Apr 16, 2007
Facility Advisory Committee Meeting
500
1023
Stephanie Allison
saa@slac.stanford.edu
Tallies for 2007 Commissioning
# EVRs = 31 (mostly PMC)
# IOCs with EVRs = 28
# EVR Fanouts = 4
# Hardware Triggers = 120
and counting
All TTL except 2 NIM triggers
for QDCs
Most require short cables
(except LLRF)
EVR with clock not yet
available
All acquisition electronics
using either internal clocks,
clock output from the RF timing
system, or other external clocks
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Progress since Oct, 2006
Installation in sector 20 and 21, commissioning inprogress, most timing delays are “iced-in”
Finished hardware bench testing
Finished PMC-EVR driver
EVG sequence RAM programming at 360Hz,
conditional logic is primitive but good enough for
now
Event pattern records and timestamp distribution
on the EVR IOCs
Finished cabling plans and documentation
Finished beam-synchronous acquisition
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Trigger Control Display
Trigger Event Assignment Display
EVG Displays
Issues
Outstanding problem (presumably software) where
CPU hangs when EVR interrupts are enabled.
Some IOCs running without interrupts (hardware
triggers but without timestamps, BSA, or event
code IRQs).
Documentation and help screens sorely lacking.
EVG IOC doesn’t know when there’s no beam,
only when beam is possible – limits trigger
conditions.
Software not yet in place to handle hardware and
communication errors.
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Tasks
Long Haul Fiber Test – next month
BC2 (sectors 21 to 30):
60 EVRs to order - 34 VME, 26 PMC
5 fanout modules (1 every 2 sectors)
$250K, needed by August
SW - only DB and display work planned
Still some existing database and display work to do
SLC-aware IOC BPM integration still in progress
Software - fix hang problem, add HW error detection and recovery
Beyond BC2:
New LCLS MPS interface
BSY/LTU/Undulator Procurement and Plans
Changes to EVR firmware to change behavior when multiple triggers on a single
pulse
Generic trigger delay scan package
Bunch charge change on pulse-by-pulse basis?
More complex conditional logic for event codes (EVG) and pulse-to-pulse changes in
trigger attributes?
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Event System – Hardware Status
The New LCLS Event System has been installed in
the Injector and S20
We have successfully integrated it with the SLC
Timing System (required some mods to EVG)
The Event System HW is performing as expected
At the first order / Still in its infancy
Other than EVR IRQ Problem / HW working
The Next HW Challenge is Long-Haul Distribution
of Timing
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
IN20 Installation
Diagram shows IN20 /
LI20 Event System
installation
Top-Level shows
connections to SLC
Timing, MDL
The EVG Crate is
shown with its fanouts
to all subsystems
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Event System HW: Long-Haul Dist.
For BC2 & beyond, the biggest challenge is LongHaul distribution of timing data via the Fiber-Optical
(F-O) Links
Why?
MRF Event System is designed around Multi-Mode SFP
(Small Form-Factor Pluggable) F-O links which allow a
maximum fiber length of ~300 meters
Our requirements include runs of several Kilometers (at
least)
Cannot daisy-chain the event F-O links: exceeds the
jitter budget
Temperature effects on long fibers – drift, but how bad?
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Event System HW: Long-Haul Dist.
Proposed Solution: New HW and some testing
Single-Mode, pin-compatible SFP modules are
commercially available (Agilent ACFT-57R5)
All us to go ~10KM
Should plug right into our event system HW
Have ordered some, will test when they arrive
Still does not solve temperature-induced phasedrift problem
Propose to solve by either/and:
Running Long Fiber in temperature-controlled environment
Re-Syncing EVRs to a locally-distributed 119MHz source
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Event System HW: Special Test
Long-Haul Event Fiber System Test:
What it will determine:
Function of Single-Mode SFP Modules
Drift, Trigger time jitter, Phase Noise
How:
Compare two EVR’s triggers & RF Clocks
One EVR Short-Haul
Other EVR Long-Haul
Also compare to reference RF Clock
When:
As soon as possible, have all components except EVRs
New RF-recovery EVRs due in at end of April
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu
Event System – Long-Haul Fiber System Test
1.5KM SM Fiber
EVG
Fiber
Fanout
MCC
Optical
Coupler
1.5KM SM Fiber
Trig
EVR_1
RF Clk
(119MHz)
476MHz
Clk/Trig
Sync Chassis
Compare:
- Jitter
- Phase Noise
- Phase Drift
Fiducial
Trig
EVR_2
3M MM Fiber
RF Clk
(119MHz)
All Components located
in RF Hut
119MHz
Ref Clk
Apr 16, 2007
Facility Advisory Committee Meeting
Stephanie Allison
saa@slac.stanford.edu