Photon Systems
Controls
and Data-Acquisition
Gunther Haller
Particle Physics and Astrophysics Division
Photon Control and Data Systems (PCDS) Group
14 May 2008
‹#›
17 June 2008, FAC Meeting
v5
Photon Systems Controls & DAQ
Gunther Haller
haller@slac.stanford.edu
Outline
LCLS versus LUSI Responsibilities
PPS/HPS/LSS
XTOD Diagnstics
AMO Instrument Control and Data Acquisition
LUSI Instruments and Diagnostics
NEH Infrastructure & Networking
Femto-Second Laser Timing System
2D-Detector Control and Readout
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
FEE and NEH/FEH Hutch Assignments
**
**
XPP
Front-End Enclosure
with FEE-XTOD
XCS
CXI
**
AMO: LCLS
XPP, XCS, CXI: LUSI
** SXR Imaging, SXR Pump-Probe,
HEDS are not funded at this time
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
LCLS & LUSI
Common photon-area Controls and DAQ System Design for LCLS and LUSI
LCLS Controls & DAQ W.B.S 1.6.2
AMO experiment (NEH, hutch 2)
All controls and DAQ
Common services for all hutches
Examples
PPS/HPS/LSS
120 Hz beam-quality data interface
Machine protection system interface
Network interface
Local science data online processing & cache
2-D detector
Control and DAQ for detector
LUSI Control & DAQ W.B.S
XPP, XCS, CXI Instruments, Diagnostics
Examples of documents released
1.1.523 XES-LUSI ICD (includes Controls and Data systems)
1.1-516 XES Photon Controls to Electron Beam Controls System ICD
1.1.517 XES Photon Controls to Electron Beam Controls MPS ICD
Additional ICD’s in draft form, will be released according to P3 schedule (Network, etc)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
PPS and HPS
FEE PPS
Personnel Protection System for FEE and NEH/XRT/FEH plus Hutch Protection
System
HPS based on SSRL model with some modifications
Preliminary Design Review held 2-29-08
Final Design Review to be held ~July 08
Racks for NEH and FEH received
PLC hardware for NEH received
FEE/NEH software being written
Final Design Review in July 08
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
XR Diagnostics in Front End Enclosure
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
XTOD Controls
Final Design Review (FDR) of all
controls except vacuum held Feb
14, 08
FDR of complete XTOD vacuum
systems held March 13, 08
Have added EPICS support for
several devices
Most instrumentation for Fall 08
delivery to SLAC are installed in
racks at LLNL and are in testing
Services will be available at
SLAC to support commissioning
(see later)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
AMO Instrument Control
Requirements listed in Engineering Specification Documents (ESD’s)
1.6-108 AMO Controls ESD, status: released
1.6-109 AMO DAQ ESD, status: released
Interfaces specified in Interface Control Document (ICD)
1.1-515 XES AMO Controls ICD, status: released
Held Preliminary Design Review last Month
http://confluence.slac.stanford.edu/display/PCDS/AMO-PDR
Including identification of each component to be controlled and data to be acquired
~10 cameras
~50 stepper motors, almost all smart motors (serial port)
~ 50 channels of high-voltage bias supplies
~ 8 sets of vacuum gauges, 2 valves
misc other controllers, but mostly straight forward serial devices
All controllers identified, example motion and vacuum next slides
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
AMO Instrument Motion Control
E-STPR1
J-STPR2
H-ROTR2
H-STPR1
Beam stops (A1)
Beam Viewing (B1-B4)
Gas Jets (C1, C2)
HFP Focus Verification(D1- D4)
Equipment Stand (E1-E3)
Detectors (F1-F12)
Focusing Optics (H1, H2)
Shutter (J1, J2)
Total Energy (K1)
STPR, Stepper motor
ROTR, Rotational
SHUT, Shutter Control
PHUNU, Pneumatic
J-SHUT1
E-STPR2
E-STPR3
F-STPR12
F-STPR7
B-STPR3
D-ROTR1
D-SPTR2 F-STPR1-5
C-STPR2
F-STPR11
B-PHNU1
P
D-STPR3
D-STPR4
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
K-STPR1
C-STPR1
A-PNEU1
‹#›
Gunther Haller
haller@slac.stanford.edu
B-PHNU4
AMO Instrument Vacuum Control(1)
• Ion Pump: IONn
•Turbo/Scroll: PMPn
CCG1
•Titanium Sublimation Pump: TSPn
CCG2
•Cold Cathod Gage: CCGn
GVV2
•Thermo-Parini Gage:TPGn
• Hot Filament Gage: HFGn
•Foreline Valve: FVVn
•Gate Valve: GVVn
GVV1
ION1
ION2
TSP1
•Residual Gas Analyzer: RGAn
CCG8
CCG6
GVV5
CCG4
CCG3
GVV3
FVV4 RGA1
RGA2
PMP4
TPG1
GVV4
CCG5
PMP3
FVV8
PMP5
P
PMP2
GVV6
FVV5
PMP1
FVV1
CCG7
GVV7
TPG5,6
PMP6
FVV6,7
FVV2-3
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
TPG2-3
TPG4
‹#›
TPG7
FVV9,10
Gunther Haller
haller@slac.stanford.edu
XTOD
AMO Instrument Vacuum Control(2)
SINGLE
PULSE
SHUTTER
FOCUSING
OPTICS
BEAM
VIEW
DIANOSTICS SECTION
HIGH FIELD PHYSICS
CCGX
P
P CCG2
P
P CCG4
P CCG3
P CCG6
FVV5
P
P CCG8
CCG7
GVV6
BYPASS
GVV5
UP TO
AIR
GVV2
GVV1
GVVX
CCG1
UP TO
AIR
FVV4
ION1
ION2
GVV7
GVV3
PMP1
GVV4
GVV4
FVV8
FVV1
PMP3
GAS
JET
P
TPG1
PRIM
PMP
PMP4
NOTES:
P
FVV10
FVV7
TPG4
P TPG5
Photon Systems Controls & DAQ
SAMPLE
SOURCE
PRIM
PMP
PRIM
PMP
FVV3
PRIM
PMP
SAMPLE
SOURCE
(1) ALL PRIMARY PUMPS ARE ALWAYS ON
(2) VACUUM GROUP WILL ROUGH SINGLE PULSE SHUTTER AND FOCUSING OPTICS
(3) FORE LINE VALVES ARE CLOSED BEFORE TURBO PUMPS ARE TURNED OFF
(4) TURN ON TURBO PUMP AND FORE LINE VALVE TOGETHER
(5) ALL PARINI GAUGES ARE ONLY MONITORED
(6) ION PUMPS ARE MANUALLY POWERED ONLY
(7) ANY VACUUM TRIP WILL CAUSE ALL VALVES TO CLOSE IN THE ASSOC. SECTION
(8) TURBO PUMPS PMP1,2,3 ARE TURNED ON AND OFF TOGETHER
(9) TURBO PUMPS PMP4 & 5 ARE TURNED ON AND OFF TOGETHER
(10) ALL GATE VALVES ARE SLOW
17 June 2008, FAC Meeting
PMP6
P
FVV2
TPG2
P
FVV6
P
TPG3
TPG6
FVV9
P
CCG5
PMP2
PMP5
‹#›
GVV = GATE VALVE
FVV = FORE LINE VALVE
PMP = TURBO PUMP
ION = ION PUMP
PRIM PMP = PRIMARY PUMP
CCG = COLD CATHOD GAUGE
TPG = PARINI GAUGE
Gunther Haller
haller@slac.stanford.edu
P
TPG7
AMO Rack Profile (3 racks)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Current NEH Hutch-2 Layout
AMO
CableTrays
Racks
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Lab Test Setups
Have racks with many controllers in the lab
Incrementally add controllers and software
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
LUSI Components
Have draft ESD’s for LUSI instruments and diagnostics
Detailed lists of components
http://confluence.slac.stanford.edu/display/PCDS/LUSI+Document+
Page
ESD’s and ICD’s scheduled to be released mid July (one month
from LUSI CD-2 review)
Example
XPP
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Diagnostics/Common Optics e.g. XCS Instrument
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Common Diagnostics
E.g. Intensity, profile monitor, intensity
position monitors
For detection use Canberra PIPS or IRD
SXUV large area diodes (single or quad)
Amplifier/shaper/ADC for
control/calibration/readout
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Intensity-Position
- Back Scattering Geometry
Quad-Detector
R2
q1
q2
R1
Target
L
• Four-diode design
• On-board calibration circuits not shown
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Infrastructure & EPICS Drivers
Server Room (basement level of NEH) racks in hand, to be installed in NEH in August
Used for networking, servers, on-line processing and science data storage
Additional cable-trays will be installed in July
Fibers from Computer Center (SCCS) and MCC to NEH Server room in hand, to be
installed in July
First client will be XTOD in FEE
Hutch-2 AMO racks ordered and received, are in lab and will be built up over next few
months
EPICS drivers
Have driver lists
Only ~10 EPICS drivers remain to be written & tested
Instrumentation for AMO chambers can be connected as early as 12-08 (for instrument
build-up and check-out)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Example: XTOD EPICS Drivers
Device
EPICS Driver/Device
Support Status
Interface
Actual
Completion
Date
EVR
Existing. Adopted.
PMC
1/29/2008
Hytec Motion Controller 8601/8002
Existing. Adopted.
IndustryPack
12/21/2007
AC LVDT Highland V550
Existing. Adopted.
VME
12/21/2007
Acromag IO (AVME9670, 231/330/440/445)
Existing. Adopted.
IndustryPack
12/21/2007
Solenoid Controller
Existing. Adopted.
Digital IO
12/21/2007
Beckhoff (temperature)
Done
Ethernet
1/3/2008
ISEG-HV
Done
VME
12/21/2007
Kramert ECM505
Done
VME
12/21/2007
ICS-121
Done
VME
1/25/2008
ICS-130
Done
VME
1/28/2008
Acqiris DC282 RTEMS support
Done need more test
cPCI
1/29/2008
Cascade 512B Linux EPICS support
Under development
PCI-X
1/31/2008
Acqiris DC282 Linux EPICS support
Done
cPCI
3/8/2008
AB PLC
Exist
Ether/IP
Digitizer for K-monochromator
Eurotherm 2404 temperature controller
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
XTOD
EPICS con’t
StreamDevices:
Gamma Ion Pump Controller
Existing. Adopted.
RS232C
12/21/2007
MKS937A Gauge Controller
Existing. Adopted.
RS232C
12/21/2007
Wiener VME Crate
Existing. Adopted.
Ethernet
12/21/2007
LakeShore 340
Done
GPIB
1/9/2008
Coherent EPM2000 power meter
Done
GPIB/RS232C
1/18/2008
Optosigma PAT-001 rotating attenuator
Done
RS232C
1/24/2008
Regular Power Supply (Xantrex)
Done
Ethernet
1/31/2008
Filter Wheel: CFW-1 filter wheel
To be developed
RS232C
MKS PDR2000 Press/Flow Monitor
Done
RS232C
3/31/2008
MKS PR4000 Press/Flow Controller
Done
RS232C
5/16/2008
MKS 146C pressure regulator
To be developed
RS232C
Varian V81, V301, V801, V1001 Turbo Pump Controller
To be developed
RS232C
EDT PMC CLINK with Fiber
Exist, need modification
PMC
PULNiX TM4200CL/TM6710CL
Not exist
CameraLink
AB PLC
Exist
Ether/IP
Lower Priority:
Digitizer for K-monochromator
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
AMO DAQ Instruments
Diagnostic Section
High Field Physics End-Station
electron time-of-flight spectrometers
magnetic bottle electron time-ofion spectrometers
flight spectrometer
time-of-flight, imaging, momentum
x-ray emission spectrometers
x-ray emission spectrometers
pulse energy monitor
beam profile screens
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Digitizer
Acqiris DC282 high-speed 10-bit cPCI Digitizer
4 channels
2-8 GS/s sampling rate
Acquisition memory from 1024 kpoints to 1024 Mpoints
Low dead time (350 ns) sequential recording with time
stamps
6U PXI/CompactPCI standard, 64 bit, 66 MHz PCI bus
Sustained transfer rate up to 400MB/s to host SBC
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Data System Architecture
AMO specific
Photon Control Data Systems (PCDS)
Beam Line
Data
L1: Acquisition
Digitizers + Cameras
Timing
L0: Control
L2: Processing
DAQ system primary features
Trigger and readout
Process and veto
Monitoring
Storage
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
L3: Data Cache
DAQ, Data Visualization and Analysis
Online
DAQ, processing, applications, 3-day storage storage,
real-time data visualization, analysis
Requirements in individual instrument ESD’s
Task of on-line group (A. Perrazzo, R. Herbst, R.
Machet, C. O’Grady, M. Weaver)
Offline
Long-term storage, data analysis, data distribution
Task of off-line group (S. Luitz, KT Lim)
Will mainly be funded from operating budget
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Network Security Enclave PCDS (Photon Control and
Data Systems)
Created PCDS Network Security Plan, approved by SLAC network security and DOE
http://confluence.slac.stanford.edu/display/PCDS/Networking
Can’t bring “ad-hoc” instrumentation/computers with data analysis software and hook up. Need to
plan ahead.
Completely fine for planned LCLS operation
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Femto-Second Laser Timing
John Byrd, et al, LBL
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Stabilized Fiber Link
Prototype test at SLAC with LBL system
Installed 1 Km fiber in tunnel from sector 20 laser room to sector 30/MCC
Also installed fiber in Klystron gallery from sector 20 laser room to MCC
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
First Timing Tests Jan 08
Some modification where implemented since January some of which happen to also
improve operation with gallery fibers (tbd)
More test planned in July, including the complete laser-timing control system
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
CXI 2D Detector
2D pixel array detector for CXI instrument
Collaboration with Sol Gruner Group at Cornell Univ.
Full-scale custom ASIC received, measurements in
progress at Cornell & SLAC
Integration with SLAC DAQ system done at SLAC
Mechanical packaging done at SLAC
Coordinated with CXI
Cornell and SLAC received bump-bonded chip last
month
Delivery of Final Detector at SLAC in May 2009
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
LCLS 2D-Detector Instrumentation and DAQ
Pixel-Detector & ASIC by Cornell
Note that additional pixel devices, e.g. BNL detector, are similar
SLAC Control & DAQ
Test Setup Chain, see below
Cornell
ASIC
Mother Board &
ASIC Daughter Board
Ribbon
Cable
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
LCLS
Front End
Development
Board
‹#›
Fiber
Optic
PGP
LCLS
RCE
Board
Fiber
Optic
1G Ethernet
Gunther Haller
haller@slac.stanford.edu
Linux
PC
2D-Detector Control and DAQ Chain
Groundisolation
Vacuum
Fiber
Cornell
detector/
ASIC
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
ATCA crate
with SLAC
DAQ Boards
SLAC FPGA
front-end
board
‹#›
Gunther Haller
haller@slac.stanford.edu
Cornell ASIC Boards
Each detector has ~36,000 pixels
Controlled and read out using custom ASIC
~36,000 front-end amplifier circuits and analogto-digital converters
Readout is 200Mbit/sec from each ASIC, 74
kbytes times 120 Hz.~ 9Mbytes/sec
Initially 16 x 32,000-pixel devices, then up to 64 x
32,000-pixel devices
4.6 Gbit/sec average with > 10 Gbit/sec peak
To be processed real-time
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
SLAC PPA Reconfigurable Cluster Element (RCE)
Boards
Reconfigurable Cluster Element
module with 2 each of following
Virtex-4 FPGA
2 PowerPC processors IP
cores
Note: Xilinx will use PPC in
next generation FPGA’s,
choice for IP-core
processors
512 Mbyte RLDRAM
8 Gbytes/sec cpu-data memory
interface
10-G Ethernet event data
interface
1-G Ethernet control interface
RTEMS operating system
up to 512 Gbyte of FLASH
memory
Rear
Transition
Module
1 TByte/board
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Reconfigurable
Cluster Element
Module
Gunther Haller
haller@slac.stanford.edu
Calibration & Distribution (using SLAC DAQ)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Gain (using SLAC DAQ)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Noise (using SLAC DAQ)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
2D-Detector Mechanical/Electrical Vacuum Assembly
Conceptual Design (SLAC PPA Engineering)
Positioning plate
•Supports quadrant raft
•Mounts to drive system
Cam follower mounted
to torque ring
Hole size remotely adjustable
Via PCDS Controls
One Quadrant
Raft Removed
Pixel Detectors
Cut-outs in base
plate for cold straps
and cables
Martin Nordby, Matt Swift (SLAC PPA Eng.)
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
Cold strap
‹#›
Gunther Haller
haller@slac.stanford.edu
Quadrant Board and Electrical Interfaces
Quadrant raft provides structural support and
stability for the double-detector packages
Feet mount on quadrant raft through holes in the
quadrant boards
This is also the thermal path
Cold strap
Quadrant boards provide grounding, power, and
signal interface to the PAD detector package
Quadrant raft
1 flex cable per detector
1 FPGA on each quadrant board
Mounting feet
Double-detector package
(4 per quadrant)
Detector
Quadrant board 2
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Quadrant board 1
Gunther Haller
haller@slac.stanford.edu
Light shroud
(cover removed)
Mechanical Design Overview
Cooling coil
Quadrant board
•Combines signals from 16 ASIC’s
•1 FPGA/quadrant
Double detector package
•4 ASIC’s
•2 pixel array detectors
Martin Nordby, Matt Swift PPA Mech. Eng.
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Front View—1 mm Aperture
Martin Nordby, Matt Swift PPA Mech. Eng.
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Front View—6 mm Aperture
Martin Nordby, Matt Swift PPA Mech. Eng.
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Front View—10 mm Aperture
Martin Nordby, Matt Swift PPA Mech. Eng.
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu
Summary
XTOD
Instrumentation to be delivered in Fall well under way, installed in racks, in testing
AMO Instrument Controls and Data Acquisition
Requirements and interface control documents released
Preliminary design complete
Test setups and software in progress
LUSI instruments and Diagnostics
Requirements and interface control documents in draft form
Preliminary design complete
Woll be released before CD-2 (August 08)
Infrastructure for NEH Server Room and Hutch 2 in progress, will be in place before September 08
Includes networking
Femto-second timing system tested at SLAC with 2 km (round-trip) of fiber in tunnel and in gallery
Already meets performance specification (< 100 fS)
More test planned for July 08
2D Pixel Detector control and data acquisition well advanced
Already controlling and acquiring data from pixel detector prototype and custom ASIC with SLAC LCSL DAQ prototype
system
No technical risks, well within expertise of Photon Control and Data Systems Group (Engineers from SLAC Particle and
Astro Physics Instrumentation and Control group)
On schedule, on budget
17 June 2008, FAC Meeting
Photon Systems Controls & DAQ
‹#›
Gunther Haller
haller@slac.stanford.edu