Photon Systems
Controls
and Data Systems
Gunther Haller
Particle Physics and Astrophysics Division
Photon Control and Data Systems (PCDS) Group
June 9, 2009
Photon Area Controls
FAC Review June 09
1
v2
1
Gunther Haller
haller@slac.stanford.edu
Outline
LCLS versus LUSI Responsibilities
PPS/HPS ( see previous presentation by Enzo)
XTOD X-Ray Transport Optics & Diagnostics
AMO Instrument Controls
LUSI Instruments
Diagnostics
Machine Protection System
Laser Safety System
EPICS Software
Femto-Second Laser Timing System
Data Acquisition (see Amedeo Perazzo presentation)
2-D Detector Control and Readout (XPP, CXI, XCSS)
Offline/Computing
Photon Area Controls
FAC Review June 09
2
2
Gunther Haller
haller@slac.stanford.edu
LCLS & LUSI
Common photon-area Controls and DAQ System Design for LCLS and LUSI
Includes Front-End Enclosure (FEE), Near Experimental Hall (NEH), X-Ray Tunnel (XRT) and Far
Experimental Hall (FEH)
Most of XTOD is located in FEE
LCLS Controls & DAQ W.B.S 1.6.2
AMO experiment (NEH, hutch 1)
All controls and DAQ
Common services for all hutches
Examples
PPS/HPS/LSS
120 Hz beam-quality data interface
Machine protection system interface
Network interface
Local science data online processing & cache
2-D detector
Control and DAQ for detector
LUSI Control & DAQ W.B.S
XPP, XCS, CXI Instruments, Diagnostics
Examples of documents released
1.1.523 XES-LUSI ICD (includes Controls and Data systems)
1.1-516 XES Photon Controls to Electron Beam Controls System ICD
1.1.517 XES Photon Controls to Electron Beam Controls MPS ICD
Photon Area Controls
FAC Review June 09
3
3
Gunther Haller
haller@slac.stanford.edu
XES Near & Far Hall Hutches and Beamline Layout (not to
scale)
230 m
AMO
SXR
MEC
Photon Area Controls
FAC Review June 09
4
4
Gunther Haller
haller@slac.stanford.edu
XR Diagnostics in Front End Enclosure
Gas
detector
Fast Valve+
Fixed Mask+
Slit
Diagnostic
Package
5 mm
diameter
collimators
Indirect Imager / K
measurement /
reticule
spectral
Gas
Attenuator
v
v
FEL Offset mirror system
u
Solid
Attenuator
Muon
Shield
Photon Area Controls
FAC Review June 09
Gas
detector
Total
Energy
Calorimeter
10-10 Torr
900 mm horizontal
Highenergy
beam
24 mm vertical
Lowenergy
beam
w
Direct
Imager
5
5
Gunther Haller
haller@slac.stanford.edu
XTOD in FEE
Phase 1 of XTOD in
commissioning
Controls installed
Being tested paced by availability
of beam-line items installed
Photon Area Controls
FAC Review June 09
6
6
Gunther Haller
haller@slac.stanford.edu
Photon Area Controls and Data Systems Devices
Created web-pages to list supported controls
and data-acquisition devices for photon area
For all experiments in xray area (AMO, XPP, XCS,
SXR, CXI, etc)
SXR might have up to 8 different end-stations
http://confluence.slac.stanford.edu/pages/viewpage.a
ction?pageId=9175609
Additional devices can be added after discussions
with Photon Controls and Data Systems (PCDS, G.
Haller) group
Photon Area Controls
FAC Review June 09
7
7
Gunther Haller
haller@slac.stanford.edu
AMO Instrument Control
Requirements listed in Engineering Specification Documents (ESD’s)
1.6-108 AMO Controls ESD, status: released
1.6-109 AMO DAQ ESD, status: released
Interfaces specified in Interface Control Document (ICD)
1.1-515 XES AMO Controls ICD, status: released
Held Final Design Review 11-10-08
http://confluence.slac.stanford.edu/display/PCDS/AMO-FDR
Purchased all hardware
Racks with infrastructure/AC installed in hutch 1
Controllers/DAQ being tested in lab before moving to NEH early next
month
See pictures later in presentation
Photon Area Controls
FAC Review June 09
8
8
Gunther Haller
haller@slac.stanford.edu
AMO and its DAQ Instruments
Diagnostic Section
High Field Physics End-Station
magnetic bottle electron time-offlight spectrometer
electron time-of-flight
spectrometers
ion spectrometers
x-ray emission spectrometers
time-of-flight, imaging, momentum
pulse energy monitor
x-ray emission spectrometers
Photon Area Controls
FAC Review June 09
beam profile screens
9
9
Gunther Haller
haller@slac.stanford.edu
Example: Vacuum
All gauge controllers are MKS 937A
Interface
Terminal server – DIGI TS16 MEI
Automation Direct PLC
All ion pump controllers are Gama Vacuum
DIGITEL MPC dual
All valves are controlled by PLC relay module
The out/not-out state of all valves go into the MPS
system to prevent damage if a valve closes
unexpectedly.
Photon Area Controls
FAC Review June 09
10 10
Gunther Haller
haller@slac.stanford.edu
Example: Motion
Control System provides support for all motions
Motors
IMS MDrive Plus2 integrated controller and motor
IMS MForce Plus2 controller for control of in vacuum and
other specialized motors
Newport motor controllers
Others as required
Pneumatic motion
Solenoid Driver chassis, SLAC 385-001
Photon Area Controls
FAC Review June 09
11
11
Gunther Haller
haller@slac.stanford.edu
AMO Test Setup in Lab
Racks in lab, AMO controllers
being installed
Photon Area Controls
FAC Review June 09
Pulse picker and also
DAQ test setup
12 12
Gunther Haller
haller@slac.stanford.edu
AMO in Hutch 1
Racks installed, AC connected
Will move loaded crates from
lab to NEH racks in July
Photon Area Controls
FAC Review June 09
13 13
Gunther Haller
haller@slac.stanford.edu
Software: AMO Example: Moving Tables
Photon Area Controls
FAC Review June 09
14 14
Gunther Haller
haller@slac.stanford.edu
AMO Test Setup in Lab
AMO table motion control mock-up for motion software
algorithm
Smart
Motors
Photon Area Controls
FAC Review June 09
15 15
Gunther Haller
haller@slac.stanford.edu
LUSI & SXR
LUSI includes the instruments
XPP (NEH, hutch 3)
XCS (FEH, hutch 4)
CXI (FEH, hutch 5)
Have written and released ESD’s and ICD’s for each instrument
Held XPP Final Design Review, CXI & XCS Preliminary Design
Review
See under AMO and CXI and XCS at
http://confluence.slac.stanford.edu/display/PCDS/PCDS+Home
SXR
Beamline was added to the scope
Have drafts of ESD and ICD’s
Held discussions with several end-stations on how to plan to
make them compatible with LCLS
Photon Area Controls
FAC Review June 09
16 16
Gunther Haller
haller@slac.stanford.edu
Common Diagnostics Readout
E.g. intensity, profile monitor,
intensity position monitors
E.g. Canberra PIPS or IRD
SXUV large area diodes (single
or quad)
Amplifier/shaper/ADC for
control/calibration/readout
Quad-Detector
R2
q1
q2
R1
Target
L
• Fourdiode
design
• On-board calibration circuits not shown
• Board designed, fabricated, loaded, is in test
Photon Area Controls
FAC Review June 09
17 17
Gunther Haller
haller@slac.stanford.edu
MPS Rack Layout (Reminder)
Dedicated Optical fiber
1 single-mode pair per Link Node for
custom MPS signaling protocol
Terminal Server
Cat-6 Ethernet
MPS
Link Node
All optical networking paths route
through experiment hall patch panels
and terminate at fiber patch panel in
MCC
Cat-6 Ethernet
Dedicated Single-Mode Optical Fiber Pair
Access to EPICS process variables on
MPS Link Nodes via local Ethernet
switch
MPS Link Node console logins via
Terminal Server
Remote power-cycling of Link Nodes
via APC AP7900 Switched Rack PDU
Dedicated Single Mode Optical Fiber Pair
Ethernet Switch
Redundant single-mode pairs for
1000Base-LX Ethernet
1000Base-LX
(Hot Spare)
TCP/IP Networking
1000Base-LX
(To Main Control Center)
RS-232
MPS
Link Node
RS-232
Cat-6 Ethernet
Network-Controlled Power Distribution Unit
Photon Area Controls
FAC Review June 09
18 18
Gunther Haller
haller@slac.stanford.edu
Installation Status
Near Experiment Hall Machine Protection
Rack, (B950S-04), Installed.
48-strand fiber trunk pulled between B950B-42,
(NEH Server Room), and B950S-04. The NEH
MPS optical networking path is complete.
Ethernet Switch Installed in rack B950S-04,
connected to fiber, and checked.
Photon Area Controls
FAC Review June 09
19 19
Gunther Haller
haller@slac.stanford.edu
MPS Installation Status
All DIN rails for rack B950S-04
assembled, tested, and installed
12 MPS Link Node Input Aggregation
Rails
6 MPS Trunk Head-End Rails
MPS Trunk Field Rails assembled
MPS trunk cables pulled between
the hutches, the FEE, and rack
B950S-04.
Photon Area Controls
FAC Review June 09
20 20
Gunther Haller
haller@slac.stanford.edu
PPS/MPS Racks
Installed in NEH
Photon Area Controls
FAC Review June 09
21 21
Gunther Haller
haller@slac.stanford.edu
NEH Laser Bay Laser Safety System Layout and
Components
Electronic
Strike
Plate
Door Interlock
Switches
LSS
Controller
Rack
Access
Control
Panel
LED Hazard
Displays
LSS Control
Panel
Timed
Door
Interlock
Bypass
Switches
EmergencyOff Buttons
LED Hazard
Display
Photon Area Controls
FAC Review June 09
22 22
Gunther Haller
haller@slac.stanford.edu
Laser Safety System
Laser
Room
Entrance
Start
installation
on first
table
Photon Area Controls
FAC Review June 09
23 23
Laser Safety
System control
rack
Gunther Haller
haller@slac.stanford.edu
EPICS/Python/Qt
EPICS (Experimental Physics and Industrial Control
System):
Control software for RT systems
Monitor (pull scheme)
Alarm
Archive
Widely used at SLAC and other labs
More: http://www.aps.anl.gov/epics/
Python/Qt is a user interface between the EPICS drivers
and records and the user
System is currently used for XTOD and AMO, provided
as part of the XES Photon Controls Infrastructure
Later used for SXR, CXI, XPP, XCS, MEC
Photon Area Controls
FAC Review June 09
24 24
Gunther Haller
haller@slac.stanford.edu
Consoles: PyQt versus EDM
Photon Area Controls
FAC Review June 09
25 25
Gunther Haller
haller@slac.stanford.edu
Status
AMO EPICS drivers almost complete
More work as scheduled over the next two
months
Plus need high level control panels
Photon Area Controls
FAC Review June 09
26 26
Gunther Haller
haller@slac.stanford.edu
Femto-Second Laser Timing System
RF Cavity System provided by SLAC
Laser Fiber driver/receiver system provided by
LBL
Tested performance of LBL system prototype:
better than 20 fsec drift performance (100 fsec is
spec)
2-Receiver, 1-Driver system will be installed at
SLAC in July 09.
Photon Area Controls
FAC Review June 09
27 27
Gunther Haller
haller@slac.stanford.edu
Designed/Built
at LBL
Photon Area Controls
FAC Review June 09
28 28
Gunther Haller
haller@slac.stanford.edu
RF Cavity Electronics for Femto-Second Timing
System in Lab
Control/DAQ
crate with IOC
and SLAC 119
MHZ waveform
digitizer
RF Cavity
SLAC
Chassis
Photon Area Controls
FAC Review June 09
29 29
Gunther Haller
haller@slac.stanford.edu
Example: LUSI Instrument CXI
Diagnostics/Common Optics
Diagnostics &
Wavefront Monitor
1 micron Sample Environment
1 micron KB
Reference Laser
* 0.1 micron KB & Sample Environment,
Particle Injector and IToF (CD-4)
All Early Science except *
Photon Area Controls
FAC Review June 09
30 30
Gunther Haller
haller@slac.stanford.edu
Controls Subsystems
Vacuum
Motion
Viewing
Power Supplies
Racks and Cabling
Other items
Software: EPICS/Python/Qt
Type of controls
Valve Control
Vacuum Controls
Pop-In Profile Monitor Controls
Pop-In Intensity Monitor Controls
Intensity-Position Monitor Controls
Slit Controls
Attenuator Controls
Pulse Picker Controls
KB Mirror Controls
X-Ray Focusing Lense Control
Sample Environment Controls
Particle Injector Controls
Ion ToF Controls
Vision Camera Controls
Detector Stage Controls
Reference Laser Controls
DAQ Controls
Photon Area Controls
FAC Review June 09
31 31
Gunther Haller
haller@slac.stanford.edu
CXI Components to Control
X-Ray Optics KB system
Motion
Vendor provided, integration with LCLS
Reference Laser
Motion
Sample Environment
Sample Chamber
Motion, vacuum, vision
Ion ToF
HV, DC/pulser, digitizer
Instrument Stand
Motion
Detector Stage
Motion, vacuum, thermal
Particle Injector
Motion, vacuum, digitizer, vision, integration of commercial component
Vacuum System
Valve and Vacuum Controls
Photon Area Controls
FAC Review June 09
32 32
Gunther Haller
haller@slac.stanford.edu
CXI Components to Control con’t
Diagnostics and Common Optics
Pop-In Profile Monitor
Motion, Viewing
Pop-In Intensity
Motion, Digitization
Intensity Position
Motion, Digitization
Slit System
Motion
Attenuator
Motion
Pulse-Picker
Motion, Viewing
X-Ray Focusing Lense
Motion
CXI specific interface and programming
Racks & Cabling
Workstations
Vision Cameras
Beam Line Processor
Channel Access Gateway
Machine Protection System
Configuration
Data Acquisition
Use standard controllers on photon area controller list as much as possible
Photon Area Controls
FAC Review June 09
33 33
Gunther Haller
haller@slac.stanford.edu
LUSI Data Acquisition for Pixel Detectors
Cornell and Brookhaven 2-D pixel
detectors are configured & read out using
the SLAC ATCA Reconfigurable Cluster
Element modules
Details in following slides
XPP XAMP Detector with custom ASIC
CXI Detector with custom ASIC
Photon Area Controls
FAC Review June 09
34 34
Gunther Haller
haller@slac.stanford.edu
Data System Architecture
XPP specific
Photon Control Data Systems (PCDS)
Beam Line
Data
L1: Acquisition
(Many)
Digitizers + Cameras
Timing
L0: Control
(One)
L2: Processing
(Many)
L3: Data Cache
(Many)
DAQ system primary features
Trigger and readout
Process and veto
Monitoring
Storage
Photon Area Controls
FAC Review June 09
35 35
Gunther Haller
haller@slac.stanford.edu
XAMP 2D-Detector Control and DAQ Chain
Beamline Instrument Detectors
Fiber
Brookhaven XPP/XCS 2D
detector-ASIC
SLAC FPGA
front-end
board
XAMP (XPP) LUSI instrument custom
integrated circuits from Brookhaven are
already connected at SLAC to SLAC LCLS
high-performance DAQ system
XPP BNL XAMP Detector 1,024 x
1,024 array
Uses 16 each 64-channel
FexAmps BNL custom ASICs
Instantaneous readout: 4 ch x
20 MHz x 16bit= 20 Gbit/sec
into FPGA
Output FPGA: 250 Mbytes/s at
120 Hz (1024x1024x2x120)
In addition BNL has ATCA crate with SLAC
modules to develop software and test with
detector
Photon Area Controls
FAC Review June 09
ATCA crate with SLAC
DAQ boards, e.g. the
SLAC Reconfigurable
Cluster Element Module
ATCA
Advanced Telecommunication Computing Architecture
Based on backplane serial communication
fabric, 10-G E
2 SLAC custom boards (also used in other SLAC
experiments)
8 x 2.5 Gbit/sec links to detector modules
Dataflow and processing
Managed 24-port 10-G Ethernet switching
Essentially 480 Gbit/sec switch capacity
Naturally scalable
36 36
Gunther Haller
haller@slac.stanford.edu
Example: XPP Online Processing
Electronics gain correction (in RCE)
Response of amplifying electronics is mapped during calibration
Science data images are corrected for channel gain non-uniformity + non-linearity.
Dark image correction (in RCE)
Dark images accumulated between x-ray pulses
Averaged dark image subtracted from each science data image
Flat field correction (in RCE)
Each science data image is corrected for non-uniform pixel response
Event filtering (in RCE or later)
Events are associated with beam line data (BLD) via timestamp and vetoed based
upon BLD values. Veto action is recorded.
Images may be sparsified by predefined regions of interest.
Event binning (processing stage)
Images (and normalization) belonging to the same bin (dt, Eg, ..) are summed
together
Photon Area Controls
FAC Review June 09
37 37
Gunther Haller
haller@slac.stanford.edu
Example: XPP Monitoring
A copy of the data is distributed (multicast) to monitoring
nodes on the DAQ subnet.
The monitoring nodes will provide displays for
experimenters’ viewing:
Corrected XAMPS images at ≥ 5 Hz
Histories of veto rates, beam intensity, + other BLD values.
Reduced analysis of sampled binned data (versus scan
parameter)
Implemented with Qt (C++/Python open source GUI)
Photon Area Controls
FAC Review June 09
38 38
Gunther Haller
haller@slac.stanford.edu
Example: XPP XAMPS Data Rates
XPP specific
Photon Control Data Systems (PCDS)
Beam Line
Data
L1: Acquisition
(Many)
Digitizers + Cameras
Timing
XAMPS
L2: Processing
(Many)
L0: Control
(One)
4 x 2.5Gb PGP
L1:
10 GbE
L2:
240 MB/s
(480 MB/s)
RCE
< 200 MB/s
Processing
L3: Data Cache
(Many)
L3:
10 GbE
Cache
n x (200 MB – 20 GB)
Binned data archived
Expect
at end of run
~ 6 – 60 GB / day
(mins – hrs)
Photon Area Controls
FAC Review June 09
39 39
Gunther Haller
haller@slac.stanford.edu
CXI
2D-Detector Mechanical/Electrical Vacuum Assembly
SLAC PPA Engineering
Electrical/Mechanical Package
Positioning plate
•Supports quadrant raft
•Mounts to drive system
Cam follower mounted
to torque ring
Hole size remotely adjustable
Via PCDS Controls
One Quadrant
Raft Removed
Pixel Detectors
Cut-outs in base
plate for cold straps
and cables
Cold strap
Photon Area Controls
FAC Review June 09
40 40
Gunther Haller
haller@slac.stanford.edu
Quadrant Board and Electrical Interfaces
Quadrant raft provides structural
support and stability for the doubledetector packages
Feet mount on quadrant raft
through holes in the quadrant
boards
This is also the thermal path
Quadrant boards provide
grounding, power, and signal
interface to the PAD detector
package
1 flex cable per detector
1 FPGA on each quadrant
Mounting feet
board
Cold strap
Quadrant raft
Double-detector package
(4 per quadrant)
Detector
Quadrant board 2
Quadrant board 1
Photon Area Controls
FAC Review June 09
41 41
Gunther Haller
haller@slac.stanford.edu
ASIC Board
Rigid-flex ASIC board (SLAC design)
ASIC Bump-bonded to detector
ASIC/detector package bonded to carrier board
Photon Area Controls
FAC Review June 09
42 42
Gunther Haller
haller@slac.stanford.edu
CXI 2D-Detector Control and DAQ Chain
Vacuum
Groundisolation
Fiber
Carrier
Board
Cornell detector/ASIC with
SLAC quadrant board
ATCA crate with SLAC DAQ Boards
S:AC RCE ATCA Module
Each Cornell detector has ~36,000 pixels
Controlled and read out using Cornell custom ASIC
~36,000 front-end amplifier circuits and analog-to-digital
converters
Initially 16 x 32,000-pixel devices, then up to 64 x 32,000-pixel
devices
4.6 Gbit/sec average with > 10 Gbit/sec peak
Photon Area Controls
FAC Review June 09
43 43
Gunther Haller
haller@slac.stanford.edu
CXI Online Processing
Electronics gain correction (in RCE)
Response of amplifying electronics is mapped during calibration
Science data images are corrected for channel gain non-uniformity + nonlinearity.
Dark image correction (in RCE)
Dark images accumulated between x-ray pulses
Averaged dark image subtracted from each science data image
Flat field correction (in RCE)
Each science data image is corrected for non-uniform pixel response
Event filtering (in RCE or later)
Events are associated with beam line data (BLD) via timestamp and vetoed
based upon BLD values. Veto action is recorded.
Images may be sparsified by predefined regions of interest.
Photon Area Controls
FAC Review June 09
44 44
Gunther Haller
haller@slac.stanford.edu
CXI Online Processing con’t
Event processing (processing stage)
Examples are
Sparcification (region of interest)
Locating center
Reducing data by binning pixels
Mask errant pixels (saturated, negative intensity from dark image subtraction due to e.g.
noise, non-functioning pixels, edge pixels from moving center)
Filling in missing data with centro-symmetric equivalent points
Transforming camera geometry due solid angle coverage and dead space between tiles
Radial averaging, showing intensity versus scattering angle or momentum transfer
Compute 2D autocorrelation function (single FFT) and store. Essentially at rate of 1 Hz with
4 MB (2Mpixel x 2 bytes) frames.
Peak finding (locate and fit Gaussian intensity peaks). There may be multiple peaks in some
cases and the peak finding algorithms should be able to identify up to a few thousand peaks.
The CXI instrument will have an Ion Time-of-Flight which will produce data at 120Hz. The
online processing of this data involves data reduction based on thresholding and vetoing
based on thresholding or the fitting of peak positions and height.
Photon Area Controls
FAC Review June 09
45 45
Gunther Haller
haller@slac.stanford.edu
CXI Monitoring
A copy of the data is distributed (multicast) to monitoring
nodes on the DAQ subnet.
The monitoring nodes will provide displays for
experimenters’ viewing:
corrected detector images at ≥ 5 Hz
histories of veto rates, beam intensity, + other BLD values.
Reduced analysis of sampled binned data (versus scan
parameter) or other processing tbd
Implemented with Qt (C++/Python open source GUI)
Photon Area Controls
FAC Review June 09
46 46
Gunther Haller
haller@slac.stanford.edu
Offline
Offline Analysis Examples
XCS Multi-tau multi-Q processing
CXI Classification
Averaging
Alignment
Phase Retrieval
To be addressed
CXI
Data Sorting/Averaging (Classification)
Phase Retrieval
Photon Area Controls
FAC Review June 09
47 47
Gunther Haller
haller@slac.stanford.edu
Offline Data Management System Functions
Translate data into a form for long-term scientific
access
Translate to HDF5 format
From online, XTC, Extended tag Container format, C++
Store data attributes in a database (science metadata)
Store translated data and archive to tape
Provide access for scientists to stored and archived
data
Access by attributes (science metadata)
Manage data ownership and access restrictions
Data access for:
Processing clusters
User workstations
Offsite export (network and other means)
Manage derived data products
Photon Area Controls
FAC Review June 09
48 48
Gunther Haller
haller@slac.stanford.edu
Offline System Architecture
Steffen Luitz
Photon Area Controls
FAC Review June 09
49 49
Gunther Haller
haller@slac.stanford.edu
Components
Data Format
HDF5/NeXUS chosen because of support, wide availability of language bindings
File Manager
iRODS (next-generation of SDSC SRB)
Tape Manager
Interface iRODS to SLAC HPSS and mass storage system (SL8500 )
High performance file systems
Lustre
Databases
MySQL
Monitoring
Ganglia
Data transfer
Currently performing data transfer studies with CAMP group in Germany (Peter Holl)
to write and retrieve few hundred Gigabyte files using a few data transfer options
Sftp (expect 2 Mbytes/s)
Bbcp (expect 20 Mbyte/sec using around 15 streams)
FTD (around 20 Mbytes/sec using 64 streams)
User accounts, etc via SCCS, unix
In 2010 will move to server-based solution
XROOTD
iRods
http
Photon Area Controls
FAC Review June 09
Steffen Luitz
50 50
Gunther Haller
haller@slac.stanford.edu
Some Analysis Options
Can use NeXus or HDF5 Tools and Analysis Packages
Open Genie
LAMP
GumTree
Nathan
Redas
Scilab
Amortool
More …
IDL-HDF5
Matlab
Mathematica
O-Matrix
ViTables
More …
Photon Area Controls
FAC Review June 09
Open Source
NeXus API
Commercial
HDF5
File
API
Cactus
Chombo
dxhsf5
H5PartRoot
HL-HDF
ParaView
PyTables
VisAD
Open Source
Many more
…
Steffen Luitz
51 51
Gunther Haller
haller@slac.stanford.edu
Comments regarding Users
Users can
configure LCLS equipment/experiments from terminals (e.g. control room or hutch,
including photon area loaner laptop), plus remotely via tunneling
Several Phyton/Qt screens provided
User can add to it (support provided by PCDS (Photon Controls and Data Systems) group)
Or PCDS can provide requested screens
generate and apply calibration correction constants
monitor online performance on screens (plus remotely via tunneling)
Images, before/after correction, other parameters depending on experiment (e.g. centroid
movement)
including dataflow monitoring (time-stamp, event increments, missing images, etc)
Quality of data
Again users can add screens or PCDS group can provide custom screens
have access to data for post processing
File manager provided
have ability to transfer data to media or offsite
Technical support provided by group building the system
Includes Python/Qt
Integration of user data processing algorithms
Photon Area Controls
FAC Review June 09
52 52
Gunther Haller
haller@slac.stanford.edu
Comments regarding non-LCLS End Stations
XPP, CXI, XCS detectors are integrated into LCLS ATCA RCE data-acquisition
system
Configuration, readout of detectors/ASICs
Event building of images with
120-Hz beam-line data
Time-stamp
Allows real-time online vetoing, sorting, or other processing which depends on e.g.
energy, time-difference between laser and xray (for XPP)
Processing using LCLS NEH farm
Storage using LCLS storage
Retrieval
pnCCD (MPI) detector with Juelich DAQ, camera, and Acqiris waveform
digitizer for e.g. CAMP
Have been in contact for several months with MPI and DESY to work towards being
able to integrate. Recommended changes to their system to be able to integrate.
Started same with potential LBL FastCCD currently using Argonne DAQ for
SXR
Above need to be integrated into LCLS system otherwise
Results would be that end-stations would be run in “stand-alone”, potentially with
No time stamps
No 120-Hz BLD
No processing
No storage/retrieval
Not acceptable for end-stations or LCLS, especially for high data rate systems
Photon Area Controls
FAC Review June 09
53 53
Gunther Haller
haller@slac.stanford.edu
Example CAMP
Offline: latency of e.g min to hours (HDF5 format)
Depends on size of runs plus availability of offline system (not required to be 100% available)
Data from L3 storage cache (XTC format): latency 10 – 30 sec
Data processed online, C++: seconds latency
CAMP collaborators write code interfacing to DAQ online XTC data format to monitor
performance and to allow human time-scale feedback control
Runs on “Observer” L2 processing ATCA blades
Photon Area Controls
FAC Review June 09
54 54
Gunther Haller
haller@slac.stanford.edu
NEH Server Room
Houses mainly
Machine Timing Distribution
Network switches
EPICS servers
L2 processor farm
L3 local data-storage
Terminals
Phase 2
racks still
to be
installed
Existing racks with data
cache and servers
Photon Area Controls
FAC Review June 09
55 55
Gunther Haller
haller@slac.stanford.edu
Summary
All long-haul cables for FEE and NEH are installed, so are racks
XTOD
Instrumentation for phase 1 is installed, testing in progress
PPS, HPS, MPS, and LSS are all well on their way, most already installed
AMO Instrument Controls and Data Acquisition
Installation in progress, will continue over next 3 months
10 weeks of science starting October 09
LUSI Instrument Controls (XPP, CXI, XCS)
Final Design Review held for XPP
Preliminary Design Reviews held for CXI and XCS
SXR Beam-Line control components being ordered
In discussions with SXR Endstation leaders to ensure compatibility
Femto-Second Timing System well under way
RF cavity electronics to be installed in Undulator hall in July
LBl Laser Fiber system to be installed in July
Photon Area Controls
FAC Review June 09
56 56
Gunther Haller
haller@slac.stanford.edu
Summary con’t
2-D Detector
Prototype signal chains in operation from detector to DAQ
system
DAQ well under way (see Amedeo’s talk)
Offline infrastructure in design
Scientific Computing for XPP, XCS, AMO in progress
Scientific Computing for CXI still needs to be defined
Algorithms are being developed.
Resulting computing needs to be determined
Photon Area Controls
FAC Review June 09
57 57
Gunther Haller
haller@slac.stanford.edu