ADAQA
An Accelerator Data Acquisition & Analysis Framework
Bolko Beutner, Helge Brands, Rasmus Ischebeck
> The Past
> Accelerator Data Acquisition
> Design Ideas
> The Present
> Available Tools
> Demo
> The Future
> Data Acquisition with Large Data Sets at 100 Hz
© 2013 Paul Scherrer
Institut
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
Slides available at http://people.web.psi.ch/ischebeck
1
What Came Before…
(my personal collection of accelerator data acquisition)
>TESLA Test Facility at DESY, Hamburg
> Access to accelerator process variables from MATLAB, C++ (read/write)
> Easy to write data acquisition software, scan parameters, record variables…
> Data stored in .MAT, ASCII & ROOT files
> No common data format
> Experts required to set up scan, analyze data…
>SLAC LINAC at Stanford, California
> VAX-based control system
> Scan software to determine the dependency of a variable on a parameter
> On-line display & fits
> Writes MATLAB 4 files for further analysis
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
2
Design Goals
> Have a universal scan and data acquisition software
> Well-tested scan core
> Universal GUI to set up scan easily
> Specialized GUIs for easy access for regularly performed scans
> Error checking and treatment
> Common HDF5-based file format for easy access from any data analysis code
> Automatic generation of logbook entries
> Link to auxiliary information on accelerator configuration
> Synchronized data acquisition
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
3
Data File Format
> Hierarchical format to store Instruments
> Based on HDF5
> Readable from Matlab, Root, Mathematica…
Rasmus Ischebeck
www.nexusformat.org
4
Accelerator Data Acquisition
Many accelerator measurements
consist of the following tasks:
>Control 0…N actuators
>Acquire 1…M measurements
>Analyze data
Important features
>On-line analysis
>Reasonable error handling
>Robust code for use by other
persons than the programmer
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
5
ADAQA Layers
> ADAQA consists of several separate
building blocks
> Applications, written by different
programmers, make use of the same
data acquisition code
> Instruments gather connected
devices
> Logbook
> Common data storage format
> Online model*
> Device information server*
> Synchronized data acquisition
* ongoing work, not yet ready for prime
time
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
6
ADAQA Layers
Applications
> Tree-like structure:
> Many devices (roots)
> One data acquisition core (trunk)
> Many applications (branches)
> Building blocks can be exchanged
Data
Acquisition
Devices
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
Wikipedia 7
Synchronized Data Acquisition
> All VME crates for diagnostics are equipped with
an Event Receiver
> Connected with a star topology network to the
Event Generator
> Supported for:
> BPMs, Bunching monitor, RF, …
> Ongoing work:
> Laser shutter, oscilloscopes, cameras
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
Sven Nordquist 8
ADAQA Applications
• ScreenMonitorTool
• General Purpose Scan Tools
• Specialised Scan Tools
–Slice Emittance
–Optics Matching
–Slit Scan
–…
• “Do it yourself! - Scan Tools
SwissFEL Meeting 24.1.2013
+ SwissFEL
ScreenMonitorTool
• General purpose camera display and
analysis tool
• In every day use for different kinds of
cameras and screen monitors
Dione and Saturn
http://www.ciclops.org
SwissFEL Meeting 24.1.2013
+ SwissFEL
General Purpose Scan Tool
SwissFEL Meeting 24.1.2013
+ SwissFEL
the future…
SwissFEL Meeting 24.1.2013
+ SwissFEL
Scan Templates
• Scans can be stored in Template files
• The last scan is always stored as template
–“Repeat Last Scan…”-Button
• Data files are scan templates as well
–All scans can be exactly repeated
• Templates can handed over to the scan tool at startup
–example: extracted charge vs. gun phase scan
SwissFEL Meeting 24.1.2013
+ SwissFEL
Specialised Scan Tools
SwissFEL Meeting 24.1.2013
+ SwissFEL
Specialised Scan Tools
SwissFEL Meeting 24.1.2013
+ SwissFEL
Automatised logbook entries
SwissFEL Meeting 24.1.2013
+ SwissFEL
Scan Tools Summary
• General and specialised scan tools are possible with ADAQA
–a wide variety of tools is available
• Common data format
• Templates
–data files are templates – for all tools
• Actions
–e.g. automatic screen insertion or background subtraction
• Analysis functions
• Automatic logbook entries
SwissFEL Meeting 24.1.2013
+ SwissFEL
Do it yourself! - Scan Tools
knob{1} = LoadKnobDefinition('ManagementKnob');
knob{1}.SetpointValues = [1, 2.4, 3, 3.78, 92];
% setup Knob and Instrument
% scan setpoints
instrument{1} = LoadInstrumentDefinition('SwissFEL_Meeting_Demo');
Setup.Function = 'scan';
Setup.Repeat = 10;
% define scan setup
% “shots” per scan step
Info.Author = 'Beutner';
% human readable info
Info.Title = 'SwissFEL Meeting Demo';
Info.Purpose = 'this is a small scan tool for demo...';
[DAQFileName, Setup, knob, instrument, Info] = ...
daq(Setup,knob,instrument,Info);
% …insert your analysis code here…
SwissFEL Meeting 24.1.2013
+ SwissFEL
% execute scan
Image based measurements: today
control room
IOC
EPICS Control+Data
camera
workstation
AFS
Helge Brands
Image based measurements: today
control room
IOC
EPICS Control+Data
camera
Problems:
• available buffers on IOC are not used
• „lost“ of measured data
• high frequencies can‘t be reached
• high CPU load on workstations
workstation
AFS
Helge Brands
Image based measurements : future
control room
IOC
EPICS Control+Preview
camera
workstation
DATA
Results+Data
Compute-Node
AFS
Helge Brands
Why?
•less radiation
•higher recording speed (10Hz/100Hz)
•independent of the workstation (CPU and memory)
•system is similar to SLS (Tomcat, in operation, 1* PCO Edge)
Helge Brands
Why?
•less radiation
•higher recording speed (10Hz/100Hz)
•independent of the workstation (CPU and memory)
•system is similar to SLS (Tomcat, in operation, 1* PCO Edge)
BUT
Helge Brands
Technical values
more storage needed
example: ~30TB/week
Helge Brands
Specs
storage
network
IOC
size/speed (camera IOC) local memory
compute
nodes
WHLA
Test-Injector
558
GB/50MB/s
1Gb/s
16GB
~3000 pic
1-2
SwissFEL
(?)/(?)
10Gb/s
(?)
min. 5-10
Helge Brands
Current situation
•WHLA: each Camera-IOC has 1GB/s (enough for 10Hz)
•3 PCO Edge in operation + 1 PCO Edge in preperation
•upload to AFS max 50MB/s
•software is ready for 100Hz
•needed network speed 10GBit/s for 100 Hz
•other GIGE cameras can be used too
•additional compute nodes should be planned
•storage policies are needed
Helge Brands
ADAQA
> Project started in 2008, always between 5 and 10 developers
> Written mostly in Matlab
> 89093 lines of code (including code sourced from MathWorks Central and other external sources)
> Performed 22775 scans, resulting in 558 GB of data
> A joint project between the Controls, Beam Dynamics & Diagnostics Sections at PSI
Diagnostics
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
Controls
Beam Dynamics
31
Outlook
> We are looking forward to many new applications
for SwissFEL!
> Thank You to:
> all ADAQA programmers and users, in particular
Eduard Prat, Marta Divall, Peter Peier, Marc Guetg,
Gian Luca Orlandi, Bennie Smit & Thomas
Schietinger
> support from Jan Chrin & Babak Kalantari
© 2013 Paul Scherrer
Institut
Bolko Beutner, Helge Brands, Rasmus Ischebeck: ADAQA
Slides available at http://people.web.psi.ch/ischebeck
Sven Nordquist
32