Science Studio and ANISE
(Jan 2010 update)
Elder Matias
Canadian Light Source
www.lightsource.ca
•
•
•
•
•
•
•
Overview
Beamline Control Traditional Technology
Beamline Control – ScienceStudio
Nan-Fabrication – ScienceStudio
User Office
Grid Computing
User Configurable Light Paths
www.lightsource.ca
Where is Saskatoon?
www.lightsource.ca
Scientific American May 2008
Science 2.0 – The Risk and
Reward of Web-Based Research
---------------------------------
“Our real mission isn’t to publish
journals but to facilitate scientific
communication” Timo Hannay –
Head of Web Publishing at Nature
Publishing Group
www.lightsource.ca
What Is the Web 2.0?
•
In plain English ….
– Automating tedious tasks using web technology
– Tools to help people and software collaborate
www.lightsource.ca
Project Team Members
Dionisio Medrano
Jinhui Qin
Dylan Maxwell
Yuzhen Xie
Dong Liu
Mike Bauer
Elder Matias
Stewart McIntyre
Marina Suominen Fuller
Chris Armstrong
Khalid Subratee
Ludeng (Eric) Zhao
Yuhong Yan
www.lightsource.ca
Goals of the Two Projects
• Science Studio
– User Office Automation
– Integrated Experiment Management
– Remote Access to VESPERS (XRD and XRF)
– Access Scanning Electron Microscope (UWO)
– Grid Computing (Cell Processor)
• ANISE
– Integrated Stream Computing to providing
near real-time data processing
www.lightsource.ca
Beamline Control
Traditional Technology
www.lightsource.ca
EPICS
CA
CA
CA
Touch
Panels
CA
State Machine
Engine
CA
IOC
Channel Access Protocol
Operator Workstation
User Applications
Detectors
Cameras
etc.
IOC
Modbus
TCP/IP
CA
IOC
CA
Telemecanique
Momentum PLC
GPIB
IOC
CA
RS-232
VME
CA
Single Board
Computer
www.lightsource.ca
IOC
Beamline Controls Interfaces
• EPICS Tools
• Configuration Tool
• User GUI & ROOT
www.lightsource.ca
Beamline Visualization
- Raster scan with fluorescence spectroscopy of a pine needle contaminated with arsenic. The technique
allows multiple elements to be detected simultaneously at each point of the raster. These images show
distributions of arsenic, iron, and manganese, respectively, near the tip. The highest intensity displays in
red, lowest in blue.
- Custom on-line plotting application scripted in CERN Root, with data stream from the CLS data acquisition
application.
(G. Wright, R. Igarashi, K. Chang-Yong, N.
Chen)
As Map
Fe Map
www.lightsource.ca
Traditional Beamline Visualization
Cross section views of the beam spot intensity distribution for varying degrees of
monochromator detuning (50-80%). (Only one image shown).
(G. Wright, R. Igarashi, K. Chang-Yong, N.
Chen)
www.lightsource.ca
Beamline Control
ScienceStudio Technology
www.lightsource.ca
Requirements
• New User Office Functionality
–
–
–
–
–
Proposal submission
Peer review
User Feedback Tracking
Experiment Management
User Training/ Safety Testing
• Remote Beamline Access
• Integration with grid data-storage
• Grid computing
www.lightsource.ca
Science Studio and ANISE
Science Studio
Laboratory
Control
Module
Client Services
Layer
Browser
Browser
Beamline
Business Model
Layer
IOCs
Service
Proxies
Persistence
Layer
Device
Proxies
Science Studio specific
Component
XRF Processing
Labatory
Control
Module
XRD Processing
ANISE
Data
Data
Service
Service
Beamline
Processing
Processing
Service
Service
General, common
Component
www.lightsource.ca
IOCs
System Architecture
HTTP
Web
Application
DB
1.
2.
3.
4.
5.
6.
7.
JMS
Beamline
Control
Module
VESPERS
CA
SAN
VESPERS Beamline
EPICS control system
Beamline Control Module (BCM)
Web Application
Database
File Storage
Web Interface
www.lightsource.ca
VESPERS Beamline
HTTP
Web
Application
DB
JMS
Beamline
Control
Module
VESPERS
CA
SAN
•
VESPERS — Very Sensitive Elemental and Structural Probe Employing
Radiation from a Synchrotron
•
A bending magnet beamline on sector 6 at the Canadian Light Source
synchrotron in Saskatoon, Saskatchewan.
•
A hard x-ray microprobe with an energy range of 6 to 30keV.
•
Techniques: X-Ray Fluorescence (XRF) & X-Ray Diffraction (XRD)
www.lightsource.ca
EPICS Low-level Control System
HTTP
Web
Application
JMS
DB
Beamline
Control
Module
VESPERS
CA
SAN
•
EPICS — Experimental Physics and Industrial Control System
•
The standard control system at the CLS.
•
EPICS consists of a network of Input-Output Controls (IOCs) which are connected to directly to
devices.
•
An IOC provides many Process Variables (PVs) which relate to either an input or output from a
device and have a unique name.
•
Channel Access (CA) is used to read or write to any PV without knowing which IOC provides the
PV.
•
More than 50,000 PVs in the CLS control system.
www.lightsource.ca
Beamline Control Module (BCM)
HTTP
Web
Application
DB
JMS
Beamline
Control
Module
VESPERS
CA
SAN
•
The BCM provides a high-level interface to the low-level control system (EPICS).
•
Logical and physical separation of business logic and control logic.
•
Virtual device abstraction that provides independence from low-level control system.
•
Virtual devices can be logically organized into a device hierarchy.
•
Basic devices can be combined to build more functional devices.
•
Communication with external applications using two message queues (ActiveMQ).
www.lightsource.ca
Web Application
HTTP
Web
Application
DB
JMS
Beamline
Control
Module
VESPERS
CA
SAN
•
A J2EE Servlet application that provides a web-based interface Science Studio.
•
Tools: Spring (MVC), iBATIS (ORM), JSecurity (Apache Ki), Apache Tomcat
•
Divided into two parts: the Core application and the VESPERS beamline application.
•
Core application is responsible for providing access to the business objects.
•
VESPERS application is responsible for remote control of the VESPERS beamline.
www.lightsource.ca
Database
HTTP
Web
Application
DB
JMS
Beamline
Control
Module
VESPERS
CA
SAN
•
Metadata associated with the operation of a remote controlled beamline and the organization of
experimental data collected on that beamline.
•
A project is the top level organizational unit and is associated with a project team.
•
A session defines a period of time allocated to a project team to conduct experiments.
•
An experiment relates a sample and the technique being applied to that sample.
•
A scan records the location of the acquired experimental data.
www.lightsource.ca
Experimental Data Storage
HTTP
Web
Application
DB
JMS
Beamline
Control
Module
VESPERS
CA
SAN
• Experimental data is stored at the CLS.
• Common directory structure shared with other beamlines.
• A large data storage facility is now operational at the University of
Saskatchewan as part of WestGrid.
www.lightsource.ca
VESPERS Web Interface
HTTP
Web
Application
DB
JMS
Beamline
Control
Module
VESPERS
CA
SAN
•
Rich web interface to Science Studio and the VESPERS beamline.
•
Designed to be used over commodity broadband internet.
•
Developed for the Firefox web browser without any additional plugins or extensions.
•
Known to work with other browsers, but requires the Canvas HTML tag.
•
AJAX is used for the VESPERS interface to provide device values in pseudo real time.
•
ExtJS, a JavaScript framework, provides many advanced GUI elements.
www.lightsource.ca
www.lightsource.ca
Beamline Setup
www.lightsource.ca
Experiment Setup
www.lightsource.ca
XRF (X-Ray Fluorescence)
www.lightsource.ca
Beamline Hutch Cameras
www.lightsource.ca
Experimental Data Viewer
www.lightsource.ca
www.lightsource.ca
ScienceStudio
www.lightsource.ca
Scanning
www.lightsource.ca
XRD Screen
www.lightsource.ca
(a) X ray Fluorescence spectra and mapping
Orientation
(b) Laue X ray Diffraction and mapping
www.lightsource.ca
Strain
XRF Image
XRD: orientation
XRD:strain
Optical image
VM strain
www.lightsource.ca
University of Western Ontario
Nan-Fabrication Facility
Using ScienceStudio
www.lightsource.ca
Nanofabrication Facility




The Western Nanofabrication Facility is an open user facility at the
University of Western Ontario for the fabrication of micro and nano
structures.
LEO 1540XB Scanning Electron Microscope (SEM) with an
integrated Oxford x-ray system.
Oxford
x-ray
system
controlled by INCA.
is
INCA provides limited control of
the SEM, and spectral analysis
and quantification.
www.lightsource.ca
Nanofab Integration Design
www.lightsource.ca
Nanofab Implementation
www.lightsource.ca
Initial Test: INCA Remotely
www.lightsource.ca
User Office
www.lightsource.ca
www.lightsource.ca
User Office Modules
• User Profile/Authentication
– User Profile
– User Roles
– Authentication (LDAP/AD)
Deployed
Deployed
Deployed
• Information Model
– Beamlines
– End-stations
– Techniques
Done
Done
Done
• Notification Module
www.lightsource.ca
Deployed
User Office Modules
•
•
•
•
•
•
Proposal Submission
User Feedback Survey
Scheduling Module
Workflow Engine
Sample Management
Publication Tracking
www.lightsource.ca
In Progress
Prototype
Prototype
Evaluation
Requirements
Requirements
User Office - Training Module
• First User Office Module Deployed
• User Profile
• 1,400 + user records/accounts created
• Training Module
– 53 Training Modules Setup and past user
records manually transferred
www.lightsource.ca
Calendar view in JQuery
www.lightsource.ca
User Office Workflow Example
Prototype Implementation
1. CLS issues a call for proposals and gives deadline
2. Beamline users submit proposals
3. User Office administrator ends registration or extends deadline
4. User Office administrator assigns proposals to user office reviewers
5. Reviewers look at proposals and rank them
6. User Office looks at ranking and chooses the proposals to accept
7. Accepted proposals contact persons are notified
8. Beamline User completes training (web service)
9. After training is completed (simulated by a delay) the CLS is notified
www.lightsource.ca
User Login
www.lightsource.ca
Admin – Add Proposal
www.lightsource.ca
Admin - Setup for Cycle
www.lightsource.ca
Admin – Creating Roles
www.lightsource.ca
Grid Computing
www.lightsource.ca
HPC Computing in Canada
• Compute Canada
– Responsible for High Performance Computing in Canada
– Each regional grid is a member of Compute Canada
•
•
•
•
ACEnet – Atlantic Canada
CLUMEQ - Quebec
SCINET - UofT
HPCVL – Queens, Royal Military Collage
St. Lawrence, Carlson, Ottawa, …
• RQCHP - Quebec
• SHARCNET - Ontario
• WESTGRID – Western Canada
CLUMEQ Facility
www.lightsource.ca
Grid Data Storage?
• UofS is the host for the
new WestGrid data
storage facility
• Cost: $3.2 M
• Includes on-line and
archival storage
• Two sites on campus
Photo: tape backup unit holding 6,000 tape (each @1TB)
www.lightsource.ca
IBM Cell Processor (3.2 GHz)
www.lightsource.ca
www.lightsource.ca
User Configurable Light Paths
www.lightsource.ca
Integrating LPA with User
Configurable Light Paths
Lightpath Accelerator
controls a software
virtual cross-connect
that commands
UCLP.
real device
In effect, CA*Net4 is
treated as a single
lightpath cross-connect
virtual device
www.lightsource.ca
real device
File Transfer Demo
www.lightsource.ca