The DAWNSci Eclipse science
usecases
Jun Aishima
Software Engineer
Diamond Light Source
Didcot, UK
1.
2.
3.
4.
What is a light source and why do science there?
What data do you get at a synchrotron?
What do you do to the data?
How do DAWN and DAWNSci Eclipse help with
this?
5. How do we use DAWN and DAWNSci Eclipse to
analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to
analyze your data?
Synchrotron
AKA – cool word, but what does it mean?
syn·chro·tron/ˈsiNGkrəˌträn/
Noun:A cyclotron in which the magnetic field
strength increases with the energy of the
particles to keep their orbital radius constant.
“They are machines which produce
very strong light used for many
different type of scientific experiments
and sometimes other things.”
Diamond Light Source, Didcot,
Oxfordshire, UK
• (Video has been removed to allow the talk to
fit into size guidelines – see
http://www.diamond.ac.uk/Public/Multimedia/
Animations.html to view the animation)
1.
2.
3.
4.
What is a light source and why do science there?
What data do you get at a synchrotron?
What do you do to the data?
How do DAWN and DAWNSci Eclipse help with
this?
5. How do we use DAWN and DAWNSci Eclipse to
analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to
analyze your data?
Detectors of
Various
flavors
1.
2.
3.
4.
What is a light source and why do science there?
What data do you get at a synchrotron?
What do you do to the data?
How do DAWN and DAWNSci Eclipse help with
this?
5. How do we use DAWN and DAWNSci Eclipse to
analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to
analyze your data?
The collected data may look like this…
…180 images
When what you really want is this…
1.
2.
3.
4.
What is a light source and why do science there?
What data do you get at a synchrotron?
What do you do to the data?
How do DAWN and DAWNSci Eclipse help with
this?
5. How do we use DAWN and DAWNSci Eclipse to
analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to
analyze your data?
DAWN - Acquisition and Analysis with RCP
@ Diamond Light Source
Jython
and Python
Client server
technology
Communication with
EPICS and hardware
Scan mechanism
Data read, write,
convert
Visualisation
Communication
with external
analysis
www.opengda.org
Acquisition
Analysis
tools
Metadata structure
Workflows
www.dawnsci.org
Analysis
DAWN is a
collection of
generic and
bespoke ‘views’
collated into
‘perspectives’.
The perspectives
and views can
be used in part
or whole in either
the GDA or
DAWN.
Collaborative Approach
Consultants
Members
Kichwa Coders Ltd.
External Users
Science Working Group
Now an Eclipse project!
https://projects.eclipse.org/projects/technology.dawnsci
‘Shoulders of Giants’
AKA – didn’t think of everything
•
•
•
•
•
•
•
•
RCP many of the core features, editors, toolbars, views, projects
Ptolemy 2 (a version known as ‘Passerelle’) workflow and pipelining
GEF for visualization of pipeline graphs
Draw2D for 1D and 2D plotting (SWT XY Graph)
Pydev for python/jython scripting layer used by the scientists
HDF5 libraries for storing large data sets
SWT/Jface – lazy viewers being used extensively for large trees and tables
Apache, Eclipse-WST, springsource, JDK, and many more of course...
Did you see our poster yesterday, presented by Bahá?
• The DAWNSci Eclipse Project: How Eclipse
Is Used for Science (Bahá El Kassaby)
– Printed copies or PDF file available! Ask us!
Data in DAWNSci
Example : Loading and Manipulating Data
This example
• Columns of ASCII data
• Separated by tabs
• Read as 1D arrays
Other format with a loader:
srs, dat, flt, gff, mca, csv, xy, xye,
txt, tif, tiff, cbf, img, ciff, mccd, edf,
pgm, cor, bruker, jpg, jpeg, png,
f2d, msk, mib, mar3450, pck3450,
raw, mrc, gz, bz2, zip, h5, hd5,
hdf5, nxs, nexus, hdf, mat
Extension point to add more...
Demonstration – Visual Tools
1.
2.
3.
4.
What is a light source and why do science there?
What data do you get at a synchrotron?
What do you do to the data?
How do DAWN and DAWNSci Eclipse help with
this?
5. How do we use DAWN and DAWNSci Eclipse to
analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to
analyze your data?
The Processing pipeline
• Previous systems
– Required much effort by the developer
– User experience was too generic – not optimized
for this purpose
• Simplify (coding), simplify (GUI), simplify
(capabilities)
• Result: wide adoption and rapid development
for several experimental techniques
• Developed by Jacob Filik at Diamond
• Available in DAWN
Why Processing in DAWN?
Non-crystalline diffraction
Simple? Operation
Calculation
Error
handling
Simple? Model
Mouseover in GUI
Allow drag and drop
Provide a button to open file dialog box
Radio box in GUI
Number box in GUI with limits
Simple?
File
list
Model
List of
processing steps
Output
Input
Demonstration of pipeline
1.
2.
3.
4.
What is a light source and why do science there?
What data do you get at a synchrotron?
What do you do to the data?
How do DAWN and DAWNSci Eclipse help with
this?
5. How do we use DAWN and DAWNSci Eclipse to
analyze our data?
6. How can you use DAWN and DAWNSci Eclipse to
analyze your data?
Create your own operations and pipelines!
• Create an operation –
org.eclipse.dawnsci.analysis.api.operation
extension point
– Dataset operations – including lazy operations
• Create a model – if necessary
– Any additional data required for the calculation
– Reflection on data type and annotations are used to
build the GUI for users to enter parameters
A lot of existing, reusable operations
• Sector integration
– Typically, reduce a two-dimensional image into a onedimensional dataset
• Normalization
– Remove frame-to-frame variation to put them all onto the
same scale
• Subtraction
– Removing artifacts by subtracting background data from
the sample data
• Averaging
– Multiple noisy points can be averaged
Future work on the Processing pipeline
• Support more sciences (XPDF)
• Add branching, merging, and other non-linear
flows
• Run calculations via HPC
• Passerelle workflows
• More, faster, better processing steps
Try DAWN for yourself!
• DAWN 1.7.1, available now!
• http://www.dawnsci.org
• See directions on how to use DAWNSci – “Getting started
with DAWNSci”
• Videos on how to do use slicing, data reduction, Python
scripting, processing at
https://www.youtube.com/user/DAWNScience
• Follow us on Twitter - https://twitter.com/dawnscience
• Read the article about us in the Eclipse newsletter https://www.eclipse.org/community/eclipse_newsletter/2015/j
anuary/article3.php
• DAWN paper accepted for publication, doi:
10.1107/S1600577515002283
• Email us: scientificsoftware@diamond.ac.uk
Evaluate the sessions
Sign in: www.eclipsecon.org
+1 0 -1