Status of AIDA and JAS 3
by Victor Serbo
•
•
•
•
•
•
•
What is AIDA
History/Collaboration/Documentation
Some Details
Examples
Future plans
JAS: Description and Status
Conclusion
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
1
What is AIDA
• Abstract Interfaces for Data Analysis (AIDA)
• “The goals of the AIDA project are to define abstract
interfaces for common physics analysis objects, such as
histograms, ntuples, fitters, IO etc.The adoption of these
interfaces should make it easier for developers and users
to select to use different tools without having to learn new
interfaces or change their code. In addition it should be
possible to exchange data (objects) between AIDA
compliant applications.”
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
2
Abstract Interfaces
• Only define “protocol” that analysis objects have to
understand, no internal details
• Can use object only through Abstract Interface
• Define only pure virtual methods
– AIDA defines behavior, analysis tool provides
implementation
• Use Factories to be able to switch implementations
easily:
– IAnalysisFactory af = IAnalysisFactory.create();
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
3
Motivation
• Advantages
– The user needs to learn only one set of interfaces
– Same user code can be used with different AIDAcompliant analysis applications
– Different analysis tools can exchange analysis objects
• same storage format, use functionality from other tools
– Pool experience of different developer teams
• LHC++, OpenScientist, JAS
• Two versions of AIDA interfaces
– One for C++
– One for Java
– As identical as possible
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
4
AIDA Example
• Use same code with any AIDA-compliant analysis
tool.
A
User code
(e.g.
GEANT4)
Victor Serbo, SLAC
Analysis tool 1
I
D
Analysis tool 2
A
ACAT’2002, Moscow, June 24-28, 2002
5
History
• Initial idea formed during discussion at HepVis-99
workshop at Orsay
– Informal AIDA discussions at CERN in 2000
– AIDA workshops:
• January 2001 - Paris/Orsay
• April 2001 - Boston (preceding HepVis 2001)
• Informal meetings (e.g during Geant4 meetings and video
conferences)
• June 2002 – CERN
– Interfaces have been designed by discussion and
(eventual) consensus
• Takes some time, but result is well though out and robust
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
6
Organization - Developers
• No formal collaboration/author list.
• Some people who have contributed (ideas, code,
etc).
– Guy Barrand, Pavel Binko, Grzegorz Chwajol, Mark
Donszelmann, Wolfgang Hoschek, Tony Johnson,
Emmanuel Medernach, Dino Ferrero Merlino, Lorenzo
Moneta, Jakub Moscicki, Ioannis Papadopoulos,
Andreas Pfeiffer, Max Sang, Victor Serbo, Max Turri
• Apologies to people accidentally missed
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
7
Organization – Code, Documentation
• AIDA – open source project
– CVS repository: cvs.freehep.org
• “anonymous” download available
– Web page: http://aida.freehep.org
•
•
•
•
General information, relevant links
Tutorial, users’ guide, examples
Downloads and web-browsable source code
Test cases (coming soon)
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
8
Current Status
• AIDA Version 2.2 released (December 2001)
– First “End User” release
– Three implementations of AIDA exist
• Anaphe/Lizard (C++)
– http://anaphe.web.cern.ch/anaphe
• Open Scientist (C++)
– http://www.lal.in2p3.fr/OpenScientist
• JAIDA/JAS (Java) + AIDA-JNI 1.0 (C++)
– http://java.freehep.org/lib/freehep/doc/aida
– GEANT4 adopted AIDA for analysis
• AIDA 3.0 Currently under discussion
– Release: end of September 2002
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
9
AIDA Interfaces Summary
•
•
•
•
•
AIDA Factories
ITuple
IHistogram
ICloud
ITree
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
10
Example Program (Java)
l
• Create, fill, and
view 1D and 2D
histograms
;
• Factories:
– Can switch
implementations
without recompiling
– Framework can hide
factories
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
11
ITuple
• ITuple - interface to the Data
–
–
–
–
–
“get/set” methods for double, float, int, …
Information about columns: min, max, mean, rms
Navigating: start(), next(), skip(int nRows)
Project ITuple into 1D, 2D, 3D histogram
New features for AIDA 3.0:
• Support for complex internal structures (subfolders)
• Chaining of ITuples
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
12
IHistogram (1D-3D)
• Binned histogram: IHistogram1D, 2D, 3D
–
–
–
–
–
“fill” methods (with/without weight)
Histogram info: entries, mean, rms, axis
Bin info: centre, entries, height, error
Histogram arithmetic: add, multiply, divide
Convenience methods, like coordinate-to-index
conversion
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
13
ICloud
• Unbinned collection of points: ICloud1D, 2D, 3D
– Can represent scatter plot, dynamically
rebinnable histogram
– Can be converted to a binned histogram
– Standard “get/set” methods for entries
– Collection info: lower, upper, mean, rms
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
14
IFunction and Fitting
• Fitting: IFunction, IFitFunction
– IFunction – simple interface, allows to set parameters
and get function value
– IFitFunction – fit function to a histogram
•
•
•
•
Extends IFunction
Various fit control methods: step size, bounds, etc.
Allows to perform fit and get results
AIDA 2.2 fitting functionality fairly limited
– AIDA 3.0 (Under discussion) extended functionality
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
15
ITree
• ITree
– directory-like structure (Unix directory convention)
• Methods like: cd, ls, mkdir, etc.
– AIDA analysis objects (tuples, histograms, clouds, ets.)
exist within ITree directories
– “save/restore” functionality, hides storage details from
the user
• Compatible with database or file storage
• Can support multiple file formats
• Mount/Unmount functionality (like unix) allows multiple
stores to be seamlessly merged
• AIDA XML format is defined for data interchange
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
16
Future (unofficial ideas)
• Developer-level interfaces
– Code sharing
– More robust operation
• Put AIDA-based utilities in CVS
– Utility to test AIDA implementations
– User contributions
• Common binary storage format
• More development in fitting interfaces
– IFitter, IOptimizer, IFitResult, IPlotter
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
17
JAS
• Java Analysis Studio (JAS)
– Written entirely in Java – really portable
– Originally targeted at offline analysis – but also used
extensively for online monitoring
– Rich, easy to use GUI
• Can view trees, plots, HTML pages with embedded active objects
– Built-in editor/compiler for writing analysis code
• Based on open source JEdit editor (pure Java editor, syntax
highlighting, auto indent, parenthesis matching, etc.)
– Local and Client-Server Operation
• Access local or remote analysis objects
• Can send analysis code to the data and ship back only the results
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
18
JAS (continue)
• Java Analysis Studio
– Modular/Extensible via Plugins
• Plugins extend basic JAS functionality
– Define experiment-specific utilities (event display, analysis utilities,
specialized tables)
– Add menus, create control areas, consoles, and output pages
• Data Format Independent through Data Interface Modules (DIM)
– DIM is special type of Plugin – makes data available for JAS
– DIMs exist for PAW, Root, Flat Files, SQL, stdHEP, HepTuples
– You can write your own DIM for your data format
• Plugin examples:
– Particle decay tree
– WIRED event display
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
19
Tree area
provides
access to
analysis
objects. Each
item has
popup menu
and doubleclick action.
Pages can display
histograms. User can
control layout, add,
remove plots.
JAS GUI
Built-in HTML viewer
with embeddable
“objects” (plots,
buttons, etc.).
Plot Widget,
shows data in
real time,
optimized for
fast refresh
performance.
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
20
JAS Usage
• BaBar Online Monitoring
– Online Monitoring API
– Static HTML pages with embedded live and reference
plots. Plots can be overlaid.
• US Linear Collider Studies
– Have an entire recon+analysis package written in Java
• Using JAS as analysis interface
• Making use of remote data access using repository at
University of Pennsylvania
• Other smaller scale users
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
21
JAS 3
• Major rewrite of many JAS components
– Use AIDA interfaces and JAIDA implementation for
analysis: histograming, histogram arithmetic, fitting,
tuples etc.
– The JAS plot widget will become the FreeHEP plotter
– JAS itself will become a modular application
framework, into which various analysis components
can be plugged. Major plugins will include WIRED, the
JAS Plotter, and JAIDA.
– Scripting support (Jython ?)
– New module Example: JAS Tuple Explorer, is being
used by the Babar Ambient Data explorer
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
22
Tuple Explorer
• Display Tree
Leaves
–
–
–
–
Plot
Histogram
XY Plot
Create new
columns
– Apply cuts
• BaBar uses
Tuple Explorer
as Ambient DB
browser
• Demo after talk
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
23
Conclusion
• AIDA is becoming mature project
– Functionally sound
– Regular meetings
– Release schedule
• Next release of AIDA and AIDA-compliant
analysis tools – September 2002
• JAS 3
– Easy to use and extend
– Uses Java implementation of AIDA for analysis
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
24
Details (backup):
•
•
•
•
•
Histogram
Cloud
Tuple
Tree
FreeHEP Plotter
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
25
Details - ITuple
• Interface to
the Data:
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
26
Details – IHistogram (1D-3D)
IHistogram:
Common functionality for
all histograms (like
entries, label, dimension,)
IHistogram1D
IHistogram2D
IHistogram3D
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
27
Details - ICloud
• Unbinned collection of points: ICloud1D, 2D, 3D
– Can represent scatter plot, dynamically rebinnable histogram
– Can be automatically converted to a binned histogram
ICloud:
Common functionality for
all histograms (like
entries, label, dimension,)
ICloud1D
ICloud2D
ICloud3D
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
28
Details – ITree
• Directory-like
structures:
ITree
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
29
Details - FreeHEP Plotter
• FreeHEP Plotter
The
FreeHEP plotter will be based on the existing JAS Plotter (JASHist), but
with changes to add new features requested by users, a plugin structure
to make adding new plot types easier, and a more flexible data model to
make attaching various different data types possible. The FreeHEP plotter
will fully support displaying all AIDA plots types.
• A partial list of design goals include:
– Support for histogram, XYPlot, ScatterPlot, 2D color maps, lego plots, contour plots ..
– Must be able to use any object as annotation or labels (arrows, latex, etc.)
– Must be able to build complex plots (multiple axes, shared axes, etc.)
– Extensible set of renderer
– Extensible set of data source interfaces
– Arbitrary map from axis to coordinate system (log, linear)
– Support 2D and 3D drawing (and in principle also inventor, opengl etc.)
– Able to support strings (sorted?), dates, numeric axis
– Binned and non-binned axes, rebinning for binned axis
– Output as postscript, SVG etc.
– Variable bin size
– Able to support legends, titles, annotations etc
– Able to select a range in 1D and 2D plots for zooming, fitting in range, cuts etc
– Labels should resize to fit bounds
– Support for functions, directly setting initial values
(next page …)
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
30
Details - FreeHEP Plotter (continue)
– User able to directly interact with objects, move them around (including
outside plot boundaries)
– Property pages and popup menus for customization
– Arbitrary interactions between plot and data (rebinning, zooming etc.)
– External objects should be able to insert (or delete) items into property
pages, popup menus
– Extensible using OO techniques (e.g. Specialized Axes)
– Plot should be able to provide menus to external GUI
– Saving/Restoring from XML
– Real time updating
– High performance (rapid updates)
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
31
June 2002 Developers Workshop
• 2 Day “Users Workshop”
• 3 Day “Developer Workshop”
– Items under discussion
•
•
•
•
•
Fitting (two proposals, from SLAC, CERN)
Improved plotting (IPlotter)
Graph (XYData)
Tuple chaining, merging
Small updates/extensions to API
– Input/Participation from new people welcomed!
Victor Serbo, SLAC
ACAT’2002, Moscow, June 24-28, 2002
32