Introduction to TAGs
Elizabeth Gallas
Oxford
ATLAS-UK
ATLAS
UK
Distributed Computing Tutorial
Edinburgh United Kingdom
Edinburgh,
29-30 January 2009
30 Jan 2009
Elizabeth Gallas
1
Outline
† Introduction:
„
„
„
„
TAG feedback
f db k (meet
(
the
h developers)
d
l
)
What is a TAG ?
TAGs in the ATLAS Computing model
TAG Contents and Formats
† ELSSI – Event Level Selection Service Interface
„ A web interface using TAGs for ATLAS event selection
† Important TAG related hyperlinks
† Hands on tutorial
„ S
Selecting
l ti
events
t off interest,
i t
t refining
fi i
selection
l ti
criteria
it i
„ Extracting that selection
„ Using selection as input to Athena analysis
30 Jan 2009
Elizabeth Gallas
2
The TAG Development
p
Group
p
• Argonne:
David Malon, Jack Cranshaw, Qizhi Zhang, Peter van Gemmeren
• Glasgow:
MikeKenyon, HelenMcGlone, TomDoherty
• CERN:
Florbela Viegas,
g , Trivan Pal
• Oxford:
Elizabeth Gallas
• Chicago:
Marco Mambelli
• Grenoble:
Solveig Albrand
• Orsay
y LAL:
Julius Hrivnac
• Sheffield:
Tulay Donszelmann (PAT)
Users send feedback to:
ATLAS Physics
y
Metadata Hypernews
yp
30 Jan 2009
Elizabeth Gallas
3
What’s a “TAG” ((in this talk))
†
†
ATLAS “TAG” overload:
„
„
„
„
„
„
Not
Not
Not
Not
Not
Not
an acronym (retrospectively TAGs Are Great !)
a CVS tag or a release
l
tag
what is tracked by the ATLAS Tag Collector
a COOL (Conditions Database) Tag
a configTag or a project tag
a B, flavor, or muon tag
„
ATLAS TAGs are:
In this talk:
†
†
„
About 200 indexed variables (Event-level metadata) for each event
„
Size: ~1 kB/event -> 1 TB/year
„
„
In TAG files – distributed
dist ib ted to all Tie
Tier 1 sites
In a relational database (Oracle) – distributed some Tier 1
In 2 format:
ATLAS TAG application is
†
†
A system using ATLAS TAGs
A variety of supporting tools and infrastructure
PURPOSE: Facilitates event selection for analysis
30 Jan 2009
Elizabeth Gallas
4
ATLAS TAGs in the ATLAS Computing
p
g model
† Stages of ATLAS reconstruction
¾ RAW data file
¾ ESD (Event Summary Data) ~ 500 kB/event
¾ AOD (Analysis Object Data) ~ 100 kB/event
¾ TAG (not an acronym) ~ 1 kB/event
† TAGs
„ Are produced in reconstruction
„ Can
C
b
be re-produced
d
d iin re-processing
i
„ Customized TAGs can be defined/produced
† TAG formats:
„ File based (produced in reconstruction chain above):
† AthenaAwareNTuple format (AANT)
„ Oracle Database (populated in ‘upload’
upload process):
† Available globally through network connection
† Adding ‘Run Metadata’ at Temporal, Fill, Run, LB levels
† Used by
y the TAG Browser (ELSSI)
(
)
30 Jan 2009
Elizabeth Gallas
5
What does the “TAG” contain ?
“TAG content”: Your event selection variables
„ ~200
200 variables/event:
† Event identification (run, event, lumi block, time …)
† Trigger decisions at all three levels (bit encoded)
† Numbers of electrons, muons, photons, taus, jets
„ pT, eta, phi for highest-pT objects
† Global quantities (e.g., scalar and missing ET)
† Detector status and quality words
† Physics and performance group status words
„ references to AOD,ESD,RAW for back-navigation
PAT (Physics Analysis Tools)
„ PAT group supports
t the
th ‘Official’
‘Offi i l’ TAG content
t t with
ith
input from the physics groups (YOU !)
„ See the PAT TWiki: TagForEventSelection
g
30 Jan 2009
Elizabeth Gallas
6
TAG related hyperlinks:
yp
TWiki with TAG content in ATLAS release:
„ https://twiki
https://twiki.cern.ch/twiki/bin/view/AtlasProtected/
cern ch/twiki/bin/view/AtlasProtected/
TagForEventSelection
TWiki for Tutorial:
„ https://twiki.cern.ch/twiki/bin/view/Atlas/EventTag
Tutorials
„ 4 Tutorial
T t i lC
Components
t
† ELSSI (Browser based)
† File
File-based
based TAGs (Command line based)
† JiveXML with TAGs (Atlantis Event Display)
† TAG based selection with PAthena on the Grid
ELSSI - Event Level Selection Service Interface:
„ https://voatlas18.cern.ch/tagservices/index.htm
30 Jan 2009
Elizabeth Gallas
7
ELSSI: Implementation
p
& Features
† ELSSI is a web based supporting tool (using TAG Database)
† Running
g on server at CERN ((PHP and Javascript
p on
backend)
„ Supported: Firefox, Safari (not IE)
† H
Helps
l you navigate
i t th
through
h “TAG content“
t t“
„ Formulate multi-dimensional queries
„ At Run/LB and Event level
User
„ Query caching
Input
† Retrieve results (number of forms)
„ Event counting
g
„ Display Results
Stream Temporal Quality Trigger Physics
† Graphical
† tabular
„ Retrieve events in ROOT file
„ Register collection in AMI
30 Jan 2009
COUNT SELECT RETRIEVE
events attributes events
Elizabeth Gallas
8
https://voatlas18.cern.ch/tagservices/index.htm
ELSSI
Gateway
Choose
FDR2C
li k
link
30 Jan 2009
Elizabeth Gallas
9
ELSSI – 3 Frames: Banner, Session, Browser
Banner
ELSSI
Browser
Gateway
Session
30 Jan 2009
Elizabeth Gallas
10
ELSSI – Useful buttons
ELSSI
Gateway
30 Jan 2009
Elizabeth Gallas
11
Main Frame: the Browser Frame
1.
2.
3.
“Tabbed format”
1. Create Query
• Temporal (run,time…)
(run time )
• Streams
• Data quality
• Trigger
• Physics attributes
2. Review Query
3. Perform Query
• Count
• Display results
• Tabular
• Plot
• Retrieve event collection
30 Jan 2009
Elizabeth Gallas
12
Left Frame: the Session Frame
† Save/Share Session(s)
„ S
Save a query session
i
for later use
„ Uses cookies
† Browser specific
„ Share saved sessions
with others
† Selection Summary
y
„ Displays selection
criteria of your session
30 Jan 2009
Elizabeth Gallas
13
Summary
y
† Metadata in a database is a powerful tool
„ A great variety of dynamic selection criteria,
criteria
reporting and retrieval become possible
„ Technical challenge (databases and ATLAS tools)
† Development is ongoing to improve
„ Interfaces
† Functionality
† Usability
„ Integrate
I t
t infrastructure
i f
t
t
ttools
l
† For comments/suggestions:
ATLAS Physics Metadata Hypernews
30 Jan 2009
Elizabeth Gallas
14
Backup Slides
30 Jan 2009
Elizabeth Gallas
15
TAG job with Pathena
Panda
Pathena
jobOption
ELSSI or
POOL Collection
Utilities
Output
Dataset
events root
events.root
POOL Collection
Utilities
TAG DB
events.root
events.root
events1.root
DQ2
Catalogues
DQ2 Client
30 Jan 2009
Elizabeth Gallas
16
Ganga
g TNT
30 Jan 2009
Elizabeth Gallas
17
Working
g on … ((1))
† Incorporating Detector Status (DQ)
† Loading online Runs
„ Even in the absence of TAG production
† An ELSSI “skim”
skim button (Skimming Service)
using GangaTNT:
„ From your selected TAG events, generate an AOD
dataset with only those events
„ Output registered as new dataset
„ Runs GANGA on the server to run the job on behalf
off the
h user
† User proxy is delegated to the server
† Monitoring
g loop
p checks jjob status and notifies user
of job completion / failure
† Integrate GangaTNT into GangaGUI.
30 Jan 2009
Elizabeth Gallas
18
Working
g on … ((2))
† Using TAGs with PAthena: Steps
„ create an event collection (events.root) with ELSSI
„ setup Pathena and the Grid environment
„ use a joboption like the one you'd use to run a local
analysis based on TAGs
„ submit the Pathena job
„ use Panda monitoring to check for completion
„ use DQ2 enduser clients (dq2
(dq2-get)
get) to retrieve the
output files
† Problem: AOD to AOD (copy) is problematic in
some releases
30 Jan 2009
Elizabeth Gallas
19
Browser front page for FDR2 – ‘Temporal Tab’
30 Jan 2009
Elizabeth Gallas
20
Browser – ‘Stream Tab’
30 Jan 2009
Elizabeth Gallas
21
Browser – ‘ Trigger Tab’
30 Jan 2009
Elizabeth Gallas
22
‘Ph i Attributes
‘Physics
A ib
Tab’
T b’
30 Jan 2009
Elizabeth Gallas
23
‘Review Q
Query
y tab’
30 Jan 2009
Elizabeth Gallas
24
‘Perform Query’:
y Count tab
Count events meeting criteria … revise
criteria… count again… revise criteria …
30 Jan 2009
Elizabeth Gallas
25
‘Perform Query’: Display Results tab
† Get an idea of the
attribute distributions
† Draw histograms
g
30 Jan 2009
Elizabeth Gallas
26
Retrieve Events
† After refining query, generate a ROOT TAG
file with selected events
30 Jan 2009
Elizabeth Gallas
27