AMI
AMI T
Tutorial
t i l
J
January
2009
Solveig
S
l i Alb
Albrand
d
Jerome Fulachier
Fabian Lambert
29/01/2009
S.A.
1
AMI
General Things
• ATLAS Metadata Interface.
Interface
• A generic cataloging framwork – used in
ATLAS for
f ddataset di
discovery (and
( d also
l Tag
Collector + one or two other things)
• Portal page. http://ami.in2p3.fr/
• Deployed at CCIN2P3 (French Tier 1)
1).
• Dedicated ORACLE cluster.
• Two tomcat servers with load balancing.
• Full Tutorial – takes ~ 90 minutes
29/01/2009
S.A.
2
AMI
Things people say
say.
•“[AMI]
[
] has an impressively
p
y complete
p
information
f
content
within a somewhat complex user interface, which is by part
due to the quantity of the available information.”
•“Thanks
Thanks for AMI
AMI, it is a really great tool to help find data
sets. I noticed one thing just now that should be fixed
though.“
•“I
“I browsed
b
d AMI,
AMI but
b t I couldn't
ld 't find
fi d anything
thi off use there
th
…””
•“AMI is one of the more user-friendly ATLAS products! “
• ..it's
•“
it's been extremely frustrating …….. why can’t
can t it be more
like GOOGLE?”
Please retain that there are several ways
y of ggettingg to the same
information, that we cannot invent information, and that we
are (almost!) always pleased to receive comments, complaints
and suggestions.
gg
29/01/2009
S.A.
3
AMI
Where does AMI get its data?
• Real data : From the Tier 0 : DAQ data, and first
reconstruction
i is
i registered
i
d in
i AMI < 5 minutes
i
after
f it
i is
i
declared “ready” (both datasets and files). Config tags for
real data.
• Monte Carlo and reprocessing.
– From the Task Request DB : Tasks, dataset names, MC and
reprocessing
i config
fi tags
– From the production DB : Finished tasks – files and metadata.
• From physicists.
– M.C. Dataset number info, physics group owner,…
– Corrected cross sections and comments.(New)
– DPD tags (coming soon)
– …
29/01/2009
S.A.
4
AMI
Exercises. (or just follow the
tutorial!)
• Overview page :
– Notice that not all datasets are VALID.
• Invalid because all tasks failed, declared bad, or deleted.
• Invalid datasets are hidden by default.
– Project tags – getting more controlled, following the 2008
nomenclature. (data08_cos for example)
• If you know the project tag this is perhaps the simplest way to get to
the list.
• Note: AMI searches over all non archived catalogues in
parallel. (archived catalogues can be searched on request).
The catalogue schema can be different.
• Tutorial
i l link
li k
http://ami.in2p3.fr/opencms/opencms/AMI/www/Tutorial/
Simple search interface.html
Simple_search_interface.html
29/01/2009
S.A.
5
AMI
The “Simple”
Simple search
• Simple Search – search on part of the name.
name
– Example “L1Calo “
– Note that
h “%” iis usedd for
f wild
ild carding.
di
• Search on keywords,
y
, and comments – a
search in many fields – takes longer.
• Tutorial link:
http://ami.in2p3.fr:8080/opencms/opencms/AMI/
www/Tutorial/Simple_search_interface.html
/T t i l/Si l
h i t f
ht l
29/01/2009
S.A.
6
AMI
The “Advanced”
Advanced search
• Examples to try:
– Physics group = “phys-beauty”
• The only way to get invalid datasets.
• Notice that there are results in several catalogues.
Choose “mc08” in the drop down box.
• Tutorial link:
http://ami.in2p3.fr:8080/opencms/opencms/AMI/
www/Tutorial/Advanced_search_interface.html
29/01/2009
S.A.
7
AMI
Looking at the results
results.
• Start from the “phys-beauty”
phys beauty datasets in mc08.
• Use the “group by” under data_type to find the
RDO.
• “Filter” on the dataset name, or the dsn to find
19900.
9900.
• http://ami.in2p3.fr:8080/opencms/opencms/AMI/
www/Tutorial/Exploring
p
g_the_results.html
• http://ami.in2p3.fr:8080/opencms/opencms/AMI/
www/Tutorial/Refine_the_Results.html
29/01/2009
S.A.
8
AMI
Looking at the results some more
• Click on “additional
additional fields
fields” . Notice the prodsysStatus.
prodsysStatus
Choose one which has “EVENTS_AVAILABLE”.
mc08.019900.Pythia_directUpsilonmu6mu4.digit.RDO.e328_s404_d117
•
•
•
•
Clickk on “details”
Cli
“d il ” – to get the
h “child”
“ hild” elements.
l
Click on “event_range” to get the monte carlo task details.
Click on “files”
files to browse the files.
files
Notice the “history” buttons (“bread crumb trail”). You can
event remove them.
• Click on “Provenance” of dataset. You can use this to
navigate along the production chain. (get to EVGEN from
RDO for example)
p )
• Notice that you can obtain a list of datasets for input to
GANGA. (Advanced/Export/Ganga)
29/01/2009
S.A.
9
AMI
Gettingg to the data
• Go back to the details of the dataset we found
previously.
mc08.019900.Pythia_directUpsilonmu6mu4.digit.RDO.e328_s404_d117.
• Click on “DQ2”. This page shows an aggregation
of information from DQ2. You can subscribe to
the dataset from this page if you like, going
through the DDM permission system of course.
• AMI will also make you “dq2-get” commands, but
in general you should not do it.
• This page manages the new dataset container
concept.
29/01/2009
S.A.
10
AMI
Configg Tags
g
• Example: e377_s462_r563 (last field of dataset name)
e377 Î event generation parameters
s462 Î simulation parameters
r563 Î reconstruction parameters
(p tags
g implementation
p
(dpd
( p pproduction)) still in discussion: see r9999))
• Interpretation of Config tags
http://ami in2p3 fr/opencms/opencms/AMI/www/Referenc
http://ami.in2p3.fr/opencms/opencms/AMI/www/Referenc
eTables/
• Searchingg startingg from the Configg Tag.
g
http://ami.in2p3.fr/opencms/opencms/AMI/www/Tutorial/
ConfigTags
29/01/2009
S.A.
11
AMI
AMI Accounts
• Logging
i on to AMI.
– In general you do not need to log on to read (at the
moment))
– You can make an AMI account to access a personal
page.
– You must log on for subscription requests.
– Once
O
you llog on to
t AMI you can make
k bbookmarks.
k
k
– Tutorial link :
http://ami in2p3 fr/opencms/opencms/AMI/www/Tutori
http://ami.in2p3.fr/opencms/opencms/AMI/www/Tutori
al/Other_AMI_basic_functionalities.html
29/01/2009
S.A.
12
AMI
A real data example.
example
• Use one of the search interfaces to gget to the
datasets for “data08_1beammag”
y the RAW data.
• Refine yyour search to list only
• Use the numeric statistics to sum over the RAW
events. ((totalEvents field))
• External links
– “Run_Info” to gget to the run summaryy link. Includes
COOL data.
– DAQ_Config
• Config tags. Click on details, and then on
“Config_Tag” to find what “o4” means.
29/01/2009
S.A.
13
AMI
Other stuff
stuff.
• pyAMI.
py
Everything
y
g in AMI can be obtained from
the python client.
http://ami.in2p3.fr/opencms/opencms/AMI/www/
Tutorial/pyAMI html
Tutorial/pyAMI.html
• Ad Hoc queries. (For really advanced users!)
http://ami.in2p3.fr/opencms/opencms/AMI/www/
Tutorial/Refine_the_search.html
• Embedding AMI queries in third party web pages.
http://ami.in2p3.fr/opencms/opencms/AMI/www/
Client/DatasetQuery.html
29/01/2009
S.A.
14
AMI
AMI
AMI
AMI
AMI
AMI
AMI
AMI
AMI
AMI
AMI
AMI
AMI
AMI