The ROOT System A Data Access & Analysis Framework Overview PHYSTAT Workshop on Statistical Software March 2004, MSU René Brun CERN/PH http://root.cern.ch ROOT Overview 1 Project History Jan 95: Thinking/writing/rewriting/??? November 95: Public seminar, show Root 0.5 Spring 96: decision to use CINT Jan 97: Root version 1.0 Jan 98: Root version 2.0 Mar 99: Root version 2.21/08 (1st Root workshop FNAL) Feb 00: Root version 2.23/12 (2nd Root workshop CERN) 9 years !! Mar 01: Root version 3.00/06 Jun 01: Root version 3.01/05 (3rd Root workshop FNAL) Jan 02: Root version 3.02/07 (LCG project starts: RTAGs) Oct 02: Root version 3.03/09 (4th Root workshop CERN) Dec 03: Root version 3.10/02 (last PRO release) Feb 04: Towards version 4.00 (5th Root workshop SLAC) R.Brun ROOT Overview 2 CERN/LHC/Geneva R.Brun ROOT Overview 3 a R.Brun ROOT Overview 4 The CMS detector in construction (2000 physicists) R.Brun ROOT Overview 5 The ALICE detector (1100 physicists) R.Brun ROOT Overview 6 A collision in CMS R.Brun ROOT Overview 7 Introduction ROOT Overview 8 The ROOT web pages http://root.cern.ch General Information and News Download source and binaries Howto & tutorials User Guide & Reference Guides Roottalk Digest & Forum R.Brun ROOT Overview 9 ROOT Statistics – Supported Platforms 3 major type of OS’es Unix, Windows, Mac OS X 10 different CPU’s IA-32, IA-64, Sparc, Alpha, PA-RISC, PowerPC, MIPS, ARM 11 different compilers Gcc, kcc, ecc, icc, CC, cc, VC++, … 41 Makefiles ./configure<arch>; make R.Brun ROOT Overview 10 ROOT Statistics – Available Binaries 27 binary tar balls R.Brun ROOT Overview 11 ROOT Statistics – Distributions and Number of Users 205,000 binaries downloaded >1,000,000 clicks per month 60,000 docs in 2 years 3000 registered users 950 users subscribed to roottalk R.Brun ROOT Overview 12 Trends More and more GRID oriented data analysis More and more experiment-independent software Parallelism on the GRID Batch/Interactive Access to Catalogs Efficient Access to large and structured event collections Histogram Ntuple viewers Data Presenters R.Brun Interaction with user & experiment classes ROOT Overview Resource Brokers Process migration Progress Monitors Proxies/caches Virtual data sets 13 ROOT directories root bin R.Brun lib include tutorials ROOT Overview etc icons 14 R.Brun ROOT Overview 15 Plug-in Manager name Plugin.TFile: ^rfio: +Plugin.TFile: ^castor: char*,Int_t,Int_t)" +Plugin.TFile: ^dcache: +Plugin.TFile: ^chirp: Plugin.TSystem: ^rfio: Plugin.TSQLServer: ^mysql: +Plugin.TSQLServer: ^pgsql: +Plugin.TSQLServer: ^sapdb: +Plugin.TSQLServer: ^oracle: Plugin.TGrid: ^alien Plugin.TVirtualPad: * Plugin.TVirtualHistPainter: * Plugin.TVirtualTreePlayer: * Plugin.TVirtualTreeViewer: * Plugin.TVirtualGeoPainter: * Plugin.TVirtualUtil3D: * Plugin.TVirtualUtilHist: * Plugin.TVirtualUtilPad: * Plugin.TVirtualFitter: Minuit +Plugin.TVirtualFitter: Fumili Plugin.TVirtualPS: ps +Plugin.TVirtualPS: svg Plugin.TViewerX3D: x11 +Plugin.TViewerX3D: qt R.Brun class Shared lib How to call TRFIOFile TCastorFile RFIO RFIO "TRFIOFile(const char*,Option_t*,const char*,Int_t)" "TCastorFile(const char*,Option_t*,const TDCacheFile TChirpFile TRFIOSystem TMySQLServer TPgSQLServer TSapDBServer TOracleServer TAlien TPad THistPainter TTreePlayer TTreeViewer TGeoPainter TUtil3D TUtilHist TUtilPad TFitter TFumili TPostScript TSVG TViewerX3D TQtViewerX3D DCache "TDCacheFile(const char*,Option_t*,const char*,Int_t)" Chirp "TChirpFile(const char*,Option_t*,const char*,Int_t)" RFIO "TRFIOSystem()" MySQL "TMySQLServer(const char*,const char*,const char*)" PgSQL "TPgSQLServer(const char*,const char*,const char*)" SapDB "TSapDBServer(const char*,const char*,const char*)" Oracle "TOracleServer(const char*,const char*,const char*)" RAliEn "TAlien(const char*,const char*,const char*,const char*)" Gpad "TPad()" HistPainter "THistPainter()" TreePlayer "TTreePlayer()" TreeViewer "TTreeViewer(const TTree*)" GeomPainter "TGeoPainter()" Graf3d "TUtil3D()" Hist "TUtilHist()" Gpad "TUtilPad()" Minuit "TFitter(Int_t)" Fumili "TFumili(Int_t)" Postscript "TPostScript()" Postscript "TSVG()" X3d "TViewerX3D(TVirtualPad*,Option_t*)” QtX3d "TQtViewerX3D(TVirtualPad*,Option_t*)” ROOT Overview 16 root/tutorials (195) alien.C analyze.C anim.C approx.C archi.C arrow.C basic3d.C basic.C basic.dat benchmarks.C bent.C bill.C bug.C c1.C c1.eps canvas.C cernbuild.C cernstaff.C cernstaff.dat classcat.C cleanup.C clonesA_Event.C clonesA_Event.cxx clonesA_Event_cxx.d clonesA_Event.h compile.C copytree2.C copytree3.C copytree.C core customContextMenu.C customTH1Fmenu.C CVS demos.C demoshelp.C dirs.C R.Brun draw2dopt.C DynamicSlice.C EditorBar.C ellipse.C eval.C event.C exec1.C exec2.C exec3.C FeldmanCousins.C feynman.C fildir.C file.C fillrandom.C fillrandom.root first.C FirstContour.C fit1.C fit1_C.C fit2a.C fit2.C fit2d.C fitcont.C fitExclude.C fithist.C fitslicesy.C FittingDemo.C FittingDemo_C.d formula1.C framework.C galaxy_image.C galaxy.pal.root galaxy.root games.C gaxis.C gbasic.C geant3tasks.C geometry.C geoshapes.C gerrors2.C gerrors.C graph2derrorsfit.C graph2dfit.C graphApply.C graph.C greyscale.C guitest.C h1analysis.C h1analysis.h h1chain.C h1draw.C hadd.C hadd_old.C hbars.C hclient.C hcons.C hksimple.C hlabels1.C hlabels2.C hprod.C hserv2.C hserv.C hsimple.C hsimple.root hstack.C hsumanim.C hsum.C hsumTimer.C htest.C htmldoc htmlex.C Ifit.C img2pad.C imgconv.C io.C jets.C labels1.C labels2.C langaus.C latex2.C latex3.C latex.C LDAPExample.C limit.C logscales.C manyaxis.C markerwarning.C MDF.C miller.C mlpHiggs.C mlpHiggs.root motorcycle.C motorcycle.dat multidimfit.C multifit.C multigraph.C MviaS.C myfit.C MyTasks.cxx na49.C na49geomfile.C na49.root na49view.C na49visible.C ntuple1.C ntuple1_C.d oldbenchmarks.C pad2png.C ROOT Overview pclient.C peaks.C PhaseSpace.C principal.C pserv.C psexam.C pstable.C pythiaExample.C pythiaExample_C.d Quad.cxx Quad.h quantiles.C quarks.C Reset rootalias.C rootenv.C rootgeom.C rootlogoff.C rootlogon.C rootmarks.C rose512.jpg rose512.png rose512.tiff rose512.xpm rose_image.C runcatalog.sql runzdemo.C second.C seism.C shapesAnim.C shapes.C shared.C splines.C spy.C spyserv.C sqlcreatedb.C sqlfilldb.C sqlselect.C staff.C surfaces.C tasks.C T.C tcl.C TestAuth.C testrandom.C threads.C timeonaxis2.C timeonaxis.C tornado.C tree0.C tree1.C tree2a.C tree2.C tree3.C tree4.C tree.C triangles.C tv3.C tvdemo.C two.C twoscales.C waves.C work.C WorldMap.C worldmap.xpm xtruDraw.C xtruSamples.C xysliderAction.C xyslider.C zdemo.C zones.C 17 root.exe or root libs #include “TRint.h” int main(int argc, char **argv) { TRint theApp("Rint", &argc, argv); root.exe //enter the event loop... theApp.Run(); root > gSystem->Load(“libMyclasses”) root > Myclass a; return 0; root > a.DoSomething(..); myapp.exe } #include myclasses.h” #include “TApplication.h” int main(int argc, char **argv) { TApplication theApp(“MyApp", &argc, argv); MyClass a; a.DoSomething(); //enter the event loop... theApp.Run(); return 0; } R.Brun ROOT Overview 18 My first session root root [0] 344+76.8 (const double)4.20800000000000010e+002 root [1] float x=89.7; root [2] float y=567.8; root [3] x+sqrt(y) (double)1.13528550991510710e+002 root [4] float z = x+2*sqrt(y/6); root [5] z (float)1.09155929565429690e+002 root [6] .q root See file $HOME/.root_hist root [0] try up and down arrows R.Brun ROOT Overview 19 My second session root root [0] .x session2.C for N=100000, sum= 45908.6 root [1] sum (double)4.59085828512453370e+004 Root [2] r.Rndm() (Double_t)8.29029321670533560e-001 root [3] .q session2.C { int N = 100000; TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); unnamed macro executes in global scope } R.Brun ROOT Overview 20 My third session root root [0] .x session3.C for N=100000, sum= 45908.6 root [1] sum Error: Symbol sum is not defined in current scope *** Interpreter error recovered *** Root [2] .x session3.C(1000) for N=1000, sum= 460.311 root [3] .q session3.C Named macro Normal C++ scope rules R.Brun void session3 (int N=100000) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } ROOT Overview 21 My third session with ACLIC root [0] gROOT->Time(); root [1] .x session4.C(10000000) for N=10000000, sum= 4.59765e+006 Real time 0:00:06, CP time 6.890 root [2] .x session4.C+(10000000) for N=10000000, sum= 4.59765e+006 Real time 0:00:09, CP time 1.062 root [3] session4(10000000) for N=10000000, sum= 4.59765e+006 Real time 0:00:01, CP time 1.052 root [4] .q File session4.C Automatically compiled and linked by the native compiler. Must be C++ compliant R.Brun session4.C #include “TRandom.h” void session4 (int N) { TRandom r; double sum = 0; for (int i=0;i<N;i++) { sum += sin(r.Rndm()); } printf("for N=%d, sum= %g\n",N,sum); } ROOT Overview 22 Macros with more than one function root [0] .x session5.C >session5.log root [1] .q root [0] .L session5.C root [1] session5(100); >session5.log root [2] session5b(3) sum(0) = 0 sum(1) = 1 sum(2) = 3 root [3] .q .x session5.C executes the function session5 in session5.C use gROOT->ProcessLine to execute a macro from a macro or from compiled code R.Brun session5.C void session5(int N=100) { session5a(N); session5b(N); gROOT->ProcessLine(“.x session4.C+(1000)”); } void session5a(int N) { for (int i=0;i<N;i++) { printf("sqrt(%d) = %g\n",i,sqrt(i)); } } void session5b(int N) { double sum = 0; for (int i=0;i<N;i++) { sum += i; printf("sum(%d) = %g\n",i,sum); } } ROOT Overview 23 Root prompt At the Root prompt, you can execute a C/C++ statement a CINT command (start with “.”) .? Show list of CINT commands .q to quit the application .x file.C .L file.C .T trace mode for macros .!system_command, eg .!ls R.Brun ROOT Overview 24 Root in batch mode root –b –q session5.C root –b –q “session5.C(500)” root –b root [0] .x session5.C root [1] .q R.Brun ROOT Overview 25 Graphics ROOT Overview 26 Gui/Graphics interfaces High level pad graphics User/Root GUI and Graphics classes Applications see only Abstract Interfaces TVirtualPad Low level screen graphics and GUI TGWin32 R.Brun TVirtualX TGQt TG?? ROOT Overview TGX11 TPad TGWin32GDK 27 Coordinate Systems Pixel coordinates [0,32000] (TVirtualX) NDC (Normalized Device Coordinates) [0,1], (0,0) at bottom left corner System used for annotations Normal User coordinates (TVirtualPad) (0,0) at top left corner Linear/Log scale in x,y,z Can switch between lin/log scale without calling Draw again. TVirtualPad/TPad have functions to convert between systems R.Brun ROOT Overview 28 TPad: main graphics container Root > TLine line(.1,.9,.6,.6) Root > line.Draw() Root > TText text(.5,.2,”Hello”) Root > text.Draw() Hello The Draw function adds the object to the list of primitives of the current pad. If no pad exists, a pad is automatically created with a default range [0,1]. When the pad needs to be drawn or redrawn, the object Paint function is called. R.Brun ROOT Overview Only objects deriving from TObject may be drawn in a pad Root Objects or User objects 29 Basic Primitives TLine TArrow TEllipse TButton TBox TDiamond TCurvyLine TText TPave TMarker TPavesText TCrown TPaveLabel TCurlyArc TPolyLine TLatex R.Brun ROOT Overview 30 Graphics Editor 1-Toggle Editor mode 3-Edit object attributes 2-Click On the object to be edited R.Brun ROOT Overview 31 Gui Builder/Gui Code Generator Type CRTL/S anywhere on a widget To generate the C++ code for this widget Then do root > .x guisave.C R.Brun ROOT Overview 32 Canvas pixmaps Need Help? Scroll bars 1200x800 600x400 R.Brun ROOT Overview 33 Annotations, Paves quarks.C R.Brun ROOT Overview 34 Full LateX support on screen and postscript latex3.C Formula or diagrams can be edited with the mouse Feynman.C TCurlyArc TCurlyLine TWavyLine and other building blocks for Feynmann diagrams R.Brun ROOT Overview 35 Graphs TGraphErrors(n,x,y,ex,ey) TGraphAsymmErrors(n,x,y,exl,exh,eyl,eyh) gerrors2.C TGraph(n,x,y) TCutG(n,x,y) TMultiGraph TGraphBentErrors(n,x,y,exl,exh,eyl,eyh,exld,exhd,eyld,eyhd) R.Brun ROOT Overview 36 Graphs zdemo.C R.Brun ROOT Overview 37 Graphics : 1,2,3-D functions FittingDemo.C surfaces.C TF3 f3(“f3”,”sin(x*x+y*y+z*z-36)”,-2,2,-2,2,-2,2); f3->Draw(); R.Brun ROOT Overview 38 TGaxis gaxis.C R.Brun ROOT Overview 39 Adding dynamic objects to a pad A TExec object may be added to the pad. Its Paint function can call any CINT command. DynamicSlice.C R.Brun ROOT Overview 40 TGraph2D This class is a set of N points x[i],y[i],zi[i] in a non uniform grid. Several visualization techniques are implemented, including Delaunay triangulation. TCanvas c; Double_t x, y, z, P = 6.; Int_t np = 200; TGraph2D dt; TRandom r; for (Int_t N=0; N<np; N++) { x = 2*P*(r.Rndm(N))-P; y = 2*P*(r.Rndm(N))-P; z = (sin(x)/x)*(sin(y)/y)+0.2; dt.SetPoint(N,x,y,z); } gStyle->SetPalette(1); dt.Draw("surf1"); dt.Draw(“same p0”); R.Brun ROOT Overview 41 Using Timers with graphics Double_t pi = TMath::Pi(); TF2 *f2; Float_t t = 0; Float_t phi = 30; void anim() { gStyle->SetFrameFillColor(42); TCanvas *c1 = new TCanvas("c1"); c1->SetFillColor(17); f2 = new TF2("f2","sin(2*x)*sin(2*y)*[0]",0,pi,0,pi); f2->SetParameter(0,1); f2->SetNpx(15); f2->SetNpy(15); f2->SetMaximum(1); f2->SetMinimum(-1); f2->Draw("surf1"); TTimer *timer = new TTimer(20); timer->SetCommand("Animate()"); timer->TurnOn(); } void Animate() { t += 0.05*pi; f2->SetParameter(0,TMath::Cos(t)); phi += 2; gPad->SetPhi(phi); gPad->Modified(); gPad->Update(); } R.Brun anim.C Every 20 milliseconds: -Modify the function parameter -Change the view angle ROOT Overview 42 Picking- Graphics event loop TCanvas::HandleInput manages the event loop. When the mouse moves to a pad, TPad::Pick is called. TPad::Pick loops on all primitives in the pad, calling their function DistancetoPrimitive. The selected object is the one with the smallest distance. Activate the tool bar option “ShowEventStatus” to understand the behaviour. R.Brun ROOT Overview 43 Picking- Graphics event loop When an object has been selected by DistancetoPrimitive, its ExecuteEvent function is called. The result of ExecuteEvent is object dependent. The cursor changes shape to suggest possible actions : grow/shrink/move box, rotation, etc. Pressing the right mouse button on an object shows its context menu. R.Brun ROOT Overview 44 The Graphics Event Loop The ROOT event handler supports: Foreign systems (eg Inventor, X3d..) can easily be integrated in the ROOT loop. keyboard interrupts system signals X11, Xt, Xm events Sockets interrupts Special messages (shared memory, threads..) ROOT will dispatch the foreign events using Timers. Signals and Slots like in Qt R.Brun Qt and ROOT can work together ROOT Overview 45 GUI User example Example of GUI based on ROOT tools Each element is clickable R.Brun ROOT Overview 46 GUI Examples R.Brun ROOT Overview 47 Graphical User Interface Ex: ZEUS event display Input modes Option tabs Status information R.Brun Canvas Pads ROOT Overview Zoom controls Object information 48 TASImage classes New set of image processing classes. The TImage class is the abstract image base class and TASImage is the concrete implementation using the libAfterImage imaging library of Sasha Vasko <sasha@aftercode.net>. A large part of the development was done by Reiner Rohlfs from the ISDC based on a set of astrophysics user requirements. The image class allows for the reading and writing of images in different formats, several image manipulations (scaling, tiling, merging, etc.) and displaying in pads. The size of the image on the screen does not depend on the original size of the image but on the size of the pad.Therefore it is very easy to resize the image on the screen by resizing the pad. Galaxy_image.C Rose_image.C R.Brun ROOT Overview 49 3-D Graphics Basic primitives Geant primitives TPolyLine3D, TPolyMarker3D, THelix, TMarker3DBox,TAxis3D Support for all Geant3 volumes + a few new volume types TBRIK,TCONE,TCONS,TCTUB,TELTU,TGTRA,THYPE,TPARA,TPCON, TPGON,TSPHE,TTUBE,TTUBS,TTRAP,TTRD1,TTRD2,TXTRU Geometry package Rendering with: R.Brun TPad X3D (very fast. Unix only. Good on networks) OpenGL OpenInventor (new addition in 3.01) ROOT Overview 50 Some detectors in ROOT geometry R.Brun ROOT Overview 51 Alice R.Brun 3 million nodes ROOT Overview 52 Input/Output ROOT Overview 53 ROOT + RDBMS Model ROOT files Oracle MySQL Calibrations Event Store histograms Run/File Catalog Trees Geometries R.Brun ROOT Overview 54 How Much Data is Involved? Level 1 Rate (Hz) 106 High Level-1 Trigger (1 MHz) LHCB 105 1 billion people surfing the Web 104 HERA-B ATLAS CMS KLOE CDF II CDF 103 102 104 R.Brun High No. Channels High Bandwidth (500 Gbit/s) High Data Archive (5 PetaBytes/year) 10 Gbits/s in Data base H1 ZEUS NA49 UA1 LEP 105 106 ALICE STAR 107 Event Size (bytes) ROOT Overview 55 Object Persistency (in a nutshell) Two I/O modes supported (Keys and Trees). Key access: simple object streaming mode. A ROOT file is like a Unix directory tree Very convenient for objects like histograms, geometries, mag.field, calibrations Trees A generalization of ntuples to objects Designed for storing events split and no split modes query processor Chains: Collections of files containing Trees ROOT files are self-describing Interfaces with RDBMS also available Access to remote files (RFIO, DCACHE, GRID) ROOT Overview 56 R.Brun ROOT I/O : An Example Program Writing demoh.C TFile f(“example.root”,”new”); TH1F h(“h”,”My histogram”,100,-3,3); h.FillRandom(“gaus”,5000); h.Write(); Program Reading TFile f(“example.root”); demohr.C TH1F *h = (TH1F*)f.Get(“h”): h->Draw(); f.Map(); 20010831/171903 20010831/171941 20010831/171946 20010831/171946 20010831/171946 20010831/171946 R.Brun At:64 At:154 At:607 At:2971 At:3067 At:3123 N=90 N=453 N=2364 N=96 N=56 N=1 TFile TH1F StreamerInfo KeysList FreeSegments END CX = CX = ROOT Overview 2.09 3.25 57 A Root file pippa.root with two levels of directories R.Brun ROOT Overview Objects in directory /pippa/DM/CJ eg: /pippa/DM/CJ/h15 58 Root objects or any User Object can be stored in ROOT folders and browsed R.Brun ROOT Overview 59 LAN/WAN files Files and Directories a directory holds a list of named objects a file may have a hierarchy of directories (a la Unix) ROOT files are machine independent built-in compression Local file Remote file access via a Web server Support for local, LAN and WAN files TFile f1("myfile.root") TFile f2("http://pcbrun.cern.ch/Renefile.root") TFile f3("root://cdfsga.fnal.gov/bigfile.root") TFile f4("rfio://alice/run678.root") R.Brun ROOT Overview Remote file access via the ROOT daemon Access to a file on a mass store hpps, castor, via RFIO 60 Streaming Objects ROOT Overview 61 ROOT I/O -- Sequential/Flat Object Object in Object in Object memory in Object memory in memory in memory memory Streamer Transient Object is serialized by the Streamer No need for transient/persistent classes TMapFile shared memory http TWebFile web server sockets TRFIOFile RFIO daemon TNetFile rootd TFile R.Brun ROOT Overview 62 Rootcint Preprocessor UserClass1.h UserCint.C UserClass1.h UserClass1.h UserClass1.h UserClass1.h UserClass1.h C++ code to create the RTTI rootcint Interface for CINT interpreter Streamers R.Brun ROOT Overview 63 Automatic Schema Evolution R.Brun ROOT Overview 64 Auto Schema Evolution (2) R.Brun ROOT Overview 65 Normal Streaming mode References using C++ pointers Only one copy of each object A in the graph saved to buffer TBuffer b; A.Streamer(b) R.Brun ROOT Overview 66 Normal Streaming mode References using C++ pointers TBuffer b1; A.Streamer(b1) Objects in red TBuffer b2; are in b1 and b2 B.Streamer(b2) C++ pointer B A R.Brun ROOT Overview 67 Normal Streaming mode References using TRef pointers TBuffer b1; A.Streamer(b1) TBuffer b2; Objects in blue B.Streamer(b2) are only in b1 C++ pointer A Bz B TRef z Set pointer to z with: TRef Bz = z; Get pointer to z with: z = Bz.GetObject() R.Brun ROOT Overview 68 TRees ROOT Overview 69 Memory <--> Tree Each Node is a branch in the Tree Memory 0 T.GetEntry(6) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 T.Fill() T R.Brun ROOT Overview 70 Tree Creation Example A few lines of code to create a Tree for structures that may be very complex R.Brun ROOT Overview 71 8 leaves of branch Electrons A double-click to histogram the leaf 8 Branches of T R.Brun ROOT Overview 72 19 leaves in branch fDele 36 branches in Tree T R.Brun ROOT Overview 73 The Tree Viewer & Analyzer A very powerful class supporting complex cuts, event lists, 1-d,2-d, 3-d views parallelism R.Brun ROOT Overview 74 Tree Friends 01 23 45 67 89 10 11 12 13 14 15 16 17 18 R.Brun 0 1 2 3 4 5 6 7 8 Entry # 8 01 23 45 67 89 10 11 12 13 14 15 16 17 18 9 10 11 12 13 14 15 16 17 18 Public Public User read read Write ROOT Overview 75 Tree Friends Collaboration-wide public read Analysis group protected user private Processing time independent of the number of friends unlike table joins in RDBMS x Root > TFile f1(“tree1.root”); Root > tree.AddFriend(“tree2”,“tree2.root”) Root > tree.AddFriend(“tree3”,“tree3.root”); Root > tree.Draw(“x:a”,”k<c”); Root > tree.Draw(“x:tree2.x”,”sqrt(p)<b”); R.Brun ROOT Overview 76 Chains of Trees A TChain is a collection of Trees. Same semantics for TChains and TTrees root > .x h1chain.C root > chain.Process(“h1analysis.C”) { //creates a TChain to be used by the h1analysis.C class //the symbol H1 must point to a directory where the H1 data sets //have been installed TChain chain("h42"); chain.Add("$H1/dstarmb.root"); chain.Add("$H1/dstarp1a.root"); chain.Add("$H1/dstarp1b.root"); chain.Add("$H1/dstarp2.root"); } R.Brun ROOT Overview 77 The Event class class Event : public TObject { private: char Int_t Int_t Int_t UInt_t Float_t Int_t Float_t Float_t EventHeader TClonesArray TRefArray TRefArray TRef TH1F fType[20]; fNtrack; fNseg; fNvertex; fFlag; fTemperature; fMeasures[10]; fMatrix[4][4]; *fClosestDistance; fEvtHdr; *fTracks; *fHighPt; *fMuons; fLastTrack; *fH; //event type //Number of tracks //Number of track segments //[fNvertex] //->array with all tracks //array of High Pt tracks only //array of Muon tracks only //reference pointer to last track //-> class EventHeader { private: Int_t Int_t Int_t R.Brun fEvtNum; fRun; fDate; See $ROOTSYS/test/Event.h ROOT Overview 78 The Track class class Track : public TObject { private: Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Float_t Int_t Short_t R.Brun fPx; fPy; fPz; fRandom; fMass2; fBx; fBy; fMeanCharge; fXfirst; fXlast; fYfirst; fYlast; fZfirst; fZlast; fCharge; fVertex[3]; fNpoint; fValid; //X component of the momentum //Y component of the momentum //Z component of the momentum //A random track quantity //The mass square of this particle //X intercept at the vertex //Y intercept at the vertex //Mean charge deposition of all hits //X coordinate of the first point //X coordinate of the last point //Y coordinate of the first point //Y coordinate of the last point //Z coordinate of the first point //Z coordinate of the last point //Charge of this track //Track vertex position //Number of points for this track //Validity criterion ROOT Overview 79 Event Builder void Event::Build(Int_t ev, Int_ntrack, Float_t ptmin) { Clear(); ……….. for (Int_t t = 0; t < ntrack; t++) AddTrack(random,ptmin); } Track *Event::AddTrack(Float_t random, Float_t ptmin) { // // // // // Add a new track to the list of tracks for this event. To avoid calling the very time consuming operator new for each track, the standard but not well know C++ operator "new with placement" is called. If tracks[i] is 0, a new Track object will be created otherwise the previous Track[i] will be overwritten. TClonesArray &tracks = *fTracks; Track *track = new(tracks[fNtrack++]) Track(random); //Save reference to last Track in the collection of Tracks fLastTrack = track; //Save reference in fHighPt if track is a high Pt track if (track->GetPt() > ptmin) fHighPt->Add(track); //Save reference in fMuons if track is a muon candidate if (track->GetMass2() < 0.11) fMuons->Add(track); return track; } R.Brun ROOT Overview 80 Tree example Event (write) void demoe(int nevents) { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //create a new ROOT file TFile f("demoe.root",”new"); All the examples can be executed with CINT or the compiler root > .x demoe.C root > .x demoe.C++ //Create a ROOT Tree with one single top level branch int split = 99; //try also split=1 and split=0 int bufsize = 16000; Event *event = new Event; TTree T("T","Event demo tree"); T.Branch("event","Event",&event,bufsize,split); //Build Event in a loop and fill the Tree for (int i=0;i<nevents;i++) { event->Build(i); T.Fill(); } T.Print(); T.Write(); //Print Tree statistics //Write Tree header to the file } R.Brun ROOT Overview 81 Tree example Event (read 1) void demoer() { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //connect ROOT file TFile *f = new TFile("demoe.root"); //Read Tree header and set top branch address Event *event = 0; TTree *T = (TTree*)f->Get("T"); T->SetBranchAddress("event",&event); //Loop on events and fill an histogram TH1F *h = new TH1F("hntrack","Number of tracks",100,580,620); int nevents = (int)T->GetEntries(); for (int i=0;i<nevents;i++) { T->GetEntry(i); Rebuild the full event h->Fill(event->GetNtrack()); in memory } h->Draw(); } R.Brun ROOT Overview 82 Tree example Event (read 2) void demoer2() { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //connect ROOT file TFile *f = new TFile("demoe.root"); //Read Tree header and set top branch address Event *event = 0; TTree *T = (TTree*)f->Get("T"); T->SetBranchAddress("event",&event); Tbranch *bntrack = T->GetBranch(“fNtrack”); //Loop on events and fill an histogram TH1F *h = new TH1F("hntrack","Number of tracks",100,580,620); int nevents = (int)T->GetEntries(); for (int i=0;i<nevents;i++) { bntrack->GetEntry(i); Read only h->Fill(event->GetNtrack()); one branch } Much faster ! h->Draw(); } R.Brun ROOT Overview 83 Tree example Event (read 3) void demoer3() { //load shared lib with the Event class gSystem->Load("$ROOTSYS/test/libEvent"); //connect ROOT file TFile *f = new TFile("demoe.root"); //Read Tree header TTree *T = (TTree*)f->Get("T"); //Histogram number of tracks via the TreePlayer T->Draw(“event->GetNtrack()”); } R.Brun ROOT Overview 84 GRIDs and PROOF ROOT Overview 85 Data Volume & Organisation 100MB 1 1GB 1 10GB 100GB 5 50 1TB 500 10TB 100TB 5000 50000 1PB TTree TChain A TFile typically contains 1 TTree A TChain is a collection of TTrees or/and TChains A TChain is typically the result of a query to the file catalogue R.Brun ROOT Overview 86 Data Volume & Processing Time Using technology available in 2003 100MB 1GB 10GB 100GB 1TB 10TB 100TB 1PB ROOT 1 Processor P IV 2.4GHz 2003 : Time for one query using 10 per cent of data 1” 10” 1’ 10’ 1h 10h 1day 1month batch Interactive PROOF 10 Processors 1” 1” 10” 1’ 10’ 1h 10h 1day 1” 10” 1’ 10’ 1h 10h 10days PROOF 100Processors 1” 1” 1day PROOF/ALIEN 1000Processors 1’ R.Brun ROOT Overview 10’ 1h 10h 87 Data Volume & Processing Time Using technology available in 2010 100MB 1GB 10GB 100GB 1TB 10TB 100TB 1PB 10h 1day 10days ROOT 1 Processor XXXXX 2010 : Time for one query using 10 per cent of data 1” 1” 10” 1’ 10’ 1h batch Interactive PROOF 10 Processors 1” 1” 1” 10” 1’ 10’ 1h 10h 1day 1” 1” 10” 1’ 10’ 1h 10h PROOF 100Processors 1” 1” PROOF/ALIEN 1000Processors 1’ R.Brun ROOT Overview 10’ 1h 88 Interactive Local Analysis On a public cluster, or the user’s laptop. Tools like ROOT are used for visualization and ntuples/trees analysis. R.Brun ROOT Overview 89 GRID: Interactive Analysis Case 1 Data transfer to user’s laptop Optional Run/File catalog Optional GRID software Optional run/File Catalog Analysis scripts are interpreted or compiled on the local machine Trees Remote file server eg rootd Trees R.Brun ROOT Overview 90 GRID: Interactive Analysis Case 2 Remote data processing Optional Run/File catalog Optional GRID software Optional run/File Catalog Analysis scripts are interpreted or compiled on the remote machine Trees Commands, scripts histograms R.Brun Remote data analyzer eg proofd ROOT Overview Trees 91 GRID: Interactive Analysis Case 3 Remote data processing Run/File catalog Full GRID software Analysis scripts are interpreted or compiled on the remote master(s) Run/File Catalog slave slave Commands, scripts Histograms,trees Remote data analyzer eg proofd slave slave slave slave R.Brun ROOT Overview Trees Trees Trees Trees Trees Trees Trees Trees 92 PROOF and GRIDs The PROOF system allows parallel processing of chains of trees on clusters of heterogeneous machines. Its main features are: Transparency, scalability, adaptivity A first prototype developed in 1997 as proof of concept (only for simple queries resulting in 1D histograms) We are now implementing the system taking into the most recent developments in the GRID middleware. R.Brun ROOT Overview 93 #proof.conf slave node1 slave node2 slave node3 slave node4 Remote PROOF Parallel Script Execution Local PC root stdout/obj ana.C proof proof node1 Cluster *.root TFile ana.C proof node2 $ root root [0] .x ana.C root [1] gROOT->Proof(“remote”) root [2] gProof->Exec(“.x ana.C”) proof = master server proof = slave server *.root TNetFile proof *.root node3 proof *.root node4 R.Brun ROOT Overview 94 Running a PROOF Job // Analyze TChains in parallel gROOT->Proof(); TChain *chain = new TChain(“AOD"); chain->Add("lfn://alien.cern.ch/alice/prod2002/P2001*"); . . . chain->Process(“myselector.C”); // Analyze generic data sets in parallel gROOT->Proof(); TDSet *objset = new TDSet("MyEvent", "*", "/events"); objset->Add("lfn://alien.cern.ch/alice/prod2002/file1"); . . . objset->Add(set2003); objset->Process(“myselector.C++”); R.Brun ROOT Overview 95 PROOF Demo at ROOT workshop Client machine PIII @ 1GHz / 512 MB Standard IDE disk Cluster with 15 nodes at CERN Dual PIII @ 800 MHz / 384 MB Standard IDE disk Cluster with 4 nodes at MIT Dual AthlonMP @ 1.4GHz / 1GB Standard IDE disk R.Brun ROOT Overview 96 ALICE GRID resources http://www.to.infn.it/activities/experiments/alice-grid OSU/OSC LBL/NERSC Dubna Birmingham NIKHEF Saclay GSI Nantes CERN Merida Padova Bologna Lyon IRB Torino 37 people 21 insitutions Bari Cagliari Yerevan Catania Kolkata, India Capetown, ZA R.Brun ROOT Overview 97 The Histogram Package ROOT Overview 98 The Histogram Classes Structure 1-Dim 2-Dim 3-Dim R.Brun ROOT Overview 99 Filling Histograms An histogram is typically filled with statements like: h1->Fill(x); h1->Fill(x,w); //fill with weight h2->Fill(x,y) h2->Fill(x,y,w) h3->Fill(x,y,z) h3->Fill(x,y,z,w) h1->Fill(string) The Fill functions return the bin number for 1-D histograms or global bin number for 2-D and 3-D histograms. If TH1::Sumw2 has been called before filling, the sum of squares of weights is also stored. One can also increment directly a bin number via TH1::AddBinContent or replace the existing content via TH1::SetBinContent. To access the bin content of a given bin, do: Double_t binContent = h->GetBinContent(bin); R.Brun ROOT Overview 100 Automatic binning #include "TH1.h" #include "TF1.h" No limits void demoauto() { TF1 *f1 = new TF1("f1","gaus",0,30); f1->SetParameters(1,10,3); TH1F *h = new TH1F("h","Automatic binning demo",100,0,0); for (Int_t i=0;i<1000000;i++) { h->Fill(f1->GetRandom()); } h->Draw(); } R.Brun ROOT Overview 101 Histograms New class THStack Long list of new functions in TH1 Plenty of new drawing options Filling with string variables TH1::Merge(TCollection *list) R.Brun ROOT Overview 102 Peak Finder + Deconvolutions by Miroslav Morach TSpectrum R.Brun ROOT Overview 103 Fitting TMinuit TFumili TFitter TVirtualFitter TH1::Fit TGraph::Fit TTree::Fit bin chisquare bin likelihood unbinned chisquare unbinned likelihood User R.Brun ROOT Overview 104 Fitting histograms Histograms (1-D,2-D,3-D and Profiles) can be fitted with a user specified function via TH1::Fit. Two Fitting algorithms are supported: Chisquare method and Log Likelihood The user functions may be of the following types: standard functions: gaus, landau, expo, poln combination of standard functions; poln + gaus A C++ interpreted function or a C++ precompiled function An option is provided to compute the integral of the function bin by bin instead of simply compute the function value at the center of the bin. When an histogram is fitted, the resulting function with its parameters is added to the list of functions of this histogram. If the histogram is made persistent, the list of associated functions is also persistent. One can retrieve the function/fit parameters with calls such as: R.Brun Double_t chi2 = myfunc->GetChisquare(); Double_t par0 = myfunc->GetParameter(0); //value of 1st parameter Double_t err0 = myfunc->GetParError(0); //error on first parameter ROOT Overview 105 Associated functions One or more object (typically a TF1*) can be added to the list of functions associated to each histogram. When TF1::Fit is invoked, the fitted function is added to this list. Given an histogram h, one can retrieve an associated function with: R.Brun TF1 *myfunc = h->GetFunction("myfunc"); ROOT Overview 106 Fitting/TMinuit/TFumili Choose Fitter at run time or via system.rootrc void FitBench(Int_t npass=100) { TCanvas *c1 = new TCanvas("c1"); c1->Divide(1,2); fitFcn = new TF1("fitFcn",fitFunction,0,3,6); fitFcn->SetNpx(200); gStyle->SetOptFit(); gStyle->SetStatY(0.6); //with Minuit c1->cd(1); TVirtualFitter::SetDefaultFitter("Minuit"); DoFit(npass); //with Fumili c1->cd(2); TVirtualFitter::SetDefaultFitter("Fumili"); DoFit("Fumili",npass); } R.Brun ROOT Overview 107 RooFit A general purpose tool kit for data modeling Wouter Verkerke (UC Santa Barbara) David Kirkby (UC Irvine) ftp://root.cern.ch/root/R2002/verkerke.ppt ROOT Overview 108 Operations on histograms Many types of operations are supported on histograms or between histograms Addition of an histogram to the current histogram Additions of two histograms with coefficients and storage into the current histogram Multiplications and Divisions are supported in the same way as additions. The Add, Divide and Multiply functions also exist to add,divide or multiply an histogram by a function. If an histogram has associated error bars (TH1::Sumw2 has been called), the resulting error bars are also computed assuming independent histograms. In case of divisions, Binomial errors are also supported. The standard operators +, -, *, / are supported. R.Brun ROOT Overview 109 Random Numbers and Histograms TH1::FillRandom can be used to randomly fill an histogram using the contents of an existing TF1 analytic function For example the following two statements create and fill an histogram 10000 times with a default gaussian distribution of mean 0 and sigma 1: another histogram (for all dimensions). TH1F h1("h1","histo from a gaussian",100,-3,3); h1.FillRandom("gaus",10000); TH1::GetRandom can be used to return a random number distributed according the contents of an histogram. R.Brun ROOT Overview 110 Drawing Histograms When you call the Draw method of a histogram for the first time (TH1::Draw), it creates a THistPainter object and saves a pointer to painter as a data member of the histogram. The THistPainter class specializes in the drawing of histograms. It is separate from the histogram so that one can have histograms without the graphics overhead, for example in a batch program. The choice to give each histogram have its own painter rather than a central singleton painter, allows two histograms to be drawn in two threads without overwriting the painter's values. When a displayed histogram is filled again you do not have to call the Draw method again. The image is refreshed the next time the pad is updated. The same histogram can be drawn with different graphics options in different pads. When a displayed histogram is deleted, its image is automatically removed from the pad. R.Brun ROOT Overview 111 1-D drawing Options Any object in the canvas is clickable and editable R.Brun ROOT Overview 112 R.Brun ROOT Overview 113 2-D drawing options R.Brun ROOT Overview 114 2-D drawing options R.Brun ROOT Overview 115 2-D drawing Options All these plots can be rotated with the mouse R.Brun ROOT Overview 116 2-D drawing Options Same output on the screen and with vector Postscript R.Brun ROOT Overview 117 THStack examples R.Brun ROOT Overview 118 Filling with string variables R.Brun ROOT Overview 119 Math Libs & Statistics Can generate random numbers In ROOT today TVector2,3 TLorentzRotation from basic distributions; gaus, poisson, etc from parametric analytic functions 1,2,3-d from histograms, 1,2,3-d TLorentzVector TRandom,2,3 TMatrix Linear Algebra package TMath TFeldmanCousins TPrincipal A collection of many algorithms TRolke CERNLIB, Numerical Recipes in C/C++ TMultidimFit TMultiLayerPerceptron TConfidenceLevel TFractionFitter R.Brun ROOT Overview 120 TMath (1) R.Brun ROOT Overview 121 TMath (2) R.Brun ROOT Overview 122 New Linear Algebra classes Eddy Offermann has kindly maintained the old TMatrix, TVector classes for a few years. Eddy is now spending one year with us at CERN.He has developed a new set of classes with more functionality and better performance. See Eddy’s talk on Friday TDecompBase TDecompSVD TMatrixDCramerInv TMatrixDSymEigen TMatrixFCramerInv TVector R.Brun TDecompChol TMatrix TMatrixDEigen TMatrixDUtils TMatrixFLazy TVectorD TDecompLU TMatrixD TMatrixDLazy TMatrixF TMatrixFSym TVectorF ROOT Overview TDecompQRH TMatrixDBase TMatrixDSym TMatrixFBase TMatrixFUtils 123 Linear Algebra classes R.Brun ROOT Overview 124 Kolmogorov Test Double_t TH1::KolmogorovTest(TH1 *h2, Option_t *option) const { // Statistical test of compatibility in shape between // THIS histogram and h2, using Kolmogorov test. // Default: Ignore under- and overflow bins in comparison // // option is a character string to specify options // "U" include Underflows in test (also for 2-dim) // "O" include Overflows (also valid for 2-dim) // "N" include comparison of normalizations // "D" Put out a line of "Debug" printout // "M" Return the Maximum Kolmogorov distance instead of prob // "X" Run the pseudo experiments post-processor with the following procedure: // make pseudoexperiments based on random values from the parent // distribution and compare the KS distance of the pseudoexperiment // to the parent distribution. Bin the KS distances in a histogram, // and then take the integral of all the KS values above the value // obtained from the original data to Monte Carlo distribution. // The number of pseudo-experiments NEXPT is currently fixed at 1000. // The function returns the integral. // (thanks to Ben Kilminster to submit this procedure). Note that // this option "X" is much slower. // // The returned function value is the probability of test // (much less than one means NOT compatible) R.Brun ROOT Overview 125 TMultiLayerPerceptron package (see Christophe Delaere talk) New MultiLayerPerceptron (mlp) package, by Christophe Delaere. The package has 3 classes TMultiLayerPerceptron, TNeuron, TSynapse These classes describe a neural network. There are facilities to train the network and use the output. The input layer is made of inactive neurons (returning the normalized input), hidden layers are made of sigmoids and output neurons are linear. The basic input is a TTree and two (training and test) TEventLists. For classification jobs, a branch (maybe in a TFriend) must contain the expected output. 6 learning methods are available: kStochastic, kBatch, kSteepestDescent, kRibierePolak, kFletcherReeves and kBFGS. This implementation is *inspired* from the mlpfit package from J.Schwindling et al. R.Brun ROOT Overview 126 New class TRolke by Jan Conrad This class computes confidence intervals for the rate of a Poisson in the presence of background and efficiency with a fully frequentist treatment of the uncertainties in the efficiency and background estimate using the profile likelihood method. The signal is always assumed to be Poisson. It allows the following Models: 1: 2: 3: 4: 5: 6: 7: Background - known, Background Background Background Background Background Background - Poisson, Efficiency - Binomial (cl,x,y,z,tau,m) Poisson, Efficiency - Gaussian (cl,xd,y,em,tau,sde) Gaussian, Efficiency - Gaussian (cl,x,bm,em,sd) Poisson, Efficiency - known (cl,x,y,tau,e) Gaussian, Efficiency - known (cl,x,y,z,sdb,e) known, Efficiency - Binomial (cl,x,z,m,b) Efficiency - Gaussian (cl,x,em,sde,b) See example in tutorial Rolke.C R.Brun ROOT Overview 127 TStatUtils: Statistics by Christian Stratowa In order to demonstrate the applicability of ROOT to microarray data, I will present a functional prototype system, called XPS - eXpression Profiling System, which can be considered to be an alternative to the Bioconductor project. The current implementation handles the storage of GeneChip schemes and data, and the pre-processing, normalization and filtering of GeneChip data. Based on this system, I will propose a novel standard for the distributed storage of microarray data. Finally, I will emphasize the similarities between R and ROOT, and show how R could be easily extended to access ROOT from within R. Proceedings of DSC 2003 R.Brun ROOT Overview 128 TStatUtils: Statistics by Christian Stratowa R.Brun ROOT Overview 129