Planning for the SLAC ATLAS Program
D. MacFarlane
Deputy ALD for PPA
Planning for the SLAC ATLAS Program
Page 1
Background to the planning process
• SLAC is a relative newcomer to ATLAS, formally joining the
collaboration in July 2006
– Overall effort was about 15 FTEs in FY08, including faculty and staff
scientists, professional staff, postdocs, and graduate students
• Curtailment of the B Factory program has accelerated the timeline for
evaluation areas of engagement on ATLAS, including upgrade R&D
– ATLAS is the highest priority accelerator-based particle physics program at
SLAC going forward, driven by our assessment of the physics opportunities
and the national priority enunciated by P-5 and HEPAP
• Discussions with OHEP in May led to the proposal that SLAC prepare a
white paper with plans for this expanded ATLAS program
– We concluded that it would be best to investigate opportunities in detail,
anticipating that the subsequent steps would thereby be better informed
Planning for the SLAC ATLAS Program
Page 2
The ATLAS White Paper
• A White Paper has been developed to define in concrete terms the
areas we see as a good match to our interests and expertise
– Our approach has been to work bottoms up to understand specific needs,
future opportunities & our ability to contribute with effectiveness and impact
– This process has also allowed us to define the potential interest within
laboratory staff, so that we now have a concrete model for the individuals,
as well as their skills and interests
– Our approach has allowed us to make an independent assessment of
opportunities and ATLAS needs
• We have used the White Paper as the basis for discussion with US
ATLAS management in formulating an overall strategy with an
integrated view of SLAC’s role
– Discussions with US ATLAS management occurred on October 21,
November 6, and November 18
– OHEP briefing occurred on October 8
– US ATLAS management, OHEP program managers, and SLAC will meet
again in Germantown on January 7 to settle on a plan for FY09
Planning for the SLAC ATLAS Program
Page 3
Potential impact on US ATLAS science & capability
• If the White Paper plan were implemented, the SLAC ATLAS Program
would represent the largest single institutional effort in the US ATLAS
community
– Further growth would be anticipated as new people joining in FY09 ramp up
to full time
– A Western Data Analysis Facility, although not yet proposed or incorporated
in the plan, could bring even more growth
– However, with the exception of construction funding for future upgrade
projects, this plan fits within a constant level of effort model for the SLAC
PPA program
• A large infusion from SLAC offers some real opportunities to further
strengthen the US ATLAS community
– US ATLAS is geographically dispersed with important eastern & western
communities that would be well served by strong local centers
– SLAC team has the depth & core capability to sustain major hardware
system operations & upgrades, ATLAS computing & analysis support, while
aiming to be a vibrant role in the US ATLAS physics community
Planning for the SLAC ATLAS Program
Page 4
Organizing themes for the white paper
Theme 1: Existing & future upgrades for the Pixel System
Sections I.A, III.A, III.B, III.C, III.D, III.E, & III.G
Theme 2: Existing & future upgrades for DAQ and trigger
Sections I.B, & III.H
Theme 3: Simulations for the existing detector and upgrades
Sections I.E, & III.F
Theme 4: Tier 2 Center & potential Western Data Analysis
Facility
Section II
Theme 5: SLAC and the Bay Area as a physics center
Section I.D, I.F, & IV
Planning for the SLAC ATLAS Program
Page 5
Theme 1: Pixel System and tracking
• Current ATLAS effort: Pixel System operation
– In partnership with LBNL, we have a significant and growing role on the
present Pixel System; engaged in commissioning, calibration, and now
extending broadly into understanding and improving the performance of the
tracking system as a whole
– More work in this area remains, particularly in the next few years, to fully
understand and optimize the capabilities of this system
• Upgrade effort: Insertable B-Layer (IBL) project
– Pixel inner B-layer will reach its radiation lifetime after a few years at 1034,
with extended 2012-2013 shutdown as the only window for intervention
– ATLAS is moving rapidly towards mounting a project to construct an
insertable B layer (IBL) on this timescale, since replacement is not practical
– Significant technical challenges need to be overcome for the compressed
schedule, suggesting an enhanced US effort will be essential for success
– Working with LBNL and UCSC to understand what role is both appropriate
and sustainable for the US, e.g., end-of-stave to ROD services design and
implementation, with SLAC as a significant partner in formulating a plan
Planning for the SLAC ATLAS Program
Page 6
First look at IBL installation & services
M.Oriunno, N.Karita
ATLAS tracker upgrade meeting
NIKHEF, Nov 5-7, 2008
PP0
Enlarged PP1 area
PP1
3650 mm
700 mm
Mounting collar at PP0
Planning for the SLAC ATLAS Program
Page 7
Theme 1: Pixel System and tracking
• Upgrade effort: replacement Pixel System
– Approval of Stage 1 upgrade to the LHC, with a goal of 3x1034 luminosity in
2013-2017, necessitates upgrade of the existing Pixel System by about
2017 (after ~300 fb-1)
– Subject of recent pre-CD-0 JOG review, where significant questions arose
about the time frame needed to construct the replacement system
– We believe that such an upgrade is inevitable and are working closely with
LBNL and UCSC, and ATLAS as a whole, to develop a more detailed plan
• Upgrade R&D
– Engaging in helping ATLAS reach conclusions about important technical
decisions, e.g., CO2 cooling, electrical design for prototype staves, data
transmission & 3d pixel sensors
– Activities such as CO2 cooling anticipate, along with a new test stand, some
of the infrastructure needed to sustain a major pixel test and assembly effort
Planning for the SLAC ATLAS Program
Page 8
Phase 1 blown CO2 system under test
M.Oriunno
ATLAS tracker upgrade meeting
NIKHEF, Nov 5-7, 2008
Planning for the SLAC ATLAS Program
Page 9
Theme 2: DAQ and HLT
• Current ATLAS effort: DAQ and HLT
– System for prompt configuration of the HLT farm
– Collaborating with CERN IT on CORAL server project for technologyindependent database access
– Development of a partial event build capability for calibration data
– HLT commissioning and trigger algorithm development
– Development of online beam spot monitoring and distribution
– Support for UCI and commissioning of the CSC ROD system
• Upgrade effort: some initial directions for DAQ and HLT
– Will likely be more adiabatic effort, so only initial ideas discussed
– Develop a HLT performance improvement program based on performance
monitoring and analysis capability (found to be very valuable at BABAR),
and improved parallelism through data sharing
– Explore adapting generic ATCA-based DAQ platform to provide a highperformance integrated solution to DAQ requirements for ATLAS at the
sLHC; may also provide an opportunity for a Level 1.5 trigger capability
Planning for the SLAC ATLAS Program
Page 10
Possible upgrade path for ATLAS TDAQ
To L2 & Event Building
(X12)
10 gb/s
ATCA crate with ROD & CIM
Rear Transition Module
Rear Transition Module
P3
10-GE
switch
CIM
L1 fanout
switch management
P3
10-GE
switch
10-GE
switch
836 RODs
1574 ROLs
537 ROBIns
68 PCs
Backplane
ROMs
(X2)
10 gb/s
25 crates
(X2)
Would greatly simplify current
2.5 gb/s system and
Shelf
Management
Planning for the SLAC
ATLAS
scales
toProgram
meet 1035 requirements
M.Huffer, G.Haller, R.Bartoldus, Su Dong
ATLAS TDAQ upgrade proposal
from L1
Page 11
To monitoring & control
CIM
235 ROMs
45 CIMs
76 crates
Read-Out Module
10-GE
L1 fanout
Cluster-Interconnect
Module
switch
switch management
from L1
Theme 3: Simulations
• Current ATLAS effort: GEANT4 core support and performance
enhancement
– Have emphasized speed optimization of the current simulation,
contributions to core muon simulation, optimization of field integration,
physics validation, and other improvements to the code
– Contributions to the development of the capability for event overlay and
zero-bias background event sampling
– Engaging at leadership level in next steps in performance improvement
• Upgrade effort: Pixel and Tracking Systems layout studies
– ATLAS is proceeding quickly to make basic layout design decisions for the
future Pixel and Silicon Tracker Systems
– We propose to use existing tools, either within the ATLAS simulation tool kit
or in conjunction with the linear collider framework LCsim, to provide
physics and performance guidance to these design decisions
Planning for the SLAC ATLAS Program
Page 12
Theme 4: Tier 2 and Analysis Facility
• Planning for ATLAS computing and analysis well advanced, in a tiered
and distributed model quite similar to that used by BABAR
• Tier 2 center at SLAC, representing about 20% of the installed BABAR
computing hardware, is up and functioning well
– When BABAR completes data reprocessing, ATLAS will benefit enormously
from large increase of available CPU cycles in the general queues
• By end of 2010, the BABAR will complete its intense analysis period,
and the computing model will change significantly
• At the same time, based on experience with BABAR and at the
Tevatron, we see many compute intensive operational or analysis
needs that will require major consolidated computing resources
– Physics tools and algorithm development based on multiple reprocessing of
significant fractions of ESD-level events
– Alignment calibration & development based on calibration data streams
– Development and validation of high level trigger algorithms
Planning for the SLAC ATLAS Program
Page 13
Theme 4: Tier 2 and Analysis Facility
• In general, effective use of ATLAS data will require a system that is
responsive to the demands of user-driven analyses and a close
relationship between the analysis community and computing facility
• Sharing hardware resources between BABAR, ATLAS, and GLAST,
while maintaining these systems with common personnel, will be both
cost effective & more flexible in responding to demand
• Working to develop a proposal for a Western Data Analysis Facility in
conjunction with US ATLAS and ATLAS computing management
– Needs to account for anticipated migration of BABAR needs, and projected
on-site power and cooling constraints
– Needs to quantify hardware and personnel requirements to meet the
identified or projected demand
– Needs to integrate into US ATLAS and ATLAS planning based on initial
experience with data
• Richard Mount is leading the development of a Analysis Facility
proposal with a 75% appointment in PPA
Planning for the SLAC ATLAS Program
Page 14
Theme 5: The bay area as a west-coast ATLAS center
• CERN cannot host major portions of the LHC collaborations long term
– Infrastructure does not exist to host such a community
– Computing resources to support physics analysis are hosted elsewhere
– Costs of sustaining a large US community in Geneva will be prohibitive
• As a regional center, the Bay Area and SLAC in particular can play a
leading role in supporting the exploitation of LHC physics
– Concentration of expertise on computing, analysis and detector systems
– Proximity of physics analysis support centers, organization of workshops,
tutorials and seminars will be attractive
– Strength of Theory Groups and their focus of interest in the LHC physics
program will be a source of analysis advice and ideas
– Attractive training centers for students & postdocs, from a combination of
tutorials, available expertise, & participation in upgrade R&D activities
• We are committed over the next several years to develop a community
and put in place the elements of a vibrant Bay Area center
Planning for the SLAC ATLAS Program
Page 15
Budget planning
• A detailed budget model has been developed on the basis of a specific
list of ATLAS and ATLAS upgrade participants and the White Paper
activities
• This model assumes no growth in the SLAC core budget outside 3.5%
inflation per year, and no growth in core PPA staff levels outside this
constraint
• In our 10-year outlook plan, we do show an upgrade construction profile
for ATLAS, but the preparation, R&D, and pre-engineering are assumed
to be part of our core program
• Our focus in the White Paper is FY08 and FY09, pending further
discussion with US ATLAS management
– Relatively small FTE fractions for many upgrade participants will grow in out
years
• Budget does not include an analysis of computing support budgets and
personnel, pending development of a detailed Analysis Facility model
Planning for the SLAC ATLAS Program
Page 16
Budget Model: ATLAS
Grow th of effort for existing ATLAS experim ent
6000
25.0
5000
20.0
4000
Total
Graduate
students
Postdocs
0
FY09
Tech/admin
0.0
Total
1000
Graduate
students
5.0
Postdocs
2000
Tech/admin
10.0
Professional
FY08
3000
Professional
FY09
Physicist
FY08
15.0
Dollars
30.0
Physicist
FTEs
Grow th of effort for existing ATLAS experim ent
Model of FTE and budget in FY09 for the current ATLAS experiment alone
Planning for the SLAC ATLAS Program
Page 17
Budget Model: ATLAS
Breakdow n of ATLAS and ATLAS upgrade effort for FY09
35.0
7000
30.0
6000
25.0
5000
Total
Graduate
students
0
Total
0.0
Graduate
students
1000
Postdocs
5.0
Tech/admin
2000
Professional
10.0
Physicist
ATLAS
3000
Postdocs
15.0
Upgrades
Tech/admin
ATLAS
4000
Professional
Upgrades
Physicist
20.0
Dollars
FTEs
Breakdow n of ATLAS and ATLAS upgrade effort for FY09
Model of FTE and budget in FY09 for the current ATLAS experiment and upgrades
Planning for the SLAC ATLAS Program
Page 18
Budget Model: ATLAS and ATLAS upgrades
Overall ATLAS and ATLAS upgrade effort
35.0
7000
30.0
6000
25.0
5000
Total
Graduate
students
0
Total
0.0
Graduate
students
1000
Postdocs
5.0
Tech/admin
2000
Professional
10.0
Physicist
FY09
3000
Postdocs
15.0
FY08
Tech/admin
FY09
4000
Professional
FY08
Physicist
20.0
Dollars
FTEs
Overall ATLAS and ATLAS upgrade effort
Growth of FTE and budget from FY08 to FY09 for the overall ATLAS program
Planning for the SLAC ATLAS Program
Page 19
Budget Model: ATLAS and ATLAS upgrades
FY 2007-2009 Total M$ by Cost Type
ATLAS
FY 2009 FTE by Job Category
ATLAS
Graduate
Students, 2.1
Permanent
PhD, 13.0
10
9
8
7
6
M
5
$
4
3
2
1
0
(
Administrative
/ Technician,
0.8
Engineer /
Computing
Professional,
9.3
)
Temporary
PhD, 6.2
Total FTE: 31.3
FY07
FY08
Labor
M &S
FY09
Allocation of PPA DPS
Breakdown of overall FTE plan and evolution of fully burdened budget
Planning for the SLAC ATLAS Program
Page 20
Summary
• White Paper provides a detailed model for a significantly expanded
SLAC role on ATLAS and the beginnings of an upgrade R&D effort
– Model is the result of a bottoms up look at opportunities, matched against
strengths, expertise, and interests
– Ideas for an expanded computing role have been suggested and we are
now in the process of developing a concrete proposal in consultation with
US ATLAS and ATLAS management
– Model helps us define the set of possible tasks and the staff who would be
interested in taking on a role in ATLAS or the ATLAS upgrades
• We are presently discussing this model with US ATLAS management
and OHEP to produce a coherent strategic plan with an integrated set
of roles for the SLAC ATLAS Program
– May result in some changes in emphasis in program, and some changes in
assumed program funding streams, e.g., ATLAS M&O and Detector R&D
– The influx and rapid growth of the SLAC ATLAS program is a significant
enhancement to the existing US ATLAS effort and the strength of the US
ATLAS R&D effort
Planning for the SLAC ATLAS Program
Page 21
Backup material
Planning for the SLAC ATLAS Program
Page 22
Current working model for ATLAS program
Planning for the SLAC ATLAS Program
Page 23
Possible ATLAS growth profile
20.00
18.00
16.00
14.00
12.00
$M
ATLAS upgrade
10.00
ATLAS M&O
ATLAS research
8.00
6.00
4.00
2.00
0.00
2008
2009
2010
2011
2012
2013
2014
2015
2016
Assumptions: Growth in FTE fraction for FY09 personnel and no SuperB program,
leading to additional staff influx following BABAR intense-analysis phase in 2011
Planning for the SLAC ATLAS Program
Page 24