HEPIX - Spring 2015
Tony Wong (BNL)




Yearly purchase cycle of hardware for RACF
timed with U.S. gov’t fiscal year (October to
September)
Aim for delivery of new equipment by end of
fiscal year to maximize value for the $
Purchase cycle has taken longer to complete in
recent years  12 months in FY14 and delivery
spilled into FY15
All three components of the purchase cycle are
to blame (evaluation, purchase and delivery)


As late as FY 2011, the cycle was ~3 months
long
The FY 2014 experience





Evaluation  Dec. 2013 to Jun. 2014 ~ 7 months
Purchase  Jul. to Oct. 2014 ~ 3 months
Delivery  Dec. 2014 ~ 2 months
Multiple hardware options to evaluate
(Intel/AMD cpu’s, drive options, RAID
controllers, IB, 10 GbE , etc)
Long lead times with vendor (~1-2 months) to
obtain parts to change hardware configuration

Onerous internal purchase procedure
Multi-level approval for large $ purchase
 Multiple bidding cycles
 RACF-external technical approval required
 Frustrating back and forth with procurement dept.


Delivery delays by supplier



Required to be within 6 weeks of awarding order
Actual delivery took 11 weeks
Since FY11, delays are the norm rather than the
exception


Of the three components (evaluation, purchase
and delivery), RACF only fully controls
evaluation
Proposed strategy to minimize delays




Shorten evaluation process by relying on remote
access
Address purchase delays internally at BNL
Address delivery delays with supplier
Purchase and delivery delays may be selfinflicted wounds – large work load near end of
U.S. fiscal year


Previous experience with remote access to special
purpose servers (ie, HP’s Moonshot system –
results presented at Annecy meeting)
Discussed remote evaluation goals with Dell and
HP
formal agreements signed
 BNL legal dept. involved


Split evaluation process in two phases
Initial (wide range of configurations – weed out
uninteresting technologies)  DONE
 Final (focus on specific configurations – may include
physical access to systems)  in progress






Use established VPN’s provided by supplier
Temporary account allows access over a
defined period (~2 weeks)
Supplier and/or RACF installs system software
RACF installs benchmark and executes
Occasional re-configuration of hardware, reinstallation of software and re-execution of
benchmark



Dell PowerEdge 430
 Dual E5-2660v3 @ 2.6 GHz (40 logical cores)
 10 x 8 GB DDR4 2133 MHz
 PERC H330
 4 x 2 TB Seagate 7200 rpm 6.0 Gbps SATA drives
 441.15 HS06
Dell PowerEdge R730xd
 Dual E5-2690v3 2.6 GHz (48 logical cores)
 8 x 8 GB DDR4 2133 MHz
 PERC H730P
 5 x 2 TB Seagate 7200 rpm 6.0 Gpbs SAS drives
 560.63 HS06 (462 Watts at peak load)
Dell PowerEdge R730xd
 Dual E5-2660v3 2.6 GHz (40 logical cores)
 16 x 6 GB DDR4 2133 MHz
 PERC H730
 12 x 6 TB Seagate 7200 rpm 6.0 Gpbs SATA drives
 449.60 HS06 (434 Watts at peak load)

HP DL160






Dual E5-2660v3 @ 2.6 GHz (40 logical cores)
12 x 8 GB DDR4 2133 MHz
SmartArray H240
4 x 4 TB 7200 rpm 6.0 Gbps SATA drives
451.48 HS06
HP DL180





Dual E5-2660v3 @ 2.6 GHz (40 logical cores)
12 x 8 GB DDR4 2133 MHz
SmartArray P840
12 x 6 TB 7200 rpm 6.0 Gbps SATA drives
452.65 HS06






Conclude hardware evaluation by March 30th
Purchase process: April 1st- May 31st
Delivery of hardware by August 1st
Seek remote evaluation agreements with other
suppliers (ie, Lenovo, etc)
Make remote access a permanent feature of the
evaluation process
Use remote access to complement on-site
evaluation of hardware


Do other institutions have remote access
agreements?
Is it possible to leverage the effort?
Provide access to multiple institutions
 Motivate suppliers with bigger demand


Legal considerations



What agreements are possible between institutions
and suppliers?
Laws not uniform among countries
Institutional rules vary within a country

In 2014, clusters were priced in July and
delivered in December





Standard prices for Intel Ivybridge in July
Discounts after Haswell became available in October
Delays affected HEPN programs at BNL
Need to minimize delays to maximize
purchasing power and meet deadline for
deliverables
Follow-up at Fall meeting