The International Workshop on HEP Data Grid
A Design for KCAF
for CDF Experiment
Kihyeon Cho
(CHEP, Kyungpook National University)
and
Jysoo Lee
(KISTI, Supercomputing Center)
November 8-9, 2002
Contents
 CHEP and Fermilab CDF
 CDF Data/Analysis Flow
 CDF Grid RoadMap (bottom –up method)
□ Step 1. CAF (Central Analysis Farm)
□ Step 2. DCAF (DeCentralized Analysis Farm)
□ Step 3. Grid
 KCAF (DeCenteralized Analysis Farm in Korea)
 Future Plan
CENTER FOR HIGH ENERGY PHYSICS
2
KIHYEON CHO
CHEP and Fermilab CDF Grid
Fermilab in USA
KCAF (DCAF in Korea)
Korea
CHEP
Network between CHEP and overseas
FNAL
CERN
TEIN
10Mbps
APII
45Mbps
Disc Storage
(NAS) 100GB
Gigabit Ethernet
KOREN
APII
2 X 1Gbps
Gigabit Ethernet
C6509
KEK
Gigabit
Ethernet
Tape Library
(0.6 TB)
DB Server
Gigabit Ethernet/Fast Ethernet
L3 Switch
(3군)
Gigabit Switch
(CHEP)
…
Server
s
CENTER FOR HIGH ENERGY PHYSICS
Servers
IBM 8271
Linux Clusters
Cluster Servers
…
PCs
4
KIHYEON CHO
Hub
Overview of Fermilab
CDF
Fixed Target
Experiment
Booster
p source
CENTER FOR HIGH ENERGY PHYSICS
Main Injector
and Recycler
5
D
D0
KIHYEON CHO
CDF Collaboration
North America
Europe
3 Natl. Labs
25 Universities
Asia
1 Research Lab
6 Universities
1 University
2 Universities
1 Research Lab
4 Universities
1 University
4 Universities

Totals
 11 countries
 52 institutions
 525 physicists
CENTER FOR HIGH ENERGY PHYSICS
2 Research Labs
KOREA
1 University
1 University
Center for High Energy Physics:
Kyungpook National University
Seoul National University
SungKyunKwan University
KIHYEON CHO
Production Farm
CENTER FOR HIGH ENERGY PHYSICS
7
KIHYEON CHO
CDF Computer Resource CAF Stage 1
FCC (Feynman Computer Center)
 Current configuration:
Stage 1
– 63 dual worker nodes.
– Roughly 160 GHz total,
compared to 38 GHz for
fcdfsgi2, and 8 GHz for
cdfsga (Run I).
– 7 fileservers for physics
(~50% full), 5 for DH, 2 for
development, 2 for user
scratch.
CENTER FOR HIGH ENERGY PHYSICS
8
KIHYEON CHO
Integer
CAF Design
 CAF design
considerations:
– Submit jobs from
'anywhere'
– Job output can be:
sent directly to
desktop
or stored on CAF for
later retrieval or
– input to subsequent
job
CENTER FOR HIGH ENERGY PHYSICS
9
KIHYEON CHO
DCAF/Grid at CDF Experiment
 Requirement in 2005
– 200 simultaneous users will analyze 107 events of secondary
data set in a day.
– Need ~700 TB of disk and ~5THz of CPU by end of FY’05
 Current CAF (Central Analysis Farm) is not enough:
– Limited resources and spaces at FCC
– In case of network problems at FCC, it is dangerous.
 We need DCAF (DeCentralized Analysis Farm) and Grid
 DCAF (DeCentralized Analysis Farm)
– Users around regional area and/or around the world
– Korea, Toronto, Karlsruhe, ….
– Let us call DCAF in Korea as “KCAF”
CENTER FOR HIGH ENERGY PHYSICS
10
KIHYEON CHO
DCAF for CDF Experiment
APII (Asia Pacific
DCAF in Karlsruhe
Information Infrastructure)
DCAF in Toronto
CAF in USA
KCAF (DeCentralized
Analysis Farm in Korea)
Center for HEP (CHEP)
Kyungpook National U.
Daegu, Korea
CENTER FOR HIGH ENERGY PHYSICS
11
KIHYEON CHO
Proposed Schemes for Final Goal
CAF or
DCAF in Korea (KCAF)
CENTER FOR HIGH ENERGY PHYSICS
12
KIHYEON CHO
The Road Map for Goal
 Step 1. To make MC production farm using KCAF
– First, we start to construct 12 CPU test bed for KCAF.
– After policy inside of CHEP (another test bed for EDG and
iVDGL), we will decide how many CPUs for actual MC
generation farm will be used among this year’s planed 140
CPUs.
 Step 2. To handle real data
– To extend the KCAF to the real data handling system using SAM
(Sequential data Access via Meta-data), Gridftp, etc after settling
down real data handling system.
 Step 3. Final goal of CDF Grid
– A gridification for KCAF related with EDG, iVDGL and CDF Grid
CENTER FOR HIGH ENERGY PHYSICS
13
KIHYEON CHO
The Road Map in Detail
1.
Software

•
2.
Kerborse
•
•
•
3.
Client : login from Korea to Fermilab
Server : login from Fermilab to Korea
KDC : Client & Server Trust
Data Handling System
•
•
4.
5.
CDF Software
FBSNG for batch job
Glasgow, Italy and Yale use Data Handling System.
SAM station needs to be installed on 1 PC work node.
OS for Fermilab Red Hat Linux 7.3 with Kernel 2.4.18
Fundamental license problem for DB
CENTER FOR HIGH ENERGY PHYSICS
14
KIHYEON CHO
A Design of KCAF for CDF (Fermilab) Grid
FCC (CAF)
STKen
fcdfsam
stager
dCache
Calibration
Data
CDFen
(Raw Data)
rcp
dCache
Technical
Request
1TB buffer (FCC)
rcp,
GridFTP
bbftp
CHEP
User Desktop
(KCAF)
KCAF Head Node
CAFGUI
KCAF Cluster
stager
rcp
ICAFGUI
cp
SAM Station (KCAF)
Smaster
FSS, Stager
Remote
Desktop
CENTER FOR HIGH ENERGY PHYSICS
ICAF FTP server
15
KIHYEON CHO
Future Plan (This year)
 Around 140 CPU linux clusters will be constructed at the end of this
year at CHEP.
 We are constructing KCAF for CDF at CHEP, Kyungpook National
University.
 Contribute 1 T byte hard disk (now) + 4 T byte hard disk
(December) to CDF for the network buffer between CHEP and
Fermilab.
 The DCAF demonstration will be shown at SC2002 on November
16, 2002.
CENTER FOR HIGH ENERGY PHYSICS
16
KIHYEON CHO