Geant4 Physics Validation
(mostly electromagnetic, but also hadronic…)
K. Amako, S. Guatelli, V. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen,
L. Pandola, S. Parlati, M.G. Pia, T. Sasaki, L. Urban
et al.
Geant4 Space User Workshop
Leuven, 5-7 October 2005
Maria Grazia Pia, INFN Genova
Geant4 Physics Models
Ample variety of physics models in the Geant4 Toolkit
– complementary and alternative
Electromagnetic physics
– Standard, LowEnergy, Muon, Optical
Hadronic physics
– data-driven, parameterised and theory-driven models
Geant4 Physics Book
– on-going project to document the performance of Geant4 physics against
experimental data and in relevant experimental application domains
Maria Grazia Pia, INFN Genova
Validation process
Geant4 test process
– Physics packages are subject to unit and system testing
– Verification, validation of single processes/models performed by Working Groups
Validation process
– systematic: cover all models of a given process
– comparison to experimental data and established reference databases
– rigorous software process to guarantee quality and reliability
– statistical analysis: quantitative mathematical evaluation
Goals
– evaluate quantitatively the accuracy Geant4 physics models
– document their respective strength
– provide guidance to users to select the models to use in their applications
Maria Grazia Pia, INFN Genova
G.A.P Cirrone, S. Donadio, S. Guatelli, A. Mantero, B. Mascialino, S. Parlati, M.G. Pia,
A. Pfeiffer, A. Ribon, P. Viarengo
“A Goodness-of-Fit Statistical Toolkit”
IEEE- Transactions on Nuclear Science (2004), 51 (5): 2056-2063
Partly funded by ESA (SEPTIMESS Project)
Maria Grazia Pia, INFN Genova
GoF algorithms
(currently implemented)
Algorithms for binned distributions
–
–
–
–
Anderson-Darling test
Chi-squared test
Fisz-Cramer-von Mises test
Tiku test (Cramer-von Mises test in chi-squared approximation)
Algorithms for unbinned distributions
–
–
–
–
–
–
Anderson-Darling test
Cramer-von Mises test
Goodman test (Kolmogorov-Smirnov test in chi-squared approximation)
Kolmogorov-Smirnov test
Kuiper test
Tiku test (Cramer-von Mises test in chi-squared approximation)
In progress
–
–
–
–
Watson test
Girone Test
Weighted Kolmogorov-Smirnov test
Weighted Cramer-von Mises test
The most complete GoF software system on the market
– even among commercial/professional statistics software products)
Maria Grazia Pia, INFN Genova
Overview of recent validation activities
Geant4 Physics Book: Electromagnetic Volume
– comparison against the NIST databases
– K. Amako, S. Guatelli, V. N. Ivanchenko, M. Maire, B. Mascialino, K. Murakami, P. Nieminen,
L. Pandola, S. Parlati, M. G. Pia, M. Piergentili, T. Sasaki, L. Urban
Comparison of Geant4 electromagnetic physics models against the
NIST reference data
IEEE Trans. Nucl. Sci., Vol. 52, Issue 4, Aug. 2005, 910-918
Current Physics Book projects (preliminary results)
–
–
–
–
Bremsstrahlung final state
Atomic relaxation and PIXE
Bragg peak
Radioactivity from rocks and sands
Other Geant4 validation activities
– LCG Simulation Validation Project: focus on hadronic physics
 see A. Ribons’s talk at EPS-HEP, Lisbon, July 2005
– Validation of specific physics models done by each Geant4 Working Groups
Maria Grazia Pia, INFN Genova
NIST Test
Photon Mass Attenuation Coefficient
Photon Partial Interaction Coefficient
– related to the cross section of a specific photon
interaction process
Electron CSDA range and Stopping Power
Proton CSDA range and Stopping Power
a CSDA range and Stopping Power
Elements
Be, Al, Si, Fe, Ge, Ag, Cs, Au, Pb, U
Geant4 models: electrons and photons
(span the periodic element table)
Energy range
photon
1 keV – 100 GeV
electron 10 keV – 1 GeV
proton
1 keV – 10 GeV
a
1 keV – 1 GeV
Simulation configuration reproducing NIST conditions
(ionisation potential, fluctuations, production of secondaries etc.)
Maria Grazia Pia, INFN Genova
Standard
Low Energy EEDL/EPDL
Low Energy Penelope
Geant4 models: protons and a
Standard
Low Energy ICRU49
Low Energy Ziegler 1977
Low Energy Ziegler 1985
Low Energy Ziegler 2000
(Low Energy: free electron gas +
parameterisations + Bethe-Bloch)
Statistical analysis
Goodness-of-Fit test (Statistical Toolkit)
Alternative hypotheses under test:
Geant4
simulation
results
+
Reference
Data
GoF test
(χ2 test)
H0: Geant4 simulation = NIST data
H1: Geant4 simulation ≠ NIST data
Distance between
Geant4 simulation
and NIST reference data
Test result
p-value
The p-value represents the probability that the test statistics has a value
at least as extreme as the one observed, assuming the null hypothesis is true
0≤p≤1
p < 0.05 Geant4 simulation and NIST data differ significantly
p > 0.05 Geant4 simulation and NIST data do not differ significantly
Maria Grazia Pia, INFN Genova
Photon mass attenuation coefficient
Geant4 models:
• Standard
• Low Energy – EPDL
• Low Energy – Penelope
Reference data: NIST - XCOM
Experimental set-up

1  I 

ln   Monochromatic

d  I 0  photon beam (I )
o
Transmitted
photons (I)
Mass attenuation coefficient in Fe
Geant4 LowE Penelope
Geant4 Standard
Geant4 LowE EPDL
NIST - XCOM
Results
All Geant4 models compatible
with NIST
p-value stability study
Best agreement:
Geant4 LowE models
H0 REJECTION AREA
Maria Grazia Pia, INFN Genova
Compton interaction coefficient
Geant4 models:
• Standard
• Low Energy – EPDL
• Low Energy – Penelope
Reference data: NIST - XCOM
Compton interaction coefficient in Ag
c 
Geant4 LowE Penelope
Geant4 Standard
Geant4 LowE EPDL
NIST - XCOM
Maria Grazia Pia, INFN Genova
(cross section)
Results
All Geant4 models compatible with NIST
Best agreement: Geant4 LowE-EPDL
p-value stability study
A 
( )c
NAV 
H0 REJECTION AREA
Photoelectric interaction coefficient
Geant4 models:
• Standard
• Low Energy – EPDL
• Low Energy – Penelope
Reference data: NIST - XCOM
Photoelectric interaction coefficient in Ge
Geant4 LowE Penelope
Geant4 Standard A 
ph

( ) ph
Geant4 LowE
EPDL
NAV 
NIST - XCOM
Geant4 LowE Penelope
Geant4 Standard
Geant4 LowE EPDL
Maria Grazia
INFN Genova
NISTPia,
- XCOM
(cross section)
Results
All Geant4 models compatible with NIST
Best agreement: Geant4 LowE models
p-value stability study
H0 REJECTION AREA
Pair production interaction coefficient (cross section)
Geant4 models:
• Standard
• Low Energy – EPDL
• Low Energy – Penelope
Reference data: NIST - XCOM
Results
All Geant4 models compatible with NIST
and equivalent
Pair production interaction coefficient in Au
A 
( ) pp
NAV 
Geant4 LowE Penelope
Geant4 Standard
Geant4 LowE EPDL
NIST - XCOM
Maria Grazia Pia, INFN Genova
p-value (pair production interaction coefficient test)
pp 
p-value stability study
H0 REJECTION AREA
Rayleigh interaction coefficient (cross section)
Geant4 models:
• Low Energy – EPDL
• Low Energy – Penelope
• (no standard Rayleigh process)
Reference data: NIST - XCOM
Results
The Geant4 Low Energy models look in
disagreement with the reference data for
some materials
EPDL
XCOM
Rayleigh interaction coefficient in Be
r 
Geant4 LowE Penelope
Geant4 LowE EPDL
NIST - XCOM
Maria Grazia Pia, INFN Genova
A 
( )r
NAV 
Penelope
XCOM
Be
0.99
1
Al
0.32
<0.05
Si
0.77
<0.05
Fe
1
<0.05
Ge
<0.05
0.39
Ag
0.36
0.08
Cs
<0.05
<0.05
Au
<0.05
<0.05
Pb
<0.05
<0.05
U
<0.05
<0.05
Rayleigh interaction coefficient
The disagreement is evident between 1 keV and 1 MeV photon energies
Rayleigh interaction coefficient in Au
For what concerns the Geant4 Low Energy
EPDL model, the effect observed derives from
an intrinsic inconsistency between Rayleigh
cross section data in NIST-XCOM and the
cross sections of EPDL97, on which the model
is based
Differences between EPDL97 and NIST-XCOM
have already been highlighted in a paper by Zaidi,
which recommends the Livermore photon and
electron data libraries as the most up-to-date and
accurate databases available for Monte Carlo
modeling.
r 
NIST
A 
( )r
NAV 
EPDL 97
Zaidi H., 2000, Comparative evaluation of photon cross section libraries for materials of interest in
PET Monte Carlo simulation IEEE Transaction on Nuclear Science 47 2722-35
Maria Grazia Pia, INFN Genova
Electron Stopping Power
Experimental set-up
Electrons are generated with
Geant4 models:
random direction at the center of
the box and stop inside the box
• Standard
• Low Energy – EEDL
Maximum step allowed in tracking particles
was set about1/10 of the expected range
• Low Energy – Penelope
Reference data: NIST – ESTAR (ICRU 37) value, to ensure the accuracy of the calculation
p-value stability study
1 dE
SP  ( )
 dx
Results
All Geant4 models compatible with NIST
and equivalent
Geant4 LowE Penelope
Geant4 Standard
Geant4 LowE Livermore
NIST - ESTAR
H0 REJECTION AREA
Maria Grazia Pia, INFN Genova
Electron CSDA Range
CSDA: particle range
without energy
loss fluctuations and multiple scattering
Geant4 models:
Results
• Standard
All Geant4 models compatible with NIST
• Low Energy – EEDL
and equivalent
• Low Energy – Penelope
Reference data: NIST – ESTAR (ICRU 37)
CSDA range in U
Geant4 LowE Penelope
Geant4 Standard
Geant4 LowE Livermore
NIST - ESTAR
Maria Grazia Pia, INFN Genova
p-value stability study
H0 REJECTION AREA
Proton stopping power - range
Stopping power: p-value stability study
Stopping power in Al
H0 REJECTION AREA
CSDA range: p-value stability study
+
Geant4 LowE Ziegler 1985
Geant4 LowE Ziegler 2000
Geant4 Standard
Geant4 LowE ICRU 49
NIST - PSTAR
Results
Ziegler parameterisations are as authoritative
as ICRU 49 ones
Maria Grazia Pia,rather
INFN Genova
Comparison
than validation
H0 REJECTION AREA
a stopping power and range
CSDA range in Si
Stopping power: p-value stability study
Geant4 LowE Ziegler 1977
Geant4 Standard
Geant4 LowE ICRU 49
NIST - ASTAR
H0 REJECTION AREA
The complex
modeling
of ion interactions
in the
low energy
is
The complex
physicsphysics
modeling
of ion interactions
in the low
energy
range isrange
addressed
by Energy
the Geant4
Lowand
Energy
package and
represented
one of the for
by the addressed
Geant4 Low
package
it represented
oneit of
the main motivations
main motivations for developing this package
the Maria
developing
of this package.
Grazia Pia, INFN Genova
Bremsstrahlung
3 sets of models:
Standard: G4eBremsstrahlung
Low Energy EEDL: G4LowEnergyBremsstrahlung
Low Energy Penelope: G4PenelopeBremsstrahlung
3 angular distributions: Tsai, 2BS, 2BN
Penelope
Standard
Penelope
LowE-EEDL TSAI (def)
LOWE-EEDL 2BS
LOWE-EEDL 2BN
Low Energy EEDL
(default)
Angle (deg)
Angle (deg)
Angular distribution of photons is strongly model-dependent
Maria Grazia Pia, INFN Genova
Reference data
Transmitted energy spectrum
at two different emission angles
for four materials (Al, Pb, W, Ag)
Absolute yields are reported
(= photons/primary), though
with an “odd” normalization
R. Ambrose et al., Nucl. Instr.
Meth. B 56/57 (1991) 327
Maria Grazia Pia, INFN Genova
The absolute Bremsstrahlung
cross section can be tested
Relative comparison...
LowEPenelope
Intensity/Z (eV/sr keV)
Intensity/Z (eV/sr keV)
Low E EEDL TSAI
Photon energy (keV)
Photon energy (keV)
Relative comparison (45 degree direction)
Shapes of the spectra are in good agreement
Work in progress, will be published
Maria Grazia Pia, INFN Genova
Proton Bragg Peak
Reference data from CATANA (INFN-LNS Hadrontherapy Group)
Geant4 models: electromagnetic
Standard
Low Energy ICRU 49
Low Energy Ziegler 1977
Low Energy Ziegler 1985
Low Energy Ziegler 2000
Systematic test in progress
Lot of work…
Preliminary results
Geant4 models: hadronic
Precompound + default de-excitation
Precompound + GEM evaporation
with/without Fermi Break-up
Binary Cascade
(including Precompound + de-excitation)
Bertini Cascade
Bertini Cascade + Bertini elastic scattering (when available)
Parameterised
Geant4 “educated guess” Medical Dosimetry Physics List
Maria Grazia Pia, INFN Genova
EM only – Standard
Maria Grazia Pia, INFN Genova
EM only – ICRU49
ENTIRE
PEAK
Exp
G4
S
2.89
2.43
T
3.26
3.83
GoF test
LEFT TAIL
Exp
S
2.89
2.43
T
3.26
3.83
GoF test
9.77
11.89
T
2.66
3.16
CVM-AD
Exp
G4
S
3.89
12.24
T
1.03
1.00
GoF test
G4
G4
S
RIGHT TAIL
X>30mm
Exp
CVM-AD
x<=30mm
GoF test
ENTIRE
PEAK
KS-AD
Maria Grazia Pia, INFN Genova
CVM-AD
EM only – ICRU49 – GoF results
ALL (N1=149 N2=67)
CVM
AD
Test statistics
0.112938
0.853737
p-value
0.525095
0.443831
CVM
AD
Test statistics
0.0701584
0.645422
p-value
0.750593
0.606120
KS
AD
Test statistics
0.333333
0.816534
p-value
0.724871
0.469251
LEFT TAIL x<=30mm
(N1=140 N2=61)
RIGHT TAIL X>30mm
(N1=9 N2=6)
Maria Grazia Pia, INFN Genova
LowE + precompound default
ALL
Exp
S
2.89
T
3.26
GoF test
LEFT TAIL
G4
CVM-AD
Exp
G4
x<=30mm
S
9.77
T
2.66
GoF test
RIGHT TAIL
X>30mm
AD
Exp
S
3.89
T
1.03
GoF test
G4
KS-AD
Maria Grazia Pia, INFN Genova
ICRU49 + precompound – GoF results
ENTIRE
PEAK
(N1=149 N2=66)
CVM
AD
ICRU 49 only
Test statistics
0.06
0.499375
p-value
0.79
0.747452
LEFT TAIL
x<=30mm
(N1=140 N2=60)
CVM
AD
Test statistics
0.03
0.232255
p-value
0.97
0.978972
RIGHT TAIL
X>30mm
(N1=9 N2=6)
KS
AD
Test statistics
0.33
0.901787
p-value
0.73
0.413129
Maria Grazia Pia, INFN Genova
0.525095
0.443831
0.750593
0.606120
0.724871
0.469251
Nuclear de-excitation alternative models
Work in progress, more to come…
Maria Grazia Pia, INFN Genova
Radioactive spectrum
Studies of environmental radioactivity from
rocks and sands at the Gran Sasso Laboratory
sample
detector
source
Geant4 (LowE EM) can the results of a calibration with a 60Co source
(in the presence of the sample) reproduce very well
Anderson-Darling test (for binned data)
Lower E peak
– A2 = 0.45
– p-value = 0.80
Higher E peak
– A2 = 1.05
– p-value = 0.33
Both peaks
– A2 = 0.80
– p-value = 0.48
Maria Grazia Pia, INFN Genova
Lower part of the histogram
is not meaningful
simulation
simulation
data
data
Maria Grazia Pia, INFN Genova
Backscattering coefficient – E=100keV
Angle of incidence (with
respect to the normal to
the sample surface) = 0°
Lockwood et al. (1981)
G4 LowE
Maria Grazia Pia, INFN Genova
…and much more
No time to present all the on-going activities…
– atomic relaxation
– LCG Simulation Validation project
– validation for specific use cases (medical dosimetry, radiation damage to
components, underground experiments etc.)
– …
Physics validation is a large component of Geant4 Collaboration’s
activities
– Geant4 Physics Validation Workshop, Genova, July 2005
– http://www.ge.infn.it/geant4/events/july2005/
Maria Grazia Pia, INFN Genova
Geant4 validation is not an easy job…
experimental data often
exhibit large differences!
Maria Grazia Pia, INFN Genova
Conclusion
More results available, no time to show them all…
Systematic, quantitative validation of Geant4 physics in progress
– all available models
– rigorous statistical analysis
A lot of work!
– first paper published
– several on-going projects
– limited resources available
The validation work provides valauable feedback for the
improvement of Geant4 physics models
Maria Grazia Pia, INFN Genova
Communication & Publications
Feedback from Geant4 users is a very helpful contribution to
Geant4 validation
– if you have a problem with Geant4 physics, please tell us
– if you get nice results, please tell us too…
IEEE Trans. Nucl. Sci.
–
–
–
–
major scientific journal about nuclear technologies and instrumentation
many Geant4-related papers published or currently in the review process
M.G. Pia Associate Editor (software - Instrumentation)
please consider publishing your results concerning Geant4 applications
Maria Grazia Pia, INFN Genova