Monte Carlo Algorithm for neutronics calculations with Thermal hydraulics feedback on parallel computers
Nigel Davies
PhD
TITLE ....
Whole Core PWR core-follow
modelling: Coupled 3D Monte Carlo
time dependent neutronics and 3D
Thermal Hydraulics calculations,
with comparative runtimes to
coupled Deterministic methods
PhD
In a nutshell ....
Calculate k-effective, flux (in 172 groups),
and power distributions for an explicitly
modelled (in 3D) whole core PWR taking
account of depletion and thermal
hydraulics feedback – and find a way of
making the calculation run in a
reasonable time.
Presentation Outline
 This presentation describes a plan for
parallelising an existing 3D neutronics reactor
physics and criticality code MONK.
 MONK is sold commercially by the ANSWERS
business – part of SERCO
 Google: “ANSWERS” “SERCO” for more details
 Or go to http://www.sercoassurance.com/answers/
Presentation Outline
It covers …
 A very basic description of the MONK TH-code coupling scheme
 Brief review of how MONK works
 A flowchart description of parallelising MONK
MONK-TH code coupling
The basic idea is ………..
 MONK is used to calculate the power distribution at each time-step
taking account of:
– depletion (i.e. a burn-up calculation)
– temperature distribution (temperatures from user input or TH code)
 At the end of each time-step the calculated power distribution is fed
into a Thermal Hydraulics calculation which in turn generates a new
temperature distribution.
 The new temperatures are applied to the materials in the MONK
calculation at the start of the next time-step
 The loop continues until all time-steps have been done.
MONK-TH code coupling overall scheme (a
picture!)
Basic idea is that MONK generates a
power distribution on a mesh that can
be equated to (or translated into) the
TH Code mesh via a Translator code.
The output from the TH Code is
converted to a file that another
Translator code can read.
That Translator creates a file
containing the temperatures & fluid
densities
Time consuming bit is MONK – hence the need to consider the
use of Parallel Computing to speed it up! ........ This is the focus of
this talk
Capabilities of MONK
 MONK is capable of modelling
very complex geometries (see e.g.
model of BWR).
 Used widely both in UK and
abroad for criticality and some
reactor physics studies
 Iterative solution to the Boltzmann
Transport Equation (each iteration
called a “stage”)
Initial Source Guess
Solution (k-eff, flux, etc)
N Stages
New Source Guess
5
Tracking Techniques – WOODCOCK TRACKING
1. Invent a region called a
HOLE MATERIAL
2. Make every material in this
region have the SAME total
cross section
3. To do this we invent (for
every material in the HOLE)
a new reaction called a
“pseudo collision” (when this
occurs – nothing happens!)
4. Cross section for a “pseudo
collision” is added to the
existing total cross section
such that all materials have
a new but equal total cross
section
 Well used technique in the Monte
Carlo world.
 Woodcock, E., Murphy, T.,
Hemmings, P., and Longworth, T.
1965. Techniques used in the GEM
code for Monte Carlo neutronics
calculations in reactors and other
systems of complex geometry. In
Proc. Conference on the Application
of Computing Methods to Reactor
Problems, ANL-7050, 557--579.
Tracking Techniques – WEIGHTED PARTICLES
 Particles are tracked and each
one has an associated weight
(start weight =1)
W
 At a “real” collision the particle
weight is reduced
 Absorption is modelled by
reducing the weight by a
proportion equal to the absorption
probability (same as multiplying
by scatter probability)
W‘
 “Russian Roulette” technique
used to determine when a particle
“dies” (when below a cutoff)
 Survivors continue with weight=1
W ‘ = W x { Σ(scat) / Σ(total) }
Monte Carlo Process in MONK
 We have a complex geometry
within the boundaries of a box
 Use “Woodcock Tracking”
through the ENTIRE geometry
1
7
2
6
1. Choose x,y,z,g,u,v,w + Wt.
2. Obtain mfp = 1/Σ(max)
3. Calculate collision point coordinates
4. Pseudo collision
5. Real collision – Wt is reduced
6. Roulette the sample?
7. Sample leaks out
5
3
4
MONK – Process flow diagram (1 of2)
Let’s convert this process into a process flow diagram….
Calc. SOURCE particle data
TRACK to collision pt. (use Σmax)
Find MATERIAL @ collision pt.
Pseudo
REACTION
Note: on next slide
I will call this part
of the flowchart :
SOURCE /
Real
Sample
dead?
No
Next Page
Yes
TRACK /
MATERIAL /
REACTION(start)
MONK – Process flow diagram (2of 2)
SOURCE / TRACK / MATERIAL /REACTION(start)
Fill the birth store (next stage source)
Accumulate Reaction Rates
Next stage source
ACCUMULATOR
Calc. new g, u, v, w & Wt.
No
Get next source
particle
Stage Over
?
Yes
Calculate K-effective &
Print stage result
Calculate & Print Results
STOP
Yes
Finish ?
Tell next source to start sampling
No
MONK Process Flow Diagram -summarised
Initial
Source
Birth
Store
Calculate
Track Length
Fluxes
Tracking
Material
Search
Σ(max)
Reaction
Processing
Reaction rates
Pseudo & scatter
Accumulate &
Print
Birth Store data (Fission & Children)
MONK Process Flow Diagram converted to a plan
S1
S2
T-1 …
T-n
F
M-1 …
M-n
Σ(max)
To Relevant
Reaction
Computer
R-1 …
R-n
Σ(max)
A
Reaction rates
Pseudo & scatter
Birth Store data (Fission & Children)
Summary
We have covered …
 A very basic description of the MONK TH-code coupling scheme
 Brief review of how MONK works
 A flowchart description of parallelising MONK
APPENDICES
The following are additional slides used in case of questions …
Overall method
In summary :
I hope I have given you a taste of how I propose to parallelise MONK.

It offers the flexibility of either the user defining how many T, M or
R computers there are, or ultimately an optimisation algorithm could
decide.

By using many R computers we can expand the detail that can be
modelled (Domain Decomposition)

Similarly by using parallelism we can speed up the process
Function of each computer
 S1 = Generates the source particle data in the first stage and every
alternate subsequent stage. In the second stage (and every alternate
stage subsequent to the second stage) it acts as the birth store.
 S2 = similar to S1 but only starts generating source in the second
stage. This is because it acts as the birth store in the first stage.
 T-1:T-n = These are the TRACKING computers. They receive data
from S1/S2, calculate collision point data and passes this on to the next
available MATERIAL SEARCH computer. (These can also pass track
length data to the FLUX computer)
 M-1:M1n = These are the MATERIAL SEARCH computers. They
search for the material number at the collision point and pass the data
on to the relevant REACTION computer.
Function of each computer
 R-1:R-n = These are the REACTION computers. Each one processes
collisions only in a given range of Artificial Materials. This enables
them to hold only a limited amount of the nuclear data (which is the
main source of the large RAM requirement).
– They also send pseudo collision and real scattered particles back to the
TRACKING computers
– They also send the reaction rates to the ACCUMULATOR computer
– They also send birth store data to the relevant SOURCE computer S1 or
S2, as well as information to S1/S2 when a particle dies (through roulette)
or leakage – this way both the ACCUMULATOR and S1/S2 know when a
stage completes and subsequently when the cycle completes.
 A and F = These are the ACCUMULATOR and FLUX computers
respectively. These process reaction rates and fluxes and control
when results are printed.
Function of each computer
 We have only looked at processing and output here but the other
aspect is input and pre-processing
 However using MPI we can differentiate between computers (by rank)
and thus extract only what input is required.
 For example only the REACTION computers need the nuclear data -
hence the need for them to pass Σ(max) values to the
ACCUMULATOR which processes them to find the true maximum in
each group. This data is then passed to the TRACKING computers
prior to them tracking (other than that the TRACKING computers
require none of the nuclear data and none of the geometry data!)
MONK-TH code coupling: BU Mesh and TH Mesh
 Enables the user to superimpose
a X-Y-Z mesh over the MONK
geometry (defines the userrequired mesh for a burn-up
calculation - called the BU-mesh)
 In addition the user superimposes
another X-Y-Z mesh to define the
mesh for calculating the power
distribution (for the TH code) and
for receiving new temperatures
and densities (from the TH code)
 Such a X-Y-Z mesh (called the TH-
mesh) is easier to translate to and
from the TH code mesh than
MONK geometry regions are
BUTH Mesh and Artificial Materials
 X-Y-Z meshing is defined by
superimposed a BOX body over the
MONK geometry.
 The 2 meshes (BU-mesh and TH-
Z
BOX
mesh) are combined by the code
into a so-called BUTH-mesh.
 Reaction rates are calculated for
each material inside a BUTH mesh
element.
 Each material inside a BUTH mesh
element is reassigned to a so-called
“Artificial Material”
X
Y
This is because each material inside a
BUTH mesh element will burn-up
differently to that in another BUTH mesh
element and maybe its temperature will
change differently with time-step too.
“Artificial Materials”
 How does the code find out what
materials there are inside a BUTH
mesh element ?
 Before any calculation is done a
set of tracks are passed through
each BUTH mesh element
 Each track stops at regular
locations along its length
 The code determines (just as it
does during its normal tracking)
what material is at each position
 An “Artificial Material number” is
assigned to each “User defined”
material number found in there.
WOODCOCK TRACKING
 The following slides give a brief overview of Woodcock tracking …
Hole Material Tracking
 Tracking body boundaries is necessary only because the mean free
path varies between materials.
 If the mean free path was constant within a multi-material region, the
tracking of boundaries would not be required.
 Since the mean free path is not naturally constant between materials,
artificial extra cross-sections would have to be added if we want the
total cross-section, and hence the mean free path, to be constant for all
materials in a region.
Hole Material Tracking
 Consider a region containing materials M1, M2, .... Mn with
macroscopic total cross-sections 1,  2, ....  n.
 In order to achieve a constant mean free path within the region the
largest cross-section is located ( max) and artificial cross-sections are
added to the other values to make them equal to  max:
i.e.  max = max( 1,  2, ....  n)
 max =  1 +  1’ =  2 +  2’ = .... =  n +  n’
where  1’,  2’, ....  n’ are additional artificial cross-sections that must
be dealt with.
Hole Material Tracking
 Tracking within the hole material now proceeds using the minimum
mean free path (1/  max).
 At each collision site the material present is identified.
 If the material present is the one with the largest cross-section then a
normal (real) collision occurs.
 If the material is one of the others (say material i), then there is a
probability of  i/  max that the collision is a real collision. Otherwise
the collision is due to the artificial cross-section  i’.
Hole Material Tracking
 The device of increasing the
total cross-sections and
introducing additional artificial
cross-sections therefore
produces additional artificial
collisions. These must clearly
not disturb the passage of the
particle, so that the distribution
of real particle collisions is not
disturbed in any way.
3
A
1
2
B
4
5
C
Artificial collisions (1- 5) do not disturb the
passage of the particle in its direction of travel
between real collisions (A - C)
Hole Material Tracking
 It can be shown that although the total number of collisions is
increased by the addition of the artificial cross-sections, the distribution
of real collisions is unchanged, and hence the process is unbiased.
 That it is correct to do this may be seen from the following
consideration. The equation from which the distribution of collision
points is derived is:
I(x+dx) = I(x) -  i I(x) dx
where  i has the meaning above, and the function I(x) is the intensity
of neutrons at a point x along an axis. The second term on the right
hand side relates to the removal of neutrons due to collisions.
Hole Material Tracking
 If one now uses a cross-section , a term must be created relating to
the production of neutrons. The equation becomes:
I(x+dx) = I(x) - 
I(x) dx + ( -  i) I(x) dx
 The two equations are therefore equivalent and hence the distribution
of real collisions will be given correctly