Une première expérience de validationdebuggage et développement communautaire
Contributions from Dominique Laurence, Tim Craft, Rob Prosser, Hector Iacovides, Peter Stansby,
David Apsley, Juan Uribe, Charles Moulinec, Flavien Billard, Stefano Rolfo, Yacine Addad, Imran
Afgan, Alex Skillen, Neil Ashton, Ruggero Poletto, Nicolas Jarrin, Dalila Ammour, Alastair West,
James McNaugthon, Mark Mawson, Mahmoud Assad, Umair Ahmed… (and yet more!)
Presenté par Alistair Revell
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 1/33
Outline
•
•
•
•
•
Introduction
Our research on turbulence
Our PhD students
Dissemination
Conclusions & Future
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 2/33
The University of Manchester
• Came into being on October 2004 as a result of the merge of
• UMIST (University of Manchester Institute of Science & Technology)
• Victoria University of Manchester.
– Now with 40 000 + students
• (70% undergraduate: 30% postgraduate)
• Manchester University is
– 3rd ranked UK University by Research Power
• most popular University in UK as voted by students
– Top UK graduate employer
• 90% in employment/further study within 6 months
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 3/33
History of Collaboration
• School of Mechanical, Aerospace & Civil Engineering
– 80+ academics
• 20+ in Thermo/Fluid Mechanics group
– Fluids is split into expert groups:
• Experimental, Computational Fluid Dynamics,
• Energy, Multi-Physics, Environmental Flows,
• Turbulence Mechanics,
Osbourne Reynolds experiment 1883
• Pr. Dominique Laurence (EDF R&D) joined part time in 1999
– introduced Code_Saturne
• primary focus on turbulence modelling and simulation
• Modelling and Simulation Centre (MaSC) launched Jan 2011!
– “building on open-source software provided by EDF and longestablished strengths of MACE”
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 4/33
Turbulence Modelling Using
Code_Saturne
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 5/33
Turbulence Modelling
• Turbulence is multi dimensional semi-deterministic motion
– Can be resolved exactly
• but requires enormous computational power
• or resolve only large scales:
– Large Eddy Simulation (LES)
• instantaneous information / expensive
• or can be modelled using semi-empirical approximations
– Reynolds Averaged Navier Stokes (RANS)
• time-averaged information / cheap
• industry needs a range of approaches to suit the problem
– LES very useful tool for high detail analysis, but expensive…
– RANS will still be used in industry for a long time
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 6/33
EU: ATAAC : 2009 - 2012
• Advanced Turbulence Simulation for Aerodynamic Application Challenges
– “Improvements to CFD for aerodynamic flows used in today’s aero
industry focusing on state of the art 3D turbulence modelling.”
• 20 partners (industries and universities) from 10 countries
– all developments implemented into Code_Saturne
– possibility to benchmark Code_Saturne against other codes
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 7/33
Code Saturne on HECToR
• HECToR: UK national super computer
– based in Edinburgh
• Code_Saturne installed and benchmarked by Daresbury
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 8/33
v2-f based DES
• Detached Eddy Simulation is a blend of RANS and LES
– trade off cost vs. accuracy
• flow separation is challenging for turbulence models
• we have developed a new v2-f based DDES
– v2-f is improved near wall model (developed at Manchester over ~8 years)
• superior to Menter SST-DDES here
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 9/33
Embedded LES
• Embedded LES is a practical compromise
– use LES where you really need it and can afford to do it well!
– use RANS in other regions to minimize cost
• BUT.. Need to generate realistic turbulence at Inlet
– computational efficiency, flexibility, generality
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 10/33
Synthetic Eddy Method (SEM)
• eddies are generated in a 3D box
–
–
–
–
synthetic shapes are convected through with imposed mean velocity
instantaneous velocities are constructed from superimposition
a single plane is extracted and used as LES inlet
Time and space coherence are preserved
fluctuations
LES channel: cost 5 days computing
University of Manchester: Alistair Revell
SEM field: cost < 1 minute
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 11/33
ongoing research projects
• UK Government funding body for Engineering & Physical Science
• EPSRC : Hybrid RANS – LES 2010-13
– Development of two velocities method
• in collaboration with Imperial College London
– Reviewers explicitly stated that open-source codes were an advantage
• EPSRC: UK Turbulence Consortium (UKTC) 2009-14
– “Develop a world lead in turbulence and usage of the resulting data”.
– Code Saturne tested and implemented by Daresbury Laboratories
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 12/33
ongoing collaborations
• Daresbury Laboratories (STFC)
– Test for scalability of Code_Saturne up to 32000 cores (JuGene)
• Daresbury awarded Code_Saturne the GOLD AWARD
• Optimisation of linear solver, I/O and preprocessor for HPC.
• Test with up to 2 billion cells.
‘Partnership for Advanced
Computing in Europe’
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 13/33
ongoing collaborations
• KNOO 2 : Keeping the Nuclear Option Open 2011-2014
– Code_Saturne selected as key code
• will be used by other academic partners
– Follow on from KNOO 2006-2010
– Next Generation reactor systems
• Fuel thermal hydraulics
• Safety and performance
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 14/33
PhD students Using Code_Saturne
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 15/33
Current Code_Saturne PhDs
work directly in
Code_Saturne
on a daily basis
contributing
developments
to the code
PhD Student
Turb Model
Devel
Heat
Transfer
Neil Ashton
✔
Alastair West
✔
✔
Dalila Ammour
✔
✔
Fire
Numerics
Moving
Mesh
✔
✔
J. McNaughton
✔
✔
M. Ya-alimadad
Ruggero Poletto
✔
✔
Mahmoud Assad
✔
✔
Umair Ahmed
✔
✔
Stuart Russant
✔
✔
Other PhDs
Indirect Benefit to Code_Saturne
Wei Wu
Transformer heat transfer; improved Network Modelling. Masters student projects
Mark Mawson
GPU testing for bottleneck algorithms. Indication of possible speed up gains
Imama Zaidi
Benchmark with Star CCM+
Ioannis Asproulias
Experience with OpenFoam, benchmarking models for Compressible flow
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 16/33
Code_Saturne PhD students…
• Dalila Ammour: Buoyancy driven flows
– Flow inside cavities with different configurations
• Development of Analytical Wall Function
• Alastair West: Thermal Mixing around In-line Tube Banks
– Simulations of square in-line tube banks
• URANS compared against LES and expt data
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 17/33
Code_Saturne PhD students…
• Mahmoud Assad: Improved Fire Modelling
– Non-premixed flames
• Development of new turbulence model for fire applications
• Umair Ahmed : Premixed combustion modelling
– Work using advanced model for better accuracy of the flame structure
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 18/33
Code_Saturne PhD students…
• James McNaughton: Marine Current
turbines modelling
– ETI project on tidal energy industry.
– development of moving mesh module
• PDRA: Imran Afgan
• Mark Mawson: Multigrid Solvers for GPUs
– highly parallel architectures, originally designed
for 3D graphics
• Tesla GPUs out-perform CPU by up to 12x, Fermi
GPUs by up to 24x.
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 19/33
Code_Saturne PhD students…
• Alex Skillen: Multiphysics modelling of transformer cooling
• complex network of vertical and horizontal oil flow ducts
– mixed convection modes
• many complex effects to incorporate
–
–
–
–
conjugate heat transfer
transient loading
variable fluid properties
inclusion of cooling cycle
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 20/33
Saturne PhD starts in past decade
6
5
4
12 more skilled PhDs
over next 3 years
-- where will they go?
3
2
1
0
2001 2002 2003 2004 2005 2006 2007 2008 2009 2010
10 graduated
People
Sofiane Benhamadouche
Yacine Addad
Juan Uribe
Charles Moulinec
Alistair Revell
Imran Afgan
Sofian Berrouk
Nicolas Jarrin
Amir Keshmiri
Stefano Rolfo
Flavien Billard
University of Manchester: Alistair Revell
Years at UniMAN
2001-2004
2001-2010
20012002-05
20022003-07, 112003-06
2004-07
2005-2010
20062006-
Now?
EDF
Khalifa University, UAE
Post-doc, University of Manchester
Daresbury Laboratory, UK
Faculty, University of Manchester
Post-doc, University of Manchester
Faculty, Petroleum Institute, UAE
AREVA, France
Faber Maunsel (energy), UK
Post-doc, University of Manchester
Post-doc, University of Manchester
Using CS?
✔
✔
✔
✔
✔
✔
✔
✔
✔
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 21/33
Critical Mass Activity in MACE
• 6 Full time PDRA :
– J. Uribe, D. Columbo, S. Rolfo, F. Billard, I. Afgan, A. Skillen
• critically important to drive forward quality research
NEWSFLASH: 3 year EPSRC Post-Doc for Verif & Valid advertised yesterday
(www.jobs.ac.uk)
• 9 academics involved in Code_Saturne (PhD projects)
– Prof. Laurence (3), Prof. Launder (1), Prof. Stansby (2), Prof. Iacovides
(2) Dr. Tim Craft (3), Dr. Apsley (1), Dr. Prosser (5), Dr. Davey (1), Dr.
Revell (8)
• important to diversify and sustain presence of code
We’ve reached critical mass…
now we’re starting to see chain reaction!
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 22/33
Dissemination of Code_Saturne
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 23/33
Dissemination: TWiki
• Code_Saturne TWiki, web based collaboration platform
wiki-based site with different areas for different projects
launched in 2007 and grown to over 1000 hits per day
test case database
90000
code tips/forum
80000
Code_Saturne News
70000
Code_Saturne FORUM
60000
User activity
50000
MAIN Twiki
hits / month
–
–
–
–
40000
University of Manchester: Alistair Revell
20000
10000
EDF R&D industrielle d'expertise et open source: Juillet 2011
May-07
Feb-07
Nov-06
Aug-06
May-06
Feb-06
Nov-05
Aug-05
May-05
Feb-05
Nov-04
Aug-04
May-04
Feb-04
Nov-03
0
Aug-03
– meetings
– news
– projects
30000
May-03
• PhD student research
Slide 24/33
Twiki (saturne.cfdtm.org)
• 316 registered users from all over the world
– users popping up in Brasil, South Korea, Venezuela, Vietman, Iran …
• Creation of a large open online database for CFD benchmarking
– contributions from different groups for a range of cases
– comparison of different CFD codes and turbulence models:
• enabling single point of reference
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 25/33
Code_Saturne related publications
• 20+ related publications in 12 different peer reviewed scientific journals:
– Int. J. Num Methods in Fluids, J. Fluids Eng., J. Fluid Mech., Phys. of Fluids, J. of
Turbulence, ASME Heat Transfer, Num. J. Heat Transfer, J. Nuc Eng & Design.
IJHFF, FTAC, Computers & Fluids.
• 40+ peer reviewed international conference papers
– TSFP4,5,6,7 THMT, TI, ETMM5,6,7,8, Congress of Advances in Nuclear Power
Plants, Int. Conf on Nuclear Engineering, Parallel CFD …
• 5 Springer online book publications
– DESider, FLOMANIA, Proceedings
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 26/33
Dissemination: Teaching
• Teaching to UG/PGT in MACE
• Masters level (yr 4)
– In class tutorials.
• ~24 students/year
– Dissertation topics
• Undergraduate (yr 3)
– Individual projects
– Summer projects
• Contributing validation results
– to the validation database
– e.g. benchmarking commerical
codes
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 27/33
Dissemination: Training
• Code_Saturne 2 day Training course (July 4/5 2011)
– organised and led by 2 PDRAs: Juan Uribe & Stefano Rolfo
• all supervision was done by PhD students!
– external participants:
• Senior experts from EDF Energy- UK, SERCO, Frazer Nash (nuclear industry)
– internal participants
• new research groups : 4 new PhD students moving from FLUENT (due to cost!)
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 28/33
Expansion of Code_Saturne
1. inter-disciplinary expansion into Electrical/Maths/Comp. Sci
–
GPUs, Transformer cooling; Materials; FSI
•
contributes to development of multiphysics
2. Installed at UK Universities
–
–
Leeds, Newcastle, Wolverhampton.
Edinburgh, Aberdeen,
3. Industry
–
growing links with industry as PhDs ‘spread the news’
•
dissemination via training sessions
4. Overseas
–
dissemination via EU projects (DESider, ATAAC, PRACE, HPC Europa)
the open-source bug is spreading on all levels !!
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 29/33
Open Source - Benefits
• accessibility: users gain real insight into the code
– offers excellent framework for research
• daily developer experience vs. button clicking
• kudos / pride
– students ‘appreciate the open-source model’
• really excites and energises them
• perceived as ‘real’ contribution
– ‘Free your CFD’ (http://code-saturne.blogspot.com/)
• personal website by one PhD student 80000+ hits from 100+ countries
• recognition from funders
– proposal reviewers stated : open source is fantastic for scientific dissemination
– commercial companies appreciate the skilled graduates we generate
• relationship with Star CD
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 30/33
Open Source - Challenges
• retaining expertise of PhD and PDRA
– academia/industry should be aware of their skills
• tempting established academics away from their in-house codes
– can be difficult to leave 15+ years of development
• academic environment doesn’t allow much free time
• open source : support
– forum is very difficult to sustain
– support is not free even in open source!
• many students/staff contribute to this in their own time
– some open source companies charge for support
• integration of developments from manchester
– can be difficult to track 10+ versions of the code in academic
environment
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 31/33
Conclusion
• Research projects using Code_Saturne
– A range of EU and UK funded research programs
– A growing network of direct collaborations
• PhD work
– Vital support role of PDRA enabling large PhD network
– Wide range of multi disciplinary topics
• Wide dissemination and uptake of Code_Saturne
– within School of MACE, Faculty, University, Nationally and beyond
• Stage 1: internal building of ‘team critical mass’ : achieved
• Stage 2: external dissemination of expertise : in progress
– we have a rich source of expert PhD graduates for industry
– where will they go?
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 32/33
Thanks!
• Big Thanks to EDF for the invitation and all their support
• particularly Frederic Archambeau, David Monfort & the Code_Saturne team
• Thanks to the team at Manchester for all their hard work
• particularly Dominique Laurence, Juan Uribe, Stefano, Flavien, Imran
University of Manchester: Alistair Revell
EDF R&D industrielle d'expertise et open source: Juillet 2011
Slide 33/33