Mälardalen University Doctoral Dissertation 399
Jerol Soibam MACHINE LEARNING TECHNIQUES FOR ENHANCED HEAT TRANSFER MODELLING
2024
ISBN 978-91-7485-625-5
ISSN 1651-4238
Address:
P.O. Box 883, SE-721 23 Västerås. Sweden
P.O. Box 325, SE-631 05 Eskilstuna. Sweden
E-mail: info@mdu.se Web: www.mdu.se
Machine Learning Techniques
for Enhanced Heat Transfer
Modelling
Jerol Soibam
!" #$ %&'(')
*+,-+),./'..
*-/.-)',
% 01&2 Mälardalen University Press Dissertations
No. 399
MACHINE LEARNING TECHNIQUES FOR ENHANCED HEAT TRANSFER MODELLING
Jerol Soibam
Akademisk avhandling
som för avläggande av teknologie doktorsexamen i energi- och miljöteknik vid
Akademin för ekonomi, samhälle och teknik kommer att offentligen försvaras
tisdagen den 13 februari 2024, 09.00 i Delta, Mälardalens universitet, Västerås.
Fakultetsopponent: Andrea Ianiro, University Charles III of Madrid
Akademin för ekonomi, samhälle och teknik
Abstract
With the continuous growth of global energy demand, processes from power generation to electronics
cooling become vitally important. The role of heat transfer in these processes is crucial, facilitating
effective monitoring, control, and optimisation. Therefore, advancements and understanding of heat
transfer directly correlate to system performance, lifespan, safety, and cost-effectiveness, and they serve
as key components in addressing the world's increasing energy needs.
The field of heat transfer faces the challenge of needing intensive studies while retaining fast
computational speeds to allow for system optimisation. While advancements in computational power
are significant, current numerical models lack in handling complex physical problems such as illposed. The domain of heat transfer is rapidly evolving, driven by a wealth of data from experimental
measurements and numerical simulations. This data influx presents an opportunity for machine learning
techniques, which can be used to harness meaningful insights about the underlying physics.
Therefore, the current thesis aims to the explore machine learning methods concerning heat transfer
problems. More precisely, the study looks into advanced algorithms such as deep, convolutional,
and physics-informed neural networks to tackle two types of heat transfer: subcooled boiling and
convective heat transfer. The thesis further addresses the effective use of data through transfer learning
and optimal sensor placement when available data is sparse, to learn the system behaviour. This
technique reduces the need for extensive datasets and allows models to be trained more efficiently. An
additional aspect of this thesis revolves around developing robust machine learning models. Therefore,
significant efforts have been directed towards accounting for the uncertainty present in the model,
which can further illuminate the model’s behaviour. This thesis shows the machine learning model's
ability for accurate prediction. It offers insights into various parameters and handles uncertainties and
ill-posed problems. The study emphasises machine learning's role in optimising heat transfer processes.
The findings highlight the potential of synergistic application between traditional methodologies and
machine learning models. These synergies can significantly enhance the design of systems, leading to
greater efficiency.
ISBN 978-91-7485-625-5
ISSN 1651-4238
To my family,
Acknowledgements
I would like to express my deep gratitude to my principal supervisor, Prof.
Rebei Bel Fdhila. His guidance and unwavering support have been invaluable throughout my research journey. I am particularly grateful for his
confidence in my ideas and the freedom he has given me to explore them.
I am thankful to my co-supervisor, Prof. Konstantinos Kyprianidis, for his
guidance and encouragement. His innovative ideas and thought-provoking
discussions have greatly enriched my research. His insightful guidance
has shaped my academic journey, offering constant inspiration and learning opportunities.
Profound thanks to my co-supervisor, Assoc Prof. Ioanna Aslanidou,
whose support has been crucial both academically and personally during
my PhD journey. Her encouragement in times of challenge was a beacon
of hope, keeping me focused and resilient. Working with her has been an
exceptionally positive and insightful experience.
I would also like to express my gratitude to my manager, Dr. Anders
Avelin, for his extensive support and guidance throughout my PhD journey. His encouragement to pursue new opportunities has been invaluable.
A heartfelt thanks to my colleagues in the department for the meaningful
conversations during lunch and fika breaks, which have significantly enhanced my experience throughout my time at the university.
I seize this moment to express my gratitude to Amare, Valentin, and Achref
for engaging conversations and enjoyable collaboration we have had.
I also take this opportunity to thank Dimitra-Eirini Diamantidou, who has
stood by my side through thick and thin. Her unwavering support and love
have been a constant source of motivation, especially in times when I lost
hope in myself. Thank you for the enriching conversations on life as well
as research, and for the wonderful trips we have had over the years. I look
forward to more of them.
Mälardalen University Press Dissertations
i
I extend my immense gratitude to my siblings and parents, who have been
my greatest support in life. Thank you for constantly believing in me, for
your endless patience, and for the warmth that our family shares. Your
love, sacrifices, and encouragement have given me the strength to keep
moving forward and to make the best out of every situation. I am eternally
grateful for the faith you have consistently shown in me.
Jerol Soibam
Västerås, Sweden, September 2023
ii
Jerol Soibam
Summary
The growing global energy demands and environmental impacts necessitate efficient industrial systems. The systems’ effectiveness heavily depends on the control of fluid flows and heat transfer. Methods such as
detailed numerical simulations have been utilised over the years, despite
their time-consuming nature and the need for intricate modelling. Such
simulations have generated significant volumes of data. The accumulated
data sets the stage for the application of machine learning techniques,
which provide a promising solution for real-time monitoring and management of heat transfer systems.
This thesis comprises several studies which focuses on the potential
use of machine learning for heat transfer problems. More specifically,
various machine learning architectures have been investigated to study
subcooled boiling and convective heat transfer. In the segment dealing
with boiling heat transfer, the research explores various methodologies
for predicting crucial parameters such as wall temperature and void fraction in subcooled flow boiling heat transfer. A deep neural network serves
as the core computational tool in this exploration. The model’s input incorporates the near local flow parameters of the heated minichannel under
subcooled boiling. The predictions obtained from the model demonstrate
notable agreement with numerical simulation data, reinforcing the efficacy of the employed machine learning techniques in modelling complex
thermal dynamics. Moreover, the study focuses on predicting the uncertainty present in the deep learning model while estimating the quantities
of interest.
The thesis also unveils a novel convolutional neural network model
for tracking bubble dynamics in a mini-channel, heated from one side.
This model utilises high-speed camera images as its primary input, despite
the inherent challenges in the form of shadows, reflections, and noise.
Through transfer learning, the model successfully detects and classifies
the bubble boundaries with impressive accuracy. Moreover, it can estimate local parameters delivering a detailed understanding of the spatialtemporal behaviour of the bubbles, an achievement that could be instrumental for various industrial applications.
The critical heat flux, a crucial parameter in the design of any industrial boiling system, is another focal point of this thesis. A parametric
Gaussian process regression approach is introduced to predict critical heat
Mälardalen University Press Dissertations
iii
flux with significantly reduced predictive error compared to traditional
empirical methods and look-up tables. This model provides robust insights and aligns well with the underlying physics, emphasising the potential of machine learning in overcoming the challenges of empirical correlation.
In the context of convective heat transfer, the thesis presents a strategy
to handle ill-posed boundary conditions when there is a lack of accurate
thermal boundary conditions. To tackle this challenge, the study employs
a physics-informed neural network. As the thermal boundary is unknown
near the source, the network has been optimised using only a few sensors points to discover the thermal boundary. It simultaneously represents
velocity and temperature fields while enforcing the Navier-Stokes and energy equations at random points within the domain, thereby functioning
as an inverse problem. The aim is to reproduce the global flow field and
temperature profile with sparse data. Furthermore, the research employs
transfer learning to account for different parameters, such as the position
and size of the source term within the enclosure domain.
To enhance the model’s robustness, the QR pivoting technique is employed to determine the optimal sensor location within the domain. Most
importantly, the sensors derived from QR pivoting ensure the effective
capture of the dynamics, leading to enhanced model accuracy when integrated with the physics-informed neural network. To further enhance the
model’s robustness and generalisability, an ensemble physics-informed
neural network is implemented. This method serves two essential purposes, it mitigates the risks associated with overfitting and underfitting,
which are prevalent in machine learning models. Furthermore, it accounts
for inherent uncertainty in model predictions, offering a measure of confidence in the model’s outputs. This acknowledgement and quantification
of uncertainty not only increases the robustness of the model but also enhances transparency, which refers to the model’s ability to indicate the reliability of its predictions. Consequently, the model can pinpoint regions
of reliable prediction and potential inaccuracies, broadening its applicability in addressing complex heat transfer problems with unknown boundary
conditions. The methods proposed in this study have demonstrated good
agreement with the underlying physics represented by numerical results.
In conclusion, this research paves the way for innovative solutions in
heat transfer studies, harnessing the capabilities of machine learning techniques. The results demonstrate that these methodologies can handle the
complexities and uncertainties inherent in predicting heat transfer phenomena, suggesting promising opportunities for further exploration and
potential industrial applications.
iv
Jerol Soibam
Sammanfattning
Den ökande globala energiefterfrågan och miljöpåverkan kräver effektiva
industriella system. Dessa systemens effektivitet beror i stor utsträckning på kontrollen av vätskeflöden och värmeöverföring. Under åren har
metoder som detaljerade numeriska simuleringar använts, trots att de är
tidskrävande och kräver komplicerad modellering. Dessa simuleringar har
genererat enorma datamängder. Den ackumulerade data lägger grunden
för användningen av maskininlärningstekniker, som erbjuder en lovande
lösning för realtidsövervakning och hantering av värmeöverföringssystem.
Denna avhandling omfattar flera studier som riktar in sig på den potentiella användningen av maskininlärning för värmeöverföringsproblem.
I avhandlingen har olika arkitekturer av maskininlärning undersökts för
att studera underkyld kokning och konvektiv värmeöverföring. I det avsnitt som handlar om kokningsvärmeöverföring utforskas olika metoder för
att förutsäga avgörande parametrar, såsom väggtemperatur och tomfraktion vid underkyld flödeskokning. Ett djupt neuralt nätverk är det centrala verktyget i denna utforskning. Modellen tar in data från närliggande
flödesparametrar i den uppvärmda minikanalen under underkyld kokning.
Förutsägelserna från modellen stämmer väl överens med numerisk simulering, vilket bekräftar maskininlärningsteknikernas effektivitet i att modellera komplexa termiska dynamiker. Vidare syftar studien till att förutsäga osäkerheten i den djupa inlärningsmodellen, vilket förstärker insikten
om modellens pålitlighet.
Avhandlingen introducerar även en innovativ modell baserad på konvolutionella neurala nätverk för att spåra bubblors dynamik i en minikanal,
uppvärmd från en sida. Modellen använder bilder från höghastighetskameror, trots utmaningar som skuggor, reflektioner och brus. Med hjälp av
överföringsinlärning kan modellen framgångsrikt detektera och klassificera bubblor med imponerande noggrannhet. Modellen ger en detaljerad
förståelse av bubblors rumsliga och temporala beteende, vilket kan vara
avgörande i industriella sammanhang.
En annan central del av denna avhandling är den kritiska värmeöverföringsflödet, en avgörande parameter i designen av industriella kokningssystem. En parametrisk Gaussian process regression-metod introduceras, som visat sig minska prediktiva fel jämfört med traditionella
metoder. Denna metod inte bara förstärker modellens robusthet utan ökar
Mälardalen University Press Dissertations
v
också transparensen, vilket understryker maskininlärningens potential att
bemöta empiriska korrelationers utmaningar.
Inom konvektiv värmeöverföring presenterar avhandlingen en strategi
för att hantera osäkra gränsförhållanden när det saknas exakta termiska
gränsbetingelser. Här används ett fysikinformerat neuralt nätverk som
kan representera både hastighets- och temperaturfält. Målet är att med
sparsam data återskapa både flödesfält och temperaturprofiler. För att
öka modellens tillförlitlighet och generaliserbarhet föreslås även en ensemblemetod för det fysikinformerade neurala nätverket. Denna metod
adresserar riskerna med överanpassning och ger en mätning av förtroendet för modellens utdata.
Sammanfattningsvis ger denna forskning nya möjligheter för innovativa lösningar inom värmeöverföringsstudier genom att använda maskininlärning. Resultaten understryker dessa metodologiers förmåga att hantera de komplexa utmaningarna och osäkerheterna i att förutsäga värmeöverföringsfenomen, vilket pekar på lovande framtida utforskningar och
industriella tillämpningar.
vi
Jerol Soibam
List of papers
Publications included in the thesis
This thesis is based on the following papers, which are referred to in the
text by their roman numerals:
I. Soibam, J., Aslanidou, I., Kyprianidis, K, and Bel Fdhila. R. (2020).
A Data-Driven Approach for the Prediction of Subcooled Boiling
Heat Transfer. In Proc. of the 61st International Conference of
Scandinavian Simulation Society, SIMS-36, September 22-24, 2020,
Oulu, Finland.
II. Soibam, J., Rabhi, A., Aslanidou, I., Kyprianidis, K, and Bel Fdhila. R.(2020). Derivation and Uncertainty Quantification of a
Data-Driven Subcooled Boiling Model. Multidisciplinary Digital
Publishing Institute (MDPI), Energies 13 (22), 5987, 2020.
III. Soibam, J., Rabhi, A., Aslanidou, I., Kyprianidis, K, and Bel Fdhila. R. (2021). Prediction of the Critical Heat Flux using Parametric Gaussian Process Regression. In proc. of the 15th International
Conference on Heat Transfer, Fluid Mechanics, and Thermodynamics, HEFAT, 26-28 July 2021, Amsterdam, Netherlands.
IV. Soibam, J., Aslanidou, I., Kyprianidis, K, and Bel Fdhila. R. (2023).
Inverse flow prediction using PINNs in an enclosure containing heat
sources. 8th Thermal and Fluids Engineering Conference , ASTFE,
27-30 March 2023, Maryland, USA.
V. Soibam, J., Scheiff, V., Aslanidou, I., Kyprianidis, K, and Bel Fdhila. R. (2023). Application of deep learning for segmentation of
bubble dynamics in subcooled boiling. International Journal of
Multiphase Flow, 2023.
VI. Soibam, J., Aslanidou, I., Kyprianidis, K, and Bel Fdhila. R. (2023).
Inverse flow prediction using ensemble PINNs and uncertainty quantification. International Journal of Heat and Mass Transfer (Under
Review), 2023.
Mälardalen University Press Dissertations
vii
The author’s contribution to the included publications
In all the appended papers, the author conceptualised, developed and performed the simulations, analysed the numerical results and wrote the drafts
and the final versions of the papers. The co-authors actively contributed
by enhancing the conceptual foundations and offered valuable insights to
improve the quality of the research.
Publications not included in the thesis
• E. Helmryd Grosfilley, G. Robertson, J. Soibam, and J.-M. Le Corre.
Investigation of Machine Learning Regression Techniques to Predict Critical Heat Flux over a Large Parameter Space. 20th International Topical Meeting on Nuclear Reactor Thermal Hydraulics
(NURETH-20), Washington, D.C., USA, Aug. 20 – 25, 2023.
• Aslanidou, I., Soibam, J. Comparison of machine learning approaches
for spectroscopy applications. In Proc. of the 63st International
Conference of Scandinavian Simulation Society, SIMS-63, Trondheim, Norway, September 20-21, 2022.
viii
Jerol Soibam
Contents
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . .
i
Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Swedish summary . . . . . . . . . . . . . . . . . . . . . . . . . . v
List of papers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
List of figures . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
List of tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Nomenclature . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
1 I NTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Challenges and motivation . . . . . . . . . . . . . . . . . . . 4
1.3 Research approach . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 Research framework . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Summary of appended papers . . . . . . . . . . . . . . . . . 9
1.6 Contribution to knowledge . . . . . . . . . . . . . . . . . . . 12
1.7 Thesis outline . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2 T HEORETICAL BACKGROUND . . . . . . . . . . . . . . . . . .
2.1 Overview of machine learning . . . . . . . . . . . . . . . . .
2.2 Application of ML in heat transfer and fluid dynamics . . . .
2.2.1 Data mining and processing . . . . . . . . . . . . . . . .
2.2.2 Control and optimisation . . . . . . . . . . . . . . . . . .
2.2.3 Flow modelling . . . . . . . . . . . . . . . . . . . . . . .
15
15
16
17
18
19
3 M ETHODOLOGY . . . . . . . . . . . . . . . . . . . . . . . . .
3.1 Framework of the thesis . . . . . . . . . . . . . . . . . . . .
3.2 Data availability . . . . . . . . . . . . . . . . . . . . . . . .
3.3 Architectures . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Deep neural network . . . . . . . . . . . . . . . . . . . .
3.3.2 Physics informed neural network . . . . . . . . . . . . . .
3.3.3 Convolutional neural network . . . . . . . . . . . . . . .
3.3.4 Gaussian process regression . . . . . . . . . . . . . . . .
3.4 Uncertainty quantification for robustness . . . . . . . . . . .
23
23
24
26
26
28
29
31
32
Mälardalen University Press Dissertations
ix
3.4.1
3.4.2
Monte Carlo dropout . . . . . . . . . . . . . . . . . . . . 32
Deep ensemble . . . . . . . . . . . . . . . . . . . . . . . 33
4 R ESULTS AND DISCUSSION . . . . . . . . . . . . . . . . . . .
4.1 Case study on boiling heat transfer . . . . . . . . . . . . . . .
4.1.1 DNN for subcooled boiling . . . . . . . . . . . . . . . . .
4.1.2 Computer vision for subcooled boiling . . . . . . . . . .
4.1.3 Critical heat flux . . . . . . . . . . . . . . . . . . . . . .
4.2 Case study on convective heat transfer . . . . . . . . . . . . .
4.2.1 Forced convection . . . . . . . . . . . . . . . . . . . . .
4.2.2 Mixed convection . . . . . . . . . . . . . . . . . . . . . .
4.3 Discussion of the research questions . . . . . . . . . . . . . .
5
35
35
35
38
41
43
43
46
50
C ONCLUSIONS AND FUTURE WORK . . . . . . . . . . . . . . . 57
B IBLIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
PAPERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
x
Jerol Soibam
List of figures
1.1
1.2
1.3
1.4
2.1
2.2
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
4.1
4.2
4.3
4.4
Brief overview of uses of machine learning for heat transfer
and fluid dynamics. . . . . . . . . . . . . . . . . . . . . . . . 3
Research approach adopted in this thesis. . . . . . . . . . . . 7
Holistic overview of the research framework employed in the
thesis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Schematic diagram of the relationship between the research
questions and the appended papers. . . . . . . . . . . . . . . . 12
Research trend of using machine learning for heat transfer and
fluid dynamics (Source: scopus & (Ardabili et al., 2023)) . . . 15
Potential applications of ML in the domain of heat transfer
and fluid mechanics. . . . . . . . . . . . . . . . . . . . . . . 16
Detailed framework of the learning system. . . . . . . . . . .
Numerical computational domain and experimental setup. . .
Numerical computational domain for forced and mixed convection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Deep neural network architecture. . . . . . . . . . . . . . . .
PINN framework for forced and mixed convection. . . . . . .
Convolutional neural network for bubble segmentation and
tracking. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Framework for parametric Gaussian process regression. . . . .
Monte Carlo dropout architecture with a dropout ratio of 20%
for training and testing. . . . . . . . . . . . . . . . . . . . . .
Deep ensemble architecture for M = 2. . . . . . . . . . . . .
Temperature field prediction and the relative error between
the CFD and DE model of the minichannel. . . . . . . . . . .
Void Fraction field prediction and the relative error between
the CFD and DE model of the minichannel. . . . . . . . . . .
Wall void fraction profile by the DNN, MC dropout and DE
models along the arc of the minichannel . . . . . . . . . . . .
Comparison of mask between ground truth (human eyes), classical image processing and CNN model. a) Ground truth mask
b) Classical mask c) CNN mask d) Normal image with a surface void fraction of 6.45% e) IoU for classical method and f)
IoU for CNN model . . . . . . . . . . . . . . . . . . . . . .
Mälardalen University Press Dissertations
23
25
26
27
29
30
31
33
34
36
37
37
38
xi
4.5
Masks given by the model with different noise conditions: a)
Original image with the mask, b) Contrast enhancement, c)
Sharpening, d) Top-hat filter with disk shape radius = 20, e)
Gaussian Blur σ = 0.8, f) Gaussian noise σ = 10, g) Gaussian
noise σ = 25, h) Black dead pixel 0.02% and i) IoU for noisy
images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.6 Results extracted from the binary mask given by CNN model
to study local bubbles behaviour: a) Trajectory of the bubbles
with velocity magnitude and growing bubble images (Note:
axes are not in scale), b) Bubble diameter with time, c) Bubble Reynolds number with time. . . . . . . . . . . . . . . . .
4.7 Validation result of the CHF for pGPR and LUT predicted
values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.8 Weighting factor and prediction obtained from pGPR model .
4.9 Case studies for different sensor placements to infer temperature distribution in the enclosure domain . . . . . . . . . . . .
4.10 Case studies for different sensor placement to infer temperature distribution in the enclosure domain . . . . . . . . . . . .
4.11 Temperature distribution: Transfer learning case study for two
rectangular sources . . . . . . . . . . . . . . . . . . . . . . .
4.12 Comparision of L2 relative error between QR and randomly
selected sensors over time for the component u∗ and θ ∗ . . . .
4.13 Sensitivity of QR sensors and uncertainty quantification of
PINN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.14 Sensitivity of QR sensors and uncertainty quantification of
PINN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.15 Comparison of flow field profiles for constant heat flux at t =
80, as predicted by PINN and DNS, with corresponding L2
error for u∗ , v∗ , p∗ , θ ∗ . . . . . . . . . . . . . . . . . . . . . .
4.16 Predictive performance of the PINN at random point which
was not a part of QR sensor nor training . . . . . . . . . . . .
xii
39
40
42
42
44
45
45
46
47
48
49
50
Jerol Soibam
List of tables
1.1
Overview of appended papers presented in this thesis. Sub.
boiling: Subcooled boiling, CHF: Critical Heat Flux & Conv.:
Convection . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.1
Data split for training, validation, and testing at different heat
flux and flow conditions. . . . . . . . . . . . . . . . . . . . . 24
4.1
Performance of the DNN, MC dropout, and DE models on
validation dataset of the computational domain, VF: Void Fraction, Temp: Temperature. . . . . . . . . . . . . . . . . . . . . 35
Mälardalen University Press Dissertations
xiii
Nomenclature
Abbreviations
Adam
Adaptive Moment Estimation
ANN
Artificial Neural Network
CFD
Computational Fluid Dynamics
CNN
Convolutional Neural Network
DE
Deep Ensemble
DL
Deep Learning
DMD
Dynamic Mode Decomposition
DNN
Deep Neural Network
DNS
Direct Numerical Simulation
DRL
Deep Reinforcement Learning
GA
Genetic Algorithms
GPR
Gaussian Process Regression
GPU
Graphical Processing Unit
HPC
High Performance Commutating
LES
Large Eddy Simulation
MC
Monte Carlo
MCMC
Markov chain Monte Carlo
ML
Machine Learning
MSE
Mean Square Error
NLL
Negative Log-Likelihood
NN
Neural Network
PCA
Principal Component Analysis
PDE
Partial Differential Equation
pGPR
parametric Gaussian Process Regression
xiv
Jerol Soibam
PINN
Physics Informed Neural Network
PIV
Particle Image Velocimetry
POD
Proper Orthogonal Decomposition
QoIs
Quantities of Interest
RANS
Reynolds Average Navier-Stokes
ReLU
Rectified Linear Units
RMSEP
Root Mean Square Error Prediction
ROI
Region of Interest
ROM
Reduced Order Model
RQ
Research Question
SA
Spallart–Allmaras
UQ
Uncertainty Quantification
VF
Void fraction
YOLO
You Only Look Once
Machine learning symbols
ŷ
Predicted value
λ
Hyperparameter
λw
Weight decay
E(y∗ )
Predictive mean
L
Loss function
τ
Gaussian process precision
b
Biases
g(x)
Activation function
l
Length scale
Lr
Learning rate
M
Number of networks
N
Size of the dataset
p
Probability of neurons not dropped
R2
Coefficient of determination
Mälardalen University Press Dissertations
xv
T
Repetition of the prediction step
Var(y∗ )
Predictive variance
W
Weights
x
Input signals
X∗
Input features
x∗
New dataset
y
Target value
L1
Lasso regression
L2
Ridge regression
Physics symbols
α
Void fraction
μ
Dynamic viscosity
θ∗
Non-dimensional temperature
l
Non dimensional arc length
p
Pressure
p∗
Non-dimensional pressure
Pec
Péclet number
q
Heat flux
Re
Reynolds number
Ri
Richardson number
T
Fluid temperature
t∗
Non-dimensional time
Twall
Wall temperature
u
Velocity in x direction
u∗
Non-dimensional velocity in x direction
v
Velocity in y direction
v∗
Non-dimensional velocity in p direction
x
Minichannel length in x axis
y
Minichannel length in y axis
xvi
Jerol Soibam
Preface
This doctoral thesis was completed at the Future Energy Center (FEC)
at the School of Business, Society and Engineering (EST) at Mälardalen
University in partial fulfilment of the requirements for acquiring the degree of Doctor of Philosophy in Energy and Environmental Engineering.
The study was carried out under the guidance of Prof. Rebei Bel Fdhila, Prof. Konstantinos Kyprianidis and Assoc Prof. Ioanna Aslanidou.
The thesis summarises the study conducted by the author during the PhD
project, which started on 18 June 2019. It also provides a concise overview
of the six enclosed publications, while the comprehensive research findings are available in the appendix.
Mälardalen University Press Dissertations
xvii
1
Introduction
This chapter introduces the research background, challenges, and motivations. It formulates research questions from literature gaps. Thereafter, it
presents the research methodology and framework, detailing the links between research topics and appended papers. The chapter concludes with
the thesis outline.
1.1
Background
Heat transfer is a fundamental process found in numerous sectors, ranging
from the cooling of electronic devices, supercomputers, and the functioning of nuclear reactors, to the design of industrial processes and equipment. It is characterised by a multitude of complex phenomena that span
a broad range of scales, including molecular, microscale, and macroscale.
The very essence of heat transfer plays a crucial role in the broader context of energy production, management, and consumption. Every device,
mechanism, or process that involves the production or consumption of
energy invariably incorporates elements of heat transfer. This includes
traditional forms of energy production such as combustion engines and
thermal power plants, as well as modern, renewable energy technologies
such as solar thermal and geothermal energy systems.
The ability to monitor and control the heat transfer process in a system
is critical for optimal performance, longevity, safety, and cost-effectiveness
in a wide range of applications. Moreover, as the world becomes increasingly energy-conscious, efficient management of heat has profound implications for energy conservation and sustainability. Improved heat transfer
systems can lead to significant energy savings, reduce environmental impact, and help meet global energy demand in a sustainable manner. For
instance, enhancing heat transfer efficiency in power generation and industrial processes could greatly reduce energy consumption and carbon
footprint.
Over the years, significant advancements have been made in understanding heat transfer principles, largely due to concerted efforts in experimental research and numerical modelling. These areas of research
have offered unique contributions that have synergistically expanded the
knowledge base. The experimental and modelling methods have generated a large volume of data over the years, enriching the scientific comMälardalen University Press Dissertations
1
Machine Learning Techniques for Enhanced Heat Transfer Modelling
munity with invaluable resources for future research and validation. However, despite these substantial advancements, modelling detailed physical problems for industrial applications remains a challenging endeavour
(Karniadakis et al., 2021). Accurately depicting heat transfer problems
often necessitates substantial computational time, a fine-tuned mesh grid
and well-defined boundary conditions (Alifanov, 2012). Moreover, setting up an accurate numerical simulation is a complex, time-consuming
process that depends on multiple factors such as defining geometry, generating mesh, selecting solvers, post-processing, and interpreting results
in a meaningful way (Molland and Turnock, 2007). Furthermore, when
these methods are subject to ill-posed problem conditions or high levels
of uncertainty, the validity of the model becomes questionable (Colaco
et al., 2006). The sensitivity of the solution can vary drastically with even
a slight change in the input parameters or the initial conditions, potentially
rendering the results less reliable or unrepeatable.
Despite these hurdles, a distinct opportunity arises from the immense
volume of data accrued from years of experimental research and numerical
modelling. This data accumulation provides an ideal basis for the application of emerging analytical tools. Specifically, the introduction of machine learning (ML) into the field of heat transfer and fluid mechanics is a
notable development (Brunton et al., 2020), reminiscent of the paradigm
shift brought about by the advent of high-performance computing (HPC)
decades ago. Just as HPC expanded the boundaries of research capability,
enabling larger and more intricate problem-solving, ML has the potential
to make a comparable, if not greater, impact. The application of ML in this
context can be used to process and learn from vast datasets, and presents
a potent approach for addressing complex and often ill-posed problems
in heat transfer and fluid mechanics (Taira et al., 2017, 2020). Moreover,
in scenarios where data is limited, machine learning holds another distinct advantage – the ability to integrate physics-based principles into the
models. This unique fusion of data-driven learning with physics-informed
insights allows for the creation of robust and reliable models even when
data is scarce (Parish and Duraisamy, 2016). By embedding the foundational principles of physics into the learning process, these models not
only adapt to data but also remain consistent with the underlying physical
laws. This approach enhances the robustness of the model, thereby further
extending the utility and impact of ML in the study of heat transfer and
fluid mechanics (Raissi et al., 2019b).
Within the domain of heat transfer and fluid mechanics, recent years
have witnessed an escalating interest in the potential of ML methods to
augment engineering solutions as shown in Figure 1.1. Utilising ML or
data-driven techniques within fluid mechanics isn’t a recent development.
2
Jerol Soibam
Chapter 1. Introduction
Figure 1.1: Brief overview of uses of machine learning for heat transfer and
fluid dynamics.
Indeed, these methods were explored in previous decades (Pollard et al.,
2016; Brunton et al., 2020). However, compared to the modern ML techniques, earlier studies were not as impactful. This transformation has been
driven by the current intersection of big data, high-performance computing, sophisticated algorithms, and significant industrial investment, leading to open-source tools and accessible benchmark datasets. An influential development in the contemporary ML age is the emergence of deep
learning (DL) (Goodfellow et al., 2016), achieving human-level performance in several demanding tasks, such as image recognition (Krizhevsky
et al., 2012) and control systems (Mnih et al., 2015; Silver et al., 2018).
The success of these data-driven techniques has primarily relied on extensive datasets (Deng et al., 2009), open-source algorithms promoting reproducible research, and powerful computational resources like graphical
processing unit (GPU) hardware.
In recent decades, substantial progress in experimental methods and
large-scale numerical simulations have made big data in fluid mechanics and heat transfer a reality (Pollard et al., 2016). Despite this, the
field of fluid mechanics remains primarily focused on the physical mechanisms underlying fluid phenomena. ML frameworks have proven crucial
in instances where these mechanisms are challenging to investigate. A
prominent example is the use of Proper Orthogonal Decomposition (POD)
(Rowley and Dawson, 2017) to examine near-wall flow structures at varying Reynolds numbers in turbulent channel flow (Podvin et al., 2010). The
Mälardalen University Press Dissertations
3
Machine Learning Techniques for Enhanced Heat Transfer Modelling
past decade has seen impressive advances in ML, spurred by commercial
successes in natural language processing and advertising sectors (Otter
et al., 2021). With the confluence of these technologies, the techniques
become relevant for heat transfer and engineering challenges, enabling
the engineering field to capitalise on these potent methods. A promising
aspect of the area of heat transfer and fluid dynamics is that these burgeoning techniques are robust enough to handle large datasets and depict
intricate nonlinear dynamics frequently encountered in these fields (Brunton et al., 2020).
1.2
Challenges and motivation
The rapid advancement of ML or DL techniques has the potential to significantly improve the solutions to engineering problems, particularly in
the areas of heat transfer and fluid mechanics. However, these domains
require extensive research before the full potential of ML/DL can be exploited. Precise quantification of underlying physical mechanisms in heat
transfer and fluid dynamics is crucial for their proper analysis. These
fields exhibit complex, multiscale phenomena, presenting particular challenges. For instance, turbulence flow fields introduce significant complexities due to their nonlinearities and multiple spatiotemporal scales, which
may not be readily addressed by currently available ML algorithms. In
addition, another notable challenge is to ensure the alignment of ML models with the conservation laws of physics. Traditional ML models might
produce physically inconsistent results, leading to inaccurate and untrustworthy predictions. To solve this problem, researchers are exploring techniques that incorporate physical laws and principles into the training process of ML models.
ML models fundamentally rely on data-driven mechanisms to formulate informed predictions or decisions. The efficacy of these ML models
is inherently contingent upon the quality and volume of the available data
(Iten et al., 2020). In situations where data is either limited or fails to
represent a wide spectrum of instances, the performance of ML models
may be critically compromised. The scarcity or lack of heterogeneous
data could lead to an inadequate learning experience for the models, potentially jeopardising their predictive accuracy and decision-making capabilities (Brunton et al., 2016). Moreover, it’s pivotal to acknowledge
situations in which data collection may not be practical or even possible (Schmidt and Lipson, 2009). Such situations introduce an extra layer
of complexity to the efficient training of ML models. Restrictions could
originate from a variety of sources, including logistical difficulties, safety
4
Jerol Soibam
Chapter 1. Introduction
protocols, privacy regulations, or the lack of requisite technological methods or tools for data collection.
A significant obstacle presented by these machine ML models is the
inherent lack of transparency, or interpretability (Lipton, 2018). Unlike
traditional numerical simulations, which often follow a clear and understandable set of steps, ML algorithms frequently function as a sort of
’black box’. In other words, while these models are capable of generating
predictions, the underlying logic or reasoning that drives these predictions
is often elusive or even hidden. This characteristic of ML models creates a
significant barrier, particularly in disciplines such as heat transfer and fluid
dynamics. In these fields, the comprehension of the underlying physical
phenomena is not just desirable but often essential. The ’black box’ nature of ML models makes it challenging to understand these phenomena
fully and therefore can limit their applicability in these areas. Therefore,
it becomes paramount to test the validity and robustness of these models
before they can be integrated into engineering systems. It is not enough
for a model to just make accurate predictions; it must also be reliable
and resistant to potential changes in the input data or underlying conditions (Zhang et al., 2019). Moreover, even a model that exhibits strong
predictive performance may not always meet the requirements of a given
situation. In such cases, quantifying the uncertainty associated with the
model’s predictions becomes necessary. Uncertainty quantification allows
us to understand the potential range and likelihood of different outcomes,
providing a fuller picture of the model’s capabilities and limitations (Yang
et al., 2020; Zhu et al., 2019). This, in turn, can inform decision-making
and risk management, adding a layer of depth and robustness to the use of
ML models in engineering and other fields.
Despite the challenges posed by the ’black box’ nature of ML models, these difficulties should not be perceived as insurmountable barriers.
Rather, they can serve as powerful catalysts, spurring the development
and refinement of more effective and efficient ML techniques. Indeed,
applying ML to fields with known physical principles, such as fluid dynamics and heat transfer, has the potential to solve complex problems in
these areas while also contributing to a deeper understanding of the underlying mechanics of ML algorithms (Brunton et al., 2020). The interplay
between ML and these traditionally physics-based fields can cultivate a
mutually beneficial relationship. For instance, the use of known physical
laws can be a valuable resource during the training of neural networks
(Karniadakis et al., 2021). These laws can provide crucial guidance to the
learning process of the model, ensuring that the networks’ predictions do
not defy the physical principles they are supposed to model. This integration of physical laws with ML may not only improve the model’s preMälardalen University Press Dissertations
5
Machine Learning Techniques for Enhanced Heat Transfer Modelling
dictive accuracy, but also its interpretability, since the predictions would
align with established physical understanding. Therefore, despite its challenges, ML represents a powerful tool with a unique capacity to handle
high dimensional, nonlinear problems common in fields like heat transfer.
The journey towards overcoming its current limitations can also open up
opportunities for enhancing the capabilities of ML models, thereby advancing the broader fields of artificial intelligence and data science.
Acknowledging the challenges and opportunities outlined, it is apparent that the utilisation of ML/DL techniques in the field of heat transfer
and fluid mechanics necessitates comprehensive investigation. The intricate complexities which characterise these fields demand a precise exploration into the capabilities and limitations of such techniques. A more
detail on the literature and gaps can be found in Chapter 2. The issues surrounding data availability and quality, alignment with physical laws, and
model interpretability play pivotal roles in shaping the efficacy of ML/DL
methods. The current scenario thus raises several research questions that
require attention to further push the boundaries of ML applications in heat
transfer. Consequently, this thesis focuses on the following research questions (RQ):
• RQ1: How effectively can deep learning techniques capture the
intricate nature of heat transfer based on data, and how successfully
can these techniques extrapolate to unseen scenarios?
• RQ2: How to incorporate known physics of heat transfer in deep
learning models to improve the generalisability in estimating unknown parameters and system behaviours?
• RQ3: How can machine learning methods secure reliable prediction outcomes in situations where heat transfer data are scarce?
• RQ4: How can uncertainty quantification contribute to enhancing
the robustness and reliability of machine learning models utilised to
simulate and predict heat transfer?
The formulated research questions are expressed in a general manner, however, the conclusions drawn are based on selected case studies
within the research framework presented in Section 1.4. The above research questions are further discussed in detail in Section 4.3.
1.3
Research approach
The research approach followed in this thesis is an iterative cycle as depicted in Figure 1.2. The process starts with the "Research Objectives,"
6
Jerol Soibam
Chapter 1. Introduction
Figure 1.2: Research approach adopted in this thesis.
which provide the foundation for the entire thesis. These objectives define
the goals and aim of the research, offering a clear direction for the investigation. Upon setting clear and achievable research objectives, the next
step is to formulate preliminary "Research Questions". These inquiries
are derived from the research objectives, focusing on and specifying the
areas of investigation. Once the initial research questions are established,
a comprehensive "Literature Review" is conducted. This step involves a
systematic analysis and interpretation of the existing body of knowledge
related to the research topic. The purpose of this review is to identify
gaps in current knowledge, understand the context of the research, and
further refine the research questions and objectives. Following the literature review, the "Research Framework" is developed. This serves as the
blueprint for the research, outlining the theoretical foundations, hypotheses or assumptions, and methodologies for data collection and analysis.
The framework provides a roadmap for how the research questions will
be answered. The last phase in this research cycle, called as "Data Collection," is the pivotal cornerstone that supports the entire research. Here,
data on heat transfer is gathered from a variety of sources, including experiments, numerical models, sensors, and existing literature. This data
then acts as the catalyst for refining the research framework. It guides
the selection of the most fitting ML algorithm and prompts the adjustment
of the other stages in the research process. The process is iterative, with
each stage revisited and refined based on the data until the final research
questions are precisely articulated. This approach ensures a thorough and
adaptable exploration of the research topic.
1.4
Research framework
The proposed research framework presents a comprehensive and adaptive strategy for ML model selection, development, and validation in the
Mälardalen University Press Dissertations
7
Machine Learning Techniques for Enhanced Heat Transfer Modelling
specific context of heat transfer data. Recognising the variability in data
availability across different tasks and domains, the framework provides a
robust plan for these variations, thereby optimising the performance and
reliability of ML models in heat transfer applications.
In scenarios where there is an abundance of heat transfer data, the
framework uses powerful architectures such as Deep Neural Networks
(DNNs) and Computer Vision models. These architectures, renowned for
their ability to model complex patterns and relationships, typically require
large datasets to achieve optimal performance. However, in many practical settings, large datasets may not be readily available. To navigate this
challenge, the framework introduces the concept of transfer learning. This
technique enables models, initially trained on extensive datasets, to be
fine-tuned for specific tasks using a smaller amount of heat transfer data,
thus capitalising on the generalised knowledge extracted from the primary
dataset.
On the contrary, when dealing with scarce heat transfer data, the research framework employs methods that can operate efficiently with smaller datasets. These include Physics-Informed Neural Networks (PINNs)
and Gaussian Processes. By incorporating domain knowledge related to
heat transfer or exploiting the inherent structure of the data, these methods
can yield substantial results even with limited data. This strategy becomes
particularly beneficial in scenarios where the collection of heat transfer
data is costly, time-consuming, or otherwise challenging.
To enhance the robustness and transparency of the developed models,
the research framework integrates the principle of Uncertainty Quantification (UQ). UQ provides a measure of the model’s confidence in its predictions, thus increasing the model’s validity and interpretability. This component is of significant importance in contemporary heat transfer appli-
Figure 1.3: Holistic overview of the research framework employed in the thesis.
8
Jerol Soibam
Chapter 1. Introduction
cations, where understanding the uncertainty associated with predictions
can critically aid in decision-making processes.
In conclusion, the proposed research framework provides an adaptable
and resilient methodology for ML model selection and validation, specifically tailored for the varying conditions of heat transfer data availability.
This framework serves as the backbone of the investigation, guiding the
development and conclusions of the associated publications in addressing
the research questions.
1.5
Summary of appended papers
Paper I: A Data-Driven Approach for the Prediction of Subcooled Boiling Heat Transfer
This study explores a data-driven technique using a supervised deep
neural network (DNN) to understand boiling heat transfer in subcooled
boiling flow within a 3 x 400 mm minichannel. The model predicts wall
temperature and void fraction, using data from numerical simulations split
into training, validation, and testing datasets. The DNN model, validated
and evaluated through these datasets, effectively predicts these quantities
with high accuracy for both interpolation and extrapolation datasets. The
results demonstrate the model’s ability to reproduce the subcooled boiling
pattern and its satisfactory generalisation property.
Paper II: Derivation and Uncertainty Quantification of a Data-Driven
Subcooled Boiling Model
The paper explores two deep learning techniques for subcooled boiling heat transfer. The first predicts deterministic quantities of interest
(QoIs), while the second predicts uncertainties in the model when estimating QoIs, utilising methods like Monte Carlo dropout and Deep Ensemble.
The results demonstrated that these uncertainty quantification (UQ) models outperformed the deterministic model. Furthermore, they exhibited
strong performance on unseen and extreme extrapolation datasets. They
also accurately reproduced physics even at a heat flux beyond training parameters. On average, all models had a root mean square error percentage
(RMSEP) under 5% for wall temperature and under 2% for a void fraction
with R2 scores of 0.998 and 0.999 respectively.
Paper III: Prediction of the Critical Heat Flux using Parametric Gaussian Process Regression
Understanding the critical heat flux is crucial for industrial boiling system design and safety, but most existing studies rely on empirical knowledge, leading to a ± 30% predictive error. This study addresses this gap by
Mälardalen University Press Dissertations
9
Machine Learning Techniques for Enhanced Heat Transfer Modelling
using machine learning to uncover hidden features in experimental data,
enhancing accuracy and efficiency in data usage. Experimental data from
low-pressure, low-flow boiling flows in tubes, a parametric Gaussian process regression model is utilised to predict the critical heat flux. This
model’s predictions are compared to experimental measurements and values from a critical heat flux look-up table, resulting in improved prediction accuracy and insight into input parameter relevance. Besides offering
prediction uncertainty, the model aligns with the underlying physics and
demonstrates robustness, which entails potential extension to other geometries, datasets, and operating conditions.
Paper IV: Inverse flow prediction using PINNs in an enclosure containing heat sources
This study focuses on addressing an ill-posed problem in simulating
heat transfer due to the lack of precise thermal boundary conditions. To
overcome this issue, a physics-informed neural network is used, with a
few known temperature measurements in the domain. The network helps
simulate the velocity and temperature fields while enforcing the NavierStokes and energy equations. The study serves as an inverse problem,
aiming to recreate the global flow field and temperature profile with limited data points. It also explores transfer learning for various parameters
like the position and size of the heat source within the enclosure domain,
aiding in the effective design of thermal systems. The results indicate
good agreement between the proposed method and the physics reflected
by the numerical outcomes.
Paper V: Application of deep learning for segmentation of bubble dynamics in subcooled boiling
In this research, a convolutional neural network model is implemented
to monitor bubble dynamics in a heated vertical rectangular mini-channel.
The model utilises images captured by a high-speed camera, overcoming
noise challenges such as shadows, reflections, and chaotic bubbles. The
model is trained using transfer learning, which reduces the need for a large
dataset and computational resources. The validated model demonstrates
98% accuracy in bubble detection and is robust under various conditions.
Moreover, the model accurately identifies bubble edges and predicts bubble masks with an 85% average intersection over union, providing a detailed understanding of individual bubbles, including their coalescence,
oscillation, and collisions. This facilitates the estimation of local parameters and a comprehensive grasp of their spatial-temporal behaviour.
10
Jerol Soibam
Chapter 1. Introduction
Paper VI: Inverse flow prediction using ensemble PINNs and uncertainty quantification
Lack of accurate thermal boundary conditions in numerical simulations often leads to challenges in heat transfer problems. This study employs a physics-informed neural network to handle these ill-posed problems with limited sensor data. The network, complying with the NavierStokes and energy equations, reconstructs the flow field around a square
cylinder, identifying unknown thermal boundaries. Optimal sensor placement is achieved through the QR pivoting technique, improving model
accuracy. An ensemble physics-informed neural network is used to enhance robustness, generalisability, and to provide a measure of model uncertainty, hence improving applicability in handling complex heat transfer
problems with unknown boundaries.
Overview
The presented research papers cover a variety of case studies within the
areas of heat transfer. In addition, they utilise different types of data and
architectures. Therefore, this information is presented in the Table 1.1, to
improve the clarity of the each research paper’s focus. Finally, the link
between the publications and the RQs is shown in Figure 1.4.
Table 1.1: Overview of appended papers presented in this thesis. Sub. boiling:
Subcooled boiling, CHF: Critical Heat Flux & Conv.: Convection
Paper
Case study
Data type
Architecture
UQ
I
Sub. boiling
Numerical
(Rabhi et al., 2021)
DNN
No
II
Sub. boiling
Numerical
(Rabhi et al., 2021)
MC Dropout
Deep Ensemble
Yes
III
CHF
Experimental
(Kim et al., 2000)
pGPR
Yes
IV
Forced conv.
Numerical
Inverse PINN
Transfer learning
No
V
Sub. boiling
Experimental (Image)
(Scheiff et al., 2023)
CNN
Transfer learning
No
VI
Mixed Conv.
Numerical
(Fraigneau, Y., 2019)
Ensemble PINN
QR-pivoting
Yes
Mälardalen University Press Dissertations
11
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Figure 1.4: Schematic diagram of the relationship between the research
questions and the appended papers.
1.6
Contribution to knowledge
• Designed and implemented a deep neural network (DNN) architecture with the capability to address complex physical phenomena,
such as subcooled boiling. By successfully implementing a DNN
for heat transfer problems, the application of artificial neural networks to challenging scientific problems is expanded.
• Introduced an innovative computer vision technique for automated
bubble detection and estimation of statistical aspects of bubble dynamics. This technique provides a significant enabler to analyse and
understand fluid behaviour by automatically detecting and analysing
bubbles instead of the manual time-consuming image processing
and tracking.
• Proposed a physics-informed neural networks (PINN) while employing a sensor selection method (QR pivoting) to unveil unknown
parameters and reconstruct the flow field. This proof-of-concept
methodology enhances the efficiency and accuracy of the model,
especially in scenarios where sensor data plays a crucial role in
comprehending complex physical processes.
• Implemented a reliable model that accounts for uncertainty (Deep
Ensemble and Monte Carlo dropout) in both deep neural network
(DNN) and physics-informed neural network (PINN) architectures.
These models enhance the trustworthiness of predictions and simulations by recognizing and quantifying uncertainty.
12
Jerol Soibam
Chapter 1. Introduction
1.7
Thesis outline
The thesis is based on the appended papers and contains the following
chapters:
Chapter 1: Introduction
This chapter provides the research background, research challenges,
research questions formulated based on the knowledge gap, and research framework. It also shows the link to the appended papers and
the research questions in this thesis.
Chapter 2: Theoretical background
In this chapter a comprehensive literature review based on the use of
ML applications for heat transfer and fluid mechanics is presented.
The chapter also highlights the type of ML algorithms that can be
used for engineering problems.
Chapter 3: Methodology
This chapter presents a detailed ML framework used in this thesis.
Furthermore, it describes the data and architectures employed in this
thesis.
Chapter 4: Results and discussion
This chapter presents the key results obtained in this study, and provides a detailed discussion of the findings based on the research questions and limitations. The results are divided into boiling heat transfer and convective heat transfer.
Chapter 5: Conclusions and future work
This chapter presents the major conclusions of the thesis and key
contributions to the knowledge. The thesis ends by discussing potential future studies.
Mälardalen University Press Dissertations
13
2
Theoretical Background
In this chapter, a comprehensive literature review based on applications
of machine learning in the field of fluid mechanics and heat transfer is
provided.
2.1
Overview of machine learning
In recent years, ML, with a particular emphasis on deep learning, has
resulted in significant progress in tasks related to classification and regression. These methods are recognised for their inherent non-linearity
and adaptability across a broad range of applications. This versatility has
led to their use in a variety of fields, enabling the development of innovative solutions based on data in areas such as natural language processing,
computer vision, and robotics. More recently, these methodologies have
begun to leave a remarkable imprint on the natural sciences.
A trend that reflects this progress is the increasing application of ML
in the fields of heat transfer and fluid dynamics, as illustrated in Figure
2.1. The Figure reveals a sharp increase in research activity in this area
starting around 2015. This increasing surge of interest and research is
fundamentally driven by two intertwined factors. Firstly, the progressive
Figure 2.1: Research trend of using machine learning for heat transfer and fluid
dynamics (Source: scopus & (Ardabili et al., 2023))
Mälardalen University Press Dissertations
15
Machine Learning Techniques for Enhanced Heat Transfer Modelling
maturation and sophistication of ML algorithms have made them more accessible, effective, and applicable to a broader range of complex problems.
This is particularly noticeable in technical fields such as heat transfer and
fluid dynamics, where the intricate systems and phenomena involved can
greatly benefit from the predictive modelling and pattern recognition capabilities offered by ML techniques.
Secondly, the present era is characterised by unprecedented access to
extensive computational resources, thanks to advances in technology and
computing power. The increasing availability of these resources has significantly lowered the barriers to implementing ML techniques, thus encouraging their widespread adoption across multiple scientific fields. In
parallel with these developments, the explosion of big data across numerous sectors has provided a wealth of training data for these models. The
availability of these vast datasets accelerates the learning process of ML
algorithms and further propels the pace of advancements in these fields.
2.2
Application of ML in heat transfer and fluid dynamics
There has been a substantial historical intertwining of fluid mechanics
and machine learning. Numerous prevalent techniques utilised in today’s
ML landscape were initially explored and introduced within the context
of fluid dynamics many years ago (Brunton et al., 2020) (Pollard et al.,
2016). However, these early methods didn’t gain prominence until the recent convergence of vast data, high-power computing, intricate algorithms
(like deep learning), and significant industry investment. The use of ML
in heat transfer and fluid dynamics domains can be broadly classified as
shown in Figure 2.2.
Figure 2.2: Potential applications of ML in the domain of heat transfer and fluid
mechanics.
16
Jerol Soibam
Chapter 2. Theoretical Background
2.2.1
Data mining and processing
ML algorithms utilise optimisation methods to process data. Their objective is to discover a low-rank subspace for ideal data embedding, useful
for tasks like regression, clustering, and classification. In essence, ML is
a set of mathematical techniques for data mining. Principal Component
Analysis (PCA) is one of the prominent ML algorithms that elucidates
correlations in high-dimensional data. Lumley (1970) employed PCA to
model turbulent flows, suggesting that the instantaneous flow fields can be
depicted by a linear weighted sum of orthogonal basis vectors. A reduced
expression of the flow field can be derived from higher PCA modes. However, as PCA treats each instantaneous field independently, it’s difficult to
understand the temporal information of the flow field solely through PCA
mode. In recent developments, a convolutional neural network (CNN)
auto-encoder was used as a non-linear decomposition method for the flow
field around a cylinder (Murata et al., 2020), demonstrating that a nonlinear activation function reduces the reconstruction error compared to
PCA. Neural networks (NN) were used to investigate the classification of
wake topology behind a pitching airfoil for local vorticity measurements
(Colvert et al., 2018). The authors stated that their NN model was capable
of extracting the features from the wakes and mapping a time series of the
local vorticity.
CNNs can facilitate automated image analysis for object detection,
segmentation, and classification (Redmon et al., 2016; He et al., 2017;
Geng and Wang, 2020). CNNs effectively extract spatial information from
images, resembling human visual perception and proving crucial in scientific image analysis (Albawi et al., 2017). In studying boiling phenomena,
CNNs analyse bubble images to garner insights, owing to the rich statistical data (Fu and Liu, 2019). CNNs have detected the shift from nucleate to
film boiling in pool boiling experiments (Hobold and da Silva, 2019a) and
measured heat flux from boiling images (Hobold and da Silva, 2019b).
This correlation between bubble shapes and heat flux values facilitated effective tracking of the boiling process. CNNs also identified critical heat
flux (CHF) in pool boiling experiments by classifying based on spatial information (Rassoulinejad-Mousavi et al., 2021). Additionally, researchers
have utilised CNNs to predict boiling regimes, proving useful in averting
thermal accidents (Sinha et al., 2021). Finally, a CNN model has linked
bubble dynamics and boiling heat transfer, predicting boiling curves with
a mean error of 6% (Suh et al., 2021). YOLO (You Only Look Once), a
single-stage object detection CNN, has been employed to identify bubbles
during collision, merging, and rupture events (Wang et al., 2021). Twostage CNNs have been previously used to accurately identify and classify bubbles in a two-phase flow circuit for a nuclear power plant cooling
Mälardalen University Press Dissertations
17
Machine Learning Techniques for Enhanced Heat Transfer Modelling
system (Serra et al., 2020). Faster-Region-CNN, another CNN, detects
bubbles and estimates their shape by elliptical fitting, effective at reconstructing overlapping bubbles under a void fraction of 2% (Haas et al.,
2020). Mask R-CNN has also been used to segment and create masks
from an upward bubbly flow, using a mix of experimental and synthetic
datasets (Kim and Park, 2021; Fu and Liu, 2019).
2.2.2
Control and optimisation
Flow control can significantly enhance system performance, making it an
attractive prospect for technological advancements. Feedback flow control
modifies the behaviour of systems through sensor-informed actuation. It
stabilises unstable systems, reduces sensor noise, and compensates for external disturbances and uncertainty. However, challenges arise from fluid
non-linearity, high-dimensional data, and time delays. Many researchers
utilise ML techniques to overcome these issues in system identification,
sensor placements, and controls.
Artificial neural networks (ANNs) have garnered attention for system
identification and control, including aerodynamic applications (PHAN et al.,
1995). They have also been used for turbulence flow control to reduce
skin friction drag (Lee et al., 1997). Genetic algorithms (GA) are widely
used in active control for engineering applications (Dracopoulos, 2013;
Fleming and Purshouse, 2002) and in various flow control plants. GAs
have been used for control design in fluids for experimental mixing optimisation (Benard et al., 2016). Deep reinforcement learning (DRL) is
a promising candidate for flow control applications (Rabault et al., 2019;
Rabault and Kuhnle, 2019). Its use in exploring strong turbulent fluctuations during glider flight was investigated by Reddy et al. (2016).
Optimisation of fin thickness, width, and spacing in a heat sink with
plate fins was shown by Alayil and Balaji (2015). Their method used
ANN to maximise thermal performance, specifically reducing the time to
reach the set temperature. A predictive model for thermal performance
was developed by Bhamare et al. (2021), employing a range of ML and
deep learning methodologies. Their ANN model was able to predict the
heat flux accurately when compared with experimental values. Utilising
considerations such as node arrangement, power consumption, and cooling system configuration, Piatek
˛ et al. (2015) proposed a framework for
thermal modelling and control of HPC systems. DRL has been used to
create room temperature control policies (Di Natale et al., 2021). The
DRL agents use an augmented reward function, to balance comfort levels
and energy conservation, outperforming traditional rule-based controllers
both in simulation and real-world application.
18
Jerol Soibam
Chapter 2. Theoretical Background
2.2.3
Flow modelling
Turbulence, a phenomenon in fluid dynamics marked by unpredictable,
time-varying fluctuations in velocity and pressure, remains a persistent,
unresolved challenge (Jiménez and Moin, 1991; Drikakis, 2003). To accurately capture the transient flow physics with Computational Fluid Dynamics (CFD), an enormous number of grid points would be needed.
The most precise solution in fluid mechanics, directly solving the NavierStokes equations, known as Direct Numerical Simulation (DNS), is computationally impractical for high Reynolds numbers. In contrast, Reynolds
Average Navier Stokes (RANS) simulations, rely on approximated equations and are widely used in the industry due to their cost-effectiveness, at
the expense of accuracy. The RANS approach requires introducing new
terms to close the system of equations, with commonly used models being
k-epsilon, k-omega, and Spallart–Allmaras. However, no current models
are universally applicable across varied conditions (Brunton and Noack,
2015). Selecting the correct model, along with the appropriate boundary
and initial conditions (such as turbulence kinetic energy and dissipation
rate), is critical for accurately portraying the average flow behaviour.
Closure model
Turbulent and bubbly flows present significant spatial-temporal scale separation, making it computationally expensive to address all the scales in
simulations. Even with significant advances in computational power, full
resolution of all scales within a configuration remains decades away. As a
result, it’s typical to truncate the smaller scales and model their influence
on larger ones using a closure model, a strategy employed in RANS and
Large Eddy Simulation (LES). However, such models necessitate meticulous adjustment to correspond with fully resolved DNS simulations or experimental data. The application of ML algorithms for the development of
turbulence closures is an active field of research (Duraisamy et al., 2019).
ML methodologies have been utilised to discover and model inconsistencies in the Reynold stress tensor between RANS models and DNS simulations (Ling and Templeton, 2015). For LES closures, ANN has been used
by Maulik et al. (2019) to predict the turbulence source term from roughly
resolved quantities. In the case of heated cavity flow, a data-driven closure
model has been adopted to solve the Boussinesq approximations (San and
Maulik, 2018). A machine learning and field inversion framework was
devised by Parish and Duraisamy (2016) to construct corrective models
based on inverse modelling. This framework was subsequently used by
Singh et al. (2017) to create an NN-enhanced correction to the SpalartAllmaras RANS model, demonstrating excellent performance. A novel
Mälardalen University Press Dissertations
19
Machine Learning Techniques for Enhanced Heat Transfer Modelling
architecture that incorporated a multiplicative layer to embed Galilean invariance into tensor predictions was developed by Ling et al. (2016b). It
offers a unique and simple way to incorporate known physical symmetries and invariances into the learning architecture (Ling et al., 2016a),
which could prove essential in future endeavours that integrate learning
and physics.
Reduced order model
Over time, a significant amount of work has been devoted to the development of precise and efficient Reduced Order Models (ROMs) that can
encapsulate crucial flow and heat transfer behaviour with minimal computational expense. One such model reduction technique is the Galerkin
projection (Proctor et al., 2016) of Navier-Stokes equations onto an orthogonal basis of Proper Orthogonal Decomposition (POD) modes, which
benefits from a strong link to the governing equations. However, this approach is intrusive and poses challenges for non-linear systems. One of
the ML techniques employed in ROM is Dynamic Mode Decomposition
(DMD) (Rowley et al., 2009), an infinite-dimensional linear operator illustrating the temporal evolution of all measurement functions of the system. However, DMD is based on linear flow field measurements and the
derived models may not successfully capture non-linear transients. In contrast, ANNs can effectively handle large data volumes and non-linear challenges, such as those presented by near-wall turbulent flow (Milano and
Koumoutsakos, 2002). ANNs have been utilised by Sarghini et al. (2003)
to learn from LES channel flow data, enabling the identification and reproduction of the highly non-linear behaviour characteristic of turbulent
flows. The fusion of CNNs with ANNs has been deployed to create a
time-dependent turbulent inflow generator for a fully developed turbulent
channel flow (Fukami et al., 2019).
In the context of heat transfer, ANN models have been successfully
used to accurately predict convective heat transfer coefficients within a
tube (Jambunathan et al., 1996). Experimental data gathered from an impingement jet with variable nozzle diameter and Reynolds number, have
been employed to construct an ANN model. Later this model was used
to predict the Nusselt number with an error below 4.5% (Celik et al.,
2009). A random forest model was developed by Breiman (1996) for
predicting the heat transfer coefficient of complex geometries. When the
results from this model were compared with those from CFD simulations,
a strong correlation was noted, demonstrated by an R2 value of 0.966.
In their research, Jiang et al. (2013) explored the application of a Support Vector Machine (SVM) for predicting the Critical Heat Flux (CHF).
20
Jerol Soibam
Chapter 2. Theoretical Background
The results obtained from their model showed a strong agreement with
experimental data. Previously, Gaussian process regression (GPR) (Jiang
et al. (2020); Baraldi and Mangili (2015)) has been used to predict CHF
in minichannel. The advantage of the Gaussian process is that it can account for a priori knowledge and estimate model uncertainty. Traverso
et al. (2023) employed GPR to predict the heat transfer coefficient in a
microchannel. Their study results indicate that the GPR model exhibited consistent and reliable performance across diverse training datasets,
demonstrating its applicability in cost-effective engineering applications.
An in-depth review of the literature on the application of ML in heat transfer was conducted by Hughes et al. (2021). They evaluated the range of
ML’s applications, from creating efficient models for precise predictions
and robust optimisation to its usage in ROMs and optimisation of largescale systems.
Despite the impressive ability of ML models to accurately map highdimensional inputs to outputs, they typically demand large amounts of
data, the acquisition of which can be computationally costly and require
meticulous experiment design (Karniadakis et al., 2021). Consequently,
other researchers have pursued methods that combine physical principles
with ML techniques. The concept of physics-constrained learning was
introduced as early as the ’90s, aiming to solve classical differential equations (Lee and Kang, 1990; Lagaris et al., 1998). Recently, Rassi et al.
proposed a deep learning framework for addressing forward and inverse
problems known as the physics-informed neural network (PINN) (Raissi
et al., 2019b, 2020, 2019c). Sun et al. (2020) demonstrated the usage
of PINN in fluid flow applications without relying on any simulation data,
achieved by rigidly imposing initial and boundary conditions. Meanwhile,
Cai et al. (2021a) used PINN to derive velocity and pressure fields from
temperature data gathered from a background-oriented Schlieren experimental setup, designing a PINN framework capable of predicting both
fields without any information on initial and boundary conditions. Lucor
et al. (2022) employed PINN for surrogate modelling of turbulent natural
convection flows, primarily using direct numerical simulation (DNS) data
for network training. Cai et al. (2021b) used PINN for inverse heat transfer
problems and two-phase Stefan problems with a moving interface. They
further suggested that placing the thermal sensors based on the residual of
the energy equation gave the best performance.
From the literature survey, it is evident that extensive efforts have previously been exerted on probing the application of ML within the field
of fluid dynamics and heat transfer. The ML algorithms typically referenced in existing literature largely utilise deterministic methods that unfortunately fall short in supplying predictive uncertainty information. Yet,
Mälardalen University Press Dissertations
21
Machine Learning Techniques for Enhanced Heat Transfer Modelling
to understand the application of these ML models to engineering complexities is still in its infancy, with a wealth of information waiting to be
discovered. Therefore, this thesis aims to delve into the exploration of ML
techniques for heat transfer problems, placing a particular focus on experimental data processing and reduced order modelling. More specifically,
the present work examines the use of deep learning models for subcooled
boiling and the efficient and effective utilisation of data. Furthermore, this
thesis explores methods to navigate the complexities of unknown thermal
boundary (ill-posed) problems. The central theme of this thesis revolves
around developing robust ML models, with significant efforts directed towards accounting for the uncertainty present in the model, which can further illuminate the model’s behaviour.
22
Jerol Soibam
3
Methodology
In this chapter, a detailed framework of machine learning algorithms used
in this thesis is provided. Relevant theories and methods are presented in
detail.
3.1
Framework of the thesis
The thesis is based on a research framework that guides the exploration
of ML applications in heat transfer problems. Central to this framework
is the evaluation of the robustness of ML models, an aspect that directly
influences their performance and reliability. The framework delineates a
clear method for selecting and verifying ML models, taking into account
the nature and availability of data. It serves as the foundation for the
research presented in the associated publications, providing a structured
approach to addressing the research questions. In essence, the framework
not only organises the research process but also directs the exploration
and findings that form the crux of this thesis. The overall learning system adapted in this thesis, including the robustness assessment of the ML
models, is illustrated in Figure 3.1. Detailed explanations of each module
are provided in the subsequent sections.
Figure 3.1: Detailed framework of the learning system.
Mälardalen University Press Dissertations
23
Machine Learning Techniques for Enhanced Heat Transfer Modelling
3.2
Data availability
The data considered in this thesis mainly relies on heat transfer data namely
subcooled boiling and convective heat transfer data. The data used is either obtained from experiments or numerical studies which serve as the
backbone of the architecture to be trained. Furthermore, the quantity of
data plays a crucial role in determining what architecture is appropriate to
follow up the study.
Data for subcooled boiling heat transfer
The data used for training the Deep Neural Network (DNN) in Papers I
and II is of a numerical nature, specifically related to the phenomenon of
subcooled boiling heat transfer. It is extracted from numerical simulations
with varying degrees of heat flux and flow inlet conditions, as displayed in
Table 3.1. The computational domain used for CFD simulation is shown
in Figure 3.2a. For a more comprehensive understanding of the details
of data, readers are directed to Paper II and Rabhi et al. (2021). Note:
Numerical simulation is beyond the scope of this thesis.
Turning to Paper III, this utilises data to predict the critical heat flux,
sourced from pre-existing literature (Kim et al., 2000). The paper offers
a detailed breakdown of the data and includes a look-up table for the data
that was used in the validation process.
Up until this point, the focus has been on field data or point data. However, Paper V presents a shift in the nature of data used, as it incorporates
image data derived from an experimental study on subcooled boiling as
shown in Figure 3.2b. The image data, captured via a high-speed camera,
was directly used for labelling, thus eliminating the requirement for any
image processing techniques. Detailed descriptions and further specifics
regarding the data used in this case study are available in Paper V.
Table 3.1: Data split for training, validation, and testing at different heat flux
and flow conditions.
Cases
Training / Validation
Interpolation
Interpolation
Interpolation
Extrapolation
Extrapolation
Extreme extrapolation
24
Heat flux (q ) [W m−2 ]
Velocity (u) [ms−1 ]
[1000 - 29,000]
14,000
17,500
19,000
30,000
30,000
40,000
[0.05 - 0.2]
0.05
0.075
0.15
0.1
0.15
0.2
Jerol Soibam
Chapter 3. Methodology
Figure 3.2: Numerical computational domain and experimental setup.
Data for convective heat transfer
The data used in Paper IV and VI is an incompressible laminar flow scenario with convective heat transfer under the Boussinesq approximation.
In Paper IV, the case considered is a steady state forced convection with
constant heat flux boundary condition on the source as shown in Figure 3.3a. Whereas, in Paper VI the data used is from a transient mixed
convection setup for constant heat flux and constant temperature boundary on the source as shown in Figure 3.3b. The forced and mixed convection problems are solved using the governing equations - the NavierStokes equations and the energy equation, which are presented in their
non-dimensional form as follows:
∂ u∗ ∂ v∗
+
=0
∂ x ∗ ∂ y∗
∗
∗
∂ u∗
1 ∂ 2 u∗ ∂ 2 u∗
∗∂u
∗∂u
∗
+u
+v
= −∇px +
+
∂t ∗
∂ x∗
∂ y∗
Re ∂ x∗2 ∂ y∗2
∗
∗
1 ∂ 2 v∗ ∂ 2 v∗
∂ v∗
∗∂v
∗∂v
∗
+
u
+
v
=
−∇p
+
+
+ Riθ ∗
y
∂t ∗
∂ x∗
∂ y∗
Re ∂ x∗2 ∂ y∗2
2 ∗
∗
∗
∂ θ
∂θ∗
1
∂ 2θ ∗
∗∂θ
∗∂θ
+u
+v
=
+ ∗2
∂t ∗
∂ x∗
∂ y∗
Pec ∂ x∗2
∂y
(3.1)
where u∗ , v∗ , p∗ , and θ ∗ are the non-dimensional velocity-x, velocityy, pressure, and temperature fields respectively. The dimensionless quantities Re, Ri, and Pec correspond to the Reynolds number, Richardson
Mälardalen University Press Dissertations
25
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Figure 3.3: Numerical computational domain for forced and mixed convection.
number, and Péclet number respectively. In the case of forced convection
Ri = 0. Moreover, data obtained from a direct numerical simulation (DNS)
(Fraigneau, Y., 2019) for the mixed convection at constant temperature on
the source was also used in this thesis. More details on the data can be
found in Papers IV and VI.
Data from experiments or numerical simulations often demonstrates
significant variations in magnitudes, uncertainties, units, and ranges. Such
variations can impact the effectiveness and efficiency of ML models, especially those that are sensitive to the scale of the features. To rectify
this, data processing is carried out, which often involves scaling. This
procedure ensures no specific feature dominates the model due to a larger
numeric range. Multiple scaling techniques exist and are selected based
on the nature of the data and the model being used. Furthermore, outliers or points that diverge significantly from other observations, can skew
model learning and hamper performance. The strategies range from eliminating outliers, and replacing them with statistical measures such as mean
or median, to employing robust models that are less sensitive to outliers.
3.3
Architectures
This thesis carefully investigates a spectrum of methodologies, aligning
each to the individual requisites and performance standards of each task.
It specifically concentrates on numerical data, experimental data, and sensor data. The ambition is to formulate robust and adaptable algorithms to
tackle both direct and inverse problems, with a focus on specific circumstances and requirements.
3.3.1
Deep neural network
An artificial neural network (ANN), with more than one hidden layer is
commonly known as a deep neural network (DNN). The DNN essentially
26
Jerol Soibam
Chapter 3. Methodology
involves a more complex structure of interconnected artificial neurons, or
nodes, extending the capabilities of traditional ANNs as shown in Figure
3.4. This complexity allows the DNN to capture higher-level features and
relationships within the data. It utilises a feed-forward phase where input
signals are passed sequentially through layers until they produce the outputs. The layers undergo a series of transformations influenced by weight
and bias parameters, paired with an activation function.
The transformations can be represented as:
h1 =
g(W1T x + b1 )
..
h5 = g(W5T h4 + b5 )
ŷ = g(W6T h5 + b6 )
0 for x < 0
g(x) =
x for x ≥ 0
(3.2)
(3.3)
where h is the hidden layer and g is the activation function. The objective is to minimise the error, calculated using the mean square error (MSE)
loss function, between the predicted and target values. The target values in
Paper I and II are wall temperature (Twall ) and void fraction (α). Once this
loss is computed, the error gradient relative to the weights (W ) and biases
(b) across all layers can be determined during a backward phase, using
the chain rule of differential calculus. The weights and biases are then updated with the Adaptive Moment Estimation (Adam) optimiser (Kingma
and Ba, 2014), known for handling large datasets and sparse gradients.
To prevent overfitting when handling a high volume of parameters,
a regularization term, specifically L2 norm (Ridge regression), is introduced into the loss function, making the network less complex by rendering some neurons negligible. This method shrinks the weight parameters
Figure 3.4: Deep neural network architecture.
Mälardalen University Press Dissertations
27
Machine Learning Techniques for Enhanced Heat Transfer Modelling
towards zero but not to zero, offering a balanced regularisation. This approach reduces the variance and improves the model’s generalisation to
new datasets.
N
L = MSE(y, ŷ) + λ ∑ w2i L2 norm
(3.4)
i=1
For an in-depth understanding of the DNN structure, Paper II offers
comprehensive insights into its architecture. It meticulously lays out the
hyperparameters employed in the training process of the DNN.
3.3.2
Physics informed neural network
The suggested strategy uses inverse PINN to tackle the heat transfer problem with unknown thermal boundary conditions (Raissi et al., 2019b). The
PINN framework utilises a two-part deep neural architecture, one trained
to represent the fields of velocities, pressure, and temperature, and another enforcing the Navier-Stokes and energy equations at random points
as shown in Figure 3.5. The network inputs are the non-dimensional coordinates and time of the given domain, mapping to expected outputs of
velocities in x and y components, pressure, and temperature.
(x∗ , y∗ ,t ∗ ) → (u∗ , v∗ , p∗ , θ ∗ )
(3.5)
The network’s effectiveness is enhanced by minimising a specified
loss term, and it comprises of four components:
L = Lr + Lub + Lθ b + LQR ,
(3.6)
where,
Lr =
1 4 Nr
∑ ∑ |e j (xi , yi ,t i )|2
Nr j=1
i=1
Lub =
1 Nub
∑ |u(xi , yi ,t i )) − uib |2
Nub Nt i=1
Lθ b =
1
Nθ b
∑
Nθ b Nt i=1
NQR
LQR =
1
(3.7)
|θ (xi , yi ,t i )) − θbi |2
∑
NQR Nt i=1
|Π(xi , yi ,t i )) − ΠiQR |2 .
Lr penalizes the continuity, momentum, and heat equation, computed
from residuals obtained through automatic differentiation. Lub is the loss
function for velocity boundary conditions on source walls, often known
28
Jerol Soibam
Chapter 3. Methodology
Figure 3.5: PINN framework for forced and mixed convection.
as no-slip conditions, are applied. Lθ b minimizes the inlet temperature.
LQR minimizes the difference between sensor values and predicted values. Automatic differentiation, which yields derivatives of the output relative to the inputs, is the crucial element of the PINN framework. This
mechanism forms the foundation upon which the PINN model operates.
In the training phase, the PINN model incorporates initial and inlet conditions from a numerical simulation and adjusts the parameters.
The optimisation process uses sensor locations identified by QR pivoting,
which serve both temperature and velocity measurements. PINN’s approach is different from traditional DNNs as it not only learns from data
but also incorporates known physical laws, enhancing its accuracy and
generalisation capabilities in solving intricate physical problems. An indepth formulation of the PINN architecture and sensor selection adopted
in this thesis can be found in Papers IV and VI.
3.3.3
Convolutional neural network
Convolutional neural networks are designed to handle multi-dimensional
data with spatial information. They require fewer parameters to optimise
than traditional feed-forward neural networks, making them more efficient. In object detection, two methods are generally employed: the onestage method and the two-stage method. The one-stage method, such as
you only look once (YOLO) (Redmon et al., 2016), detects the object’s
position, segment, and classifies it simultaneously, while the two-stage
method first predicts the position, and then classifies the object. The architecture in this thesis focuses on the one-stage method for its benefits of
speed and real-time prediction capabilities without compromising accuracy. The architecture’s strength lies in its division into three main components: the backbone, neck, and head networks as shown in Figure 3.6.
Mälardalen University Press Dissertations
29
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Figure 3.6: Convolutional neural network for bubble segmentation and tracking.
Each part performs a distinct role within the overarching system, sequentially processing input data and passing the output onto the subsequent
component. This modular approach fosters a better understanding of the
object’s spatial and contextual information, thus facilitating more precise
object detection and segmentation. Proceeding further, a comprehensive
clarification of the distinct functionalities and intricacies of the three cardinal components within the YOLO architecture (Wang et al., 2022): the
backbone, neck, and head networks, is presented.
• Backbone: Extracts various features such as shape, size, edges, texture, and background from the input image through a series of operations. It outputs multi-scale feature maps which are then processed
by an extended-efficient layer aggregation network (E-ELAN) and
a max pooling module for further feature extraction.
• Neck: Ingests the output from the backbone and integrates these
multi-scale feature maps. It uses the path aggregate feature pyramid network (PAFPN) and the cross-stage partial spatial pyramid
pooling (SPPCSPC) to pool features from all levels, bridging the
gap between lower and upper feature levels and reducing computation costs.
• Head: Processes the features extracted by the backbone and neck
with a convolution layer before passing them to the head network.
The head network, based on yolact++ (Bolya et al., 2022), divides
the complex task of instance segmentation into two parallel tasks
that merge to form the final masks.
30
Jerol Soibam
Chapter 3. Methodology
The network is trained using four types of loss:
L = lbox + lob j + lcls + lmask
(3.8)
where L is the total loss function used to optimize the model. lbox is
the box regression loss, lob j is the objectness loss, lcls is the classification
loss, and lmask is the mask loss of the segmentation. It is of note that a
technique like transfer learning learning was adopted in this study for data
efficiency and also to improve the accuracy of the model. For a more detailed description of each component’s architecture within the backbone,
neck, and head, please refer to Paper V.
3.3.4
Gaussian process regression
Gaussian process regression (GPR) is a Bayesian technique employed for
nonlinear regression, offering flexibility and probabilistic modelling that
provides robust posterior. However, the traditional GPR tends to be computationally expensive and its complexity increases with the size of the
dataset, as it requires access to the full dataset during both the training and
prediction phases. For example, for a data size of N exact inference has a
computational complexity of O(N 3 ) with a storage demand of O(N 2 ).
To address these challenges, Parametric Gaussian process regression
(pGPR) (Raissi et al., 2019a) is used. In pGPR, the model is trained on
a hypothetical dataset, which is usually smaller than the actual data as
shown in Figure 3.7. This approach enhances data efficiency as it focuses on the hypothetical dataset, obtained via k-means clustering on data
points of the critical heat flux. Once the hypothetical dataset is established, pGPR is defined by the conditional distribution and its parameters are updated via the posterior distribution conditioned on a mini-batch
of observed data. The initial parameters of the pGPR model are trained
3
f (x)
3
6000 data points
2
2
8 Hypothetical data points
True function = x ∗ cos(4 ∗ 3.14 ∗ x)
Predicted function
±3σ
f (x)
f (x)
1
0
1
0
−1
−1
−2
−1.5
−1.0
−0.5
0.0
x
0.5
1.0
1.5
−2
−1.5
−1.0
−0.5
0.0
x
0.5
1.0
1.5
Figure 3.7: Framework for parametric Gaussian process regression.
Mälardalen University Press Dissertations
31
Machine Learning Techniques for Enhanced Heat Transfer Modelling
and then used for predictions on new data points. Like GPR, uncertainty
in pGPR is quantified by calculating the predicted variance, which provides a measure of the reliability of the model’s predictions. In essence,
it gives an indication of the spread of possible output values predicted by
the model. The smaller the predicted variance, the greater the confidence
in the model’s predicted output.
The choice of kernel function significantly influences a pGPR model’s
generalisation capabilities. In this study, the kernel function is assumed to
have a squared exponential covariance function, which is chosen for its
ability to model smooth functions. This kernel function includes variance
and length-scale hyperparameters. A more detailed formulation of the
pGPR for predicting critical heat flux can be found in Paper III.
3.4
Uncertainty quantification for robustness
Uncertainty quantification (UQ) in deep learning models is critical for
a variety of reasons. It aids in gauging the model’s confidence level in
its predictions, providing a safety measure against overconfident and potentially erroneous outputs. Additionally, it facilitates risk assessment in
practical applications where understanding the risk associated with each
prediction is paramount. Furthermore, it helps identify areas requiring
further training or data acquisition, thereby improving the model performance. Lastly, in scientific research, uncertainty quantification aids in
validating findings, enhancing the overall transparency and robustness of
the research process.
This thesis explores three distinct methods. The initial approach leverages the Bayesian methodology, elaborated in Section 3.3.4. The subsequent methods concentrate on UQ within the realm of deep learning, suitable for integration within any DNN or PINN architectures. More details
on these methods can be found in Papers II, III, and VI.
3.4.1
Monte Carlo dropout
Dropout is an effective technique that has been widely used to solve overfitting problems in DNNs just like the regularisation technique. It has
been shown that it can also be used for uncertainty estimation in a DNN,
using Monte Carlo dropout (Gal and Ghahramani, 2016). This technique
interprets a DNN trained with dropout as a Bayesian approximation of
a Gaussian process. It applies dropout not only during training but also
prediction as shown in Figure 3.8.
Given a trained network with an input feature X , it provides a predictive mean E(y) and variance Var(y∗ ). A length scale l is defined to cap32
Jerol Soibam
Chapter 3. Methodology
Figure 3.8: Monte Carlo dropout architecture with a dropout ratio of 20% for
training and testing.
ture the belief over data frequency, with shorter scales indicating higher
frequency. The Gaussian process precision (τ) is given as:
τ=
l2 p
2Nλw
(3.9)
where p is the probability of the units not dropped during training,
λw is the weight decay, and N is the dataset size. Dropout is also used
during the prediction phase. Using a dropout ratio of 20% during training
and testing, the prediction step is repeated several times (T ) with different
units dropped each time, giving the results ŷt (x). The predictive mean and
variance of the test data are given by:
E(y) ≈
Var(y) ≈ τ −1 ID +
3.4.2
1 T
∑ ŷt (x∗ )
T t=1
1 T
∑ ŷt (x)T ŷt (x) − E(y)T E(y∗ )
T t=1
(3.10)
(3.11)
Deep ensemble
Deep Ensemble is a non-Bayesian technique for uncertainty quantification in ML models. It involves training several neural networks, rather
than one, to enhance generalisation capabilities (Lakshminarayanan et al.,
2017). For a regression problem, the model assumes the target has a
normal distribution with mean and variance based on the input values
as shown in Figure 3.9. The loss function is adjusted to minimise the
difference between predictive and target distribution using Negative LogLikelihood (NLL) loss:
Mälardalen University Press Dissertations
33
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Figure 3.9: Deep ensemble architecture for M = 2.
L = −logpθ
yn
logσ 2 (X) (y − μθ (X))2
=
+
+c
Xn
2
2σ 2 (X)
(3.12)
In a Deep Ensemble model, several networks are trained with different
random initialisations. Predictions are treated as a uniformly weighted
mixture model, approximated as a Gaussian distribution. The mean and
variance mixture are:
1
μθ m (X)
M∑
m
(3.13)
1
(σ 2 θ m(X) + μθ m2 (X)) − μ∗2 (X)
M∑
m
(3.14)
μ(X) =
σ 2 (X) =
The implementation involves defining a custom NLL loss function and
a custom layer to extract the mean and variance as network output.
34
Jerol Soibam
4
Results and discussion
This chapter describes the most important results from the appended papers. This chapter concludes with a discussion of the research questions
of the implications and potential contributions of these results.
4.1
Case study on boiling heat transfer
Boiling heat transfer plays a crucial role in many engineering and industrial processes such as power generation, cooling of electronics, and chemical processing, with its efficiency and effectiveness greatly influencing
the overall performance of these systems.
4.1.1
DNN for subcooled boiling
This subsection emphasises the outcomes garnered from the DNN model
implemented in Papers I and II. The purpose of this research was to ascertain if a data-driven approach, like DNN, can be harnessed to examine the
inherent physics in subcooled boiling from data, and if it can be used for
the prediction of quantities of interest (QoIs). Three architectures were investigated namely a traditional DNN, Monte Carlo dropout (MC dropout),
and deep ensemble (DE) model. The data used in all the models remains
consistent throughout the study as described in Section 3.2.1.
Validation of the models
The predictive and statistical performance for all the models was initially
evaluated using the validation dataset. The corresponding results are illustrated in Table 4.1. From the Table, it can be noted that all the models
Table 4.1: Performance of the DNN, MC dropout, and DE models on validation
dataset of the computational domain, VF: Void Fraction, Temp:
Temperature.
Case
Dataset
Validation
Models
DNN
MC Dropout
Deep Ensemble
R2
RMSEP
VF
Temp
VF
Temp
0.006
0.006
0.002
0.9982
0.9991
0.9998
0.995
0.997
0.998
Mälardalen University Press Dissertations
0.134
0.125
0.081
35
Machine Learning Techniques for Enhanced Heat Transfer Modelling
have low RMSE for both void fraction and temperature which are the
QoIs. However, it can be observed that the DE model outperforms the
other two models with a lower RMSE and a higher coefficient of determination (R2 ). The conclusion that can be drawn from this is that the DE
model fits better for the validation dataset.
Field prediction of the minichannel
The result presented here is based on an interpolation dataset as described
in Table 3.1 in Chapter 3, with a heat flux of q = 14,000 W m−2 and
an inlet velocity of u = 0.05 ms−1 . As was shown in the previous analysis that the DE model is superior compared to DNN and MC dropout,
hence the predicted fields of temperature and void fraction from the DE
model are plotted in Figure 4.1 and 4.2. The DE model’s predictions of
the minichannel temperature field closely align with the CFD values, as
depicted in Figure 4.1. Upon examination of the relative percentage error
field, it is observed that the majority of errors, lie under ± 0.2%, are concentrated where the heat flux is applied. Once again it can be seen that
the relative error is under 1% for the void fraction as shown in Figure 4.2.
The maximum error occurs at a non-dimensional arc length of 0.5 on the
y-axis. At this location, the void fraction increases suddenly, potentially
due to the formation of a larger bubble or a multitude of smaller bubbles
near the wall, which causes an abrupt increase in void fraction.
Figure 4.1: Temperature field prediction and the relative error between the CFD
and DE model of the minichannel.
36
Jerol Soibam
Chapter 4. Results and discussion
Figure 4.2: Void Fraction field prediction and the relative error between the CFD
and DE model of the minichannel.
Model sensitivity along the heated wall
The section of interest involves the near-wall region of a minichannel,
where heat flux causes sudden shifts in physics. An investigation into
model performance has been done with an extreme extrapolation dataset,
utilising a heat flux of q = 40,000 W m−2 and an inlet velocity of u = 0.2
ms−1 . Results, as seen in Figure 4.3, show the prediction of the void fraction in this area. The dimensionless arc length on the x − axis corresponds
to the minichannel height where heat flux is imposed. The DNN and MC
dropout models struggle to correctly predict the onset of nucleation sites.
Nonetheless, the MC dropout predicted values align well with the trend of
CFD void fraction, with uncertainty starting around 0.1 and peaks at 0.2
arc lengths due to liquid-to-bubble phase shifts. In contrast, the DE model
presents lower variation throughout the void fraction prediction and accu-
(a) DNN and MC dropout
(b) Deep Ensemble
Figure 4.3: Wall void fraction profile by the DNN, MC dropout and DE models
along the arc of the minichannel
Mälardalen University Press Dissertations
37
Machine Learning Techniques for Enhanced Heat Transfer Modelling
rately anticipates the onset of the nucleation site. However, the prediction
uncertainty grows with the void fraction, possibly due to the coalescing of
smaller bubbles into larger ones. The uncertainty then starts to decrease
around 0.7 arc length as the predicted void fraction from the DE model
converges with the CFD values. This suggests that the DE model can effectively recognise changes from liquid to vapour and increased bubble
presence, displaying greater uncertainty in such areas.
For a more comprehensive understanding of the predictive performance and robustness of the DNN, MC dropout and DE models, please
refer to Papers I and II. These studies delve into intricate details of the
models’ behaviour, addressing their capabilities, and discussing the uncertainties surrounding various aspects. They also further discuss the reasons
behind the uncertainties in their predictions and how these models handle
them in various situations.
4.1.2
Computer vision for subcooled boiling
This section illustrates the efficacy of the proposed CNN model in bubble identification within a constant heat flux rectangular mini-channel as
shown in Figure 3.2b. The model addresses the shortcomings of traditional methods, ensuring precise and dependable segmentation. The results underline the CNN model’s superior capability, enabling an in-depth
study of bubble behaviours.
Figure 4.4: Comparison of mask between ground truth (human eyes), classical
image processing and CNN model. a) Ground truth mask b)
Classical mask c) CNN mask d) Normal image with a surface void
fraction of 6.45% e) IoU for classical method and f) IoU for CNN
model
38
Jerol Soibam
Chapter 4. Results and discussion
Comparison and sensitivity analysis
The bubble segmentation mask from the CNN model is compared against
the ground truth masks and traditional image processing masks as shown
in Figure 4.4. The ground truth mask is converted into a binary image,
which sometimes lacks clear bubble boundaries due to uneven illumination and pixelation. Classical methods overestimate bubble area by not
distinguishing between bubbles and shadows and misinterpreting closely
located bubbles as one. Its intersection over union (IoU) score is 71%.
The CNN model performs better, as it can separate bubbles and shadows
and identify individual bubbles in close proximity. The model captures
tiny bubbles effectively and its IoU score is 88% against the ground truth
mask. The maximum pixel difference with the ground truth is 2 pixels,
consistent with the average uncertainty found in ground truth masks, indicating the model’s performance is as good as human labelling, despite
image downscaling.
The model’s sensitivity was evaluated under different conditions, as
depicted in Figure 4.5. Though the model was not trained with noise or
Figure 4.5: Masks given by the model with different noise conditions: a)
Original image with the mask, b) Contrast enhancement, c)
Sharpening, d) Top-hat filter with disk shape radius = 20, e)
Gaussian Blur σ = 0.8, f) Gaussian noise σ = 10, g) Gaussian noise
σ = 25, h) Black dead pixel 0.02% and i) IoU for noisy images
Mälardalen University Press Dissertations
39
Machine Learning Techniques for Enhanced Heat Transfer Modelling
enhancements in the raw image, it was able to handle the noisy conditions and predict the mask accurately. Enhancements such as increased
contrast and sharpness didn’t affect the model’s performance. A top-hat
filter, applied to one of the images, caused slight deviation, yet the model’s
overall predictive capability remained strong. The introduction of a Gaussian noise at σ = 10 saw the model performing robustly. It, however,
struggled slightly with tiny bubbles and shadow differentiation at higher
noise levels (σ = 25). The presence of dead pixels led to a few incorrect
predictions, showing the model’s sensitivity towards them. The model’s
performance under different noise conditions was evaluated using the IoU
measure. The IoU trend indicated the model’s ability to handle noise with
minor improvements or decreases in prediction accuracy. However, the
model showed a noticeable decline in performance under high Gaussian
noise at σ = 25.
Local bubble statistics
Figure 4.6: Results extracted from the binary mask given by CNN model to
study local bubbles behaviour: a) Trajectory of the bubbles with
velocity magnitude and growing bubble images (Note: axes are not
in scale), b) Bubble diameter with time, c) Bubble Reynolds number
with time.
40
Jerol Soibam
Chapter 4. Results and discussion
The mask generated by the CNN model is used for identifying bubble
nucleation sites, edges, and centroids. This information provides insights
into the bubbles’ trajectory, equivalent diameter, and velocity. Local bubble behaviours such as coalescence, collision, oscillations, and condensation, which lead to changes in size, shape, and speed, are examined by
tracking bubble centroids over time and correlating the images.
The mask generated by the CNN model and post-processed is shown
in Figure 4.6 for one specific nucleation site, undisturbed during the bubble life cycle. The bubble centroid’s position is used to calculate its trajectory, velocity, and magnitude. The growth of bubbles over time is shown
in Figure 4.6a, while the equivalent diameter is depicted in Figure 4.6b.
Figure 4.6c reveals the local Reynolds number during the bubble’s lifecycle until it condenses. The model immediately recognises a newly formed
bubble which grows on the nucleation site for a characteristic time tg of
5ms. In this phase, the diameter grows while the centroid and local velocity remain almost constant. After growth time, the bubble detaches and
moves with the flow, its diameter increasing until it reaches its final size,
driving up its velocity. Finally, the bubble detaches completely and condenses within the flow. A more detailed study and analysis can be found
in Paper V.
4.1.3
Critical heat flux
In this section, the predictive performance of pGPR for critical heat flux
(CHF) is discussed. The CHF represents the most efficient heat transfer regime when a liquid coolant undergoes a phase change on a heated
surface. However, there is a risk: should the heated surface temperature
rise significantly, it could reach the material’s melting temperature, causing a ’burn-out’ phenomenon which could result in catastrophic failure.
Therefore, particular efforts have been made to study the CHF from a data
perspective using pGPR for accurate prediction. A further intriguing aspect of this model is that it employs data efficiently by introducing the
concept of a hypothetical dataset, thus circumventing the need for a large
dataset.
The pGPR model and the look-up table (LUT) predictive performance
are showcased in Figure 4.7. The plot’s solid line signifies a perfect match
between predicted and experimental data, whereas the dashed line denotes
a ±10% error margin. It’s evident that most predictions fall within a ±5%
error, peaking at ±10%. Conversely, the LUT’s results diverge substantially from the optimal line, making accurate CHF predictions impossible.
The pGPR model surpasses the LUT predictions, and when compared to
several CHF correlations, previously evaluated by Kim et al. (2000), it
Mälardalen University Press Dissertations
41
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Figure 4.7: Validation result of the CHF for pGPR and LUT predicted values
shows fewer errors. Most pGPR predictions fall on the optimal line, with
a few nearing the ±10% error line, significantly improving upon the average ±15% error observed in other models.
(a) Influence of the input parameters on the
CHF
(b) Prediction of CHF with varying mass flux
with uncertainty
Figure 4.8: Weighting factor and prediction obtained from pGPR model
The model illustrates the weighting factor for each input feature contributing to the prediction of CHF in Figure 4.8a. In this dataset, the heated
length (Lh ) carries the highest absolute weighting factor, superseded only
by the mass flux (G). In contrast, the lowest weights are ascribed to the
inlet subcooling and system pressure. These patterns align with the experimental data from Kim et al. (2000), which demonstrates a pronounced
dependency of CHF on Lh and G, but lesser sensitivity to flow pressure
(P) and inlet subcooling (Δhi ). This sheds light on how variations in the
input variables might impact the prediction of CHF.
The experimental data, pGPR model predictions, and associated uncertainty are presented in Figure 4.8b. The model proves effective in mapping the trends of the experimental data when the mass flux is adjusted and
42
Jerol Soibam
Chapter 4. Results and discussion
has been rigorously tested for a range of lengths, diameters, and pressures.
Despite minor discrepancies between the pGPR model’s predictions and
the experimental data, a notable level of uncertainty is evident across all
predictions, regardless of the conditions. This considerable uncertainty is
likely attributable to the high degree of nonlinearity and uncertainty encountered in the CHF dataset, which the model uses for its predictions.
Even with these challenges, the pGPR model demonstrates its value by
effectively distilling information from the experimental data and mirroring it with the testing dataset. This alignment with the underlying physics
further attests to the model’s capability and robustness. For a comprehensive examination and analysis of this study, please refer to Paper III.
4.2
Case study on convective heat transfer
In this section, particular attention is paid to cases where the thermal
boundary remains unknown, rendering numerical simulations infeasible,
regardless of the sophistication of the tools available. As a solution, inverse PINN is employed to address this problem, using certain sensor
points to identify the thermal boundary as well as to predict the full flow
field. The L2 relative error is used to evaluate the performance of the PINN
and it is given as:
ε=
||P − R||2
,
||R||2
(4.1)
where P represents the predicted quantities obtained from PINN model
and R is the corresponding reference values.
This technique has been tested for steady-state forced convection and
transient mixed convection.
4.2.1
Forced convection
The case study considered here is a flow around an enclosure with a heated
source. The numerical results are obtained from a CFD simulation with a
constant heat flux on the source wall for the conditions Re = 100 and Pec
= 75, and are used to compare with the prediction obtained from PINN.
Sensitivity of sensor placement
The PINN model’s predictive performance was tested in relation to sensor quantity and location through various case studies as shown in Figure
4.9. Four sensors were initially placed at the square source’s corners (case
study 1), producing around 9.2% temperature error compared to CFD.
Mälardalen University Press Dissertations
43
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Figure 4.9: Case studies for different sensor placements to infer temperature
distribution in the enclosure domain
Active sensor placement added an extra point (case study 2), but this provided no significant accuracy improvement. This active sensor is achieved
based on the highest residual error when subject to the energy equation
(er = uTx + vTy − Pec−1 (Txx + Tyy )). Sensor count was increased to eight
in case study 3, slightly improving predictive accuracy, though this caused
some unrealistic temperature distribution predictions.
Sensor locations were revised for case study 4, yielding better performance than the first two case studies. Active sensor placement added
another sensor near the source wall’s right corner in case study 5, showing
immediate improvement. Another sensor, based on max residual error,
was added in case study 6, reducing the error to 1.4%. Thus, optimal
sensor location and minimum quantity, rather than sheer number, proved
crucial for accurate thermal distribution. Note that errors in u, v, and p
remained consistent across studies.
Temperature distribution
The thermal sensor positioning of case study 6 underpins the field predictions illustrated in Figure 4.10. The left portion of the figure conveys
the predictions made by the PINN model, the middle portion displays the
results of the CFD simulation, and the right portion shows the L2 relative error. A comparison of the two temperature profiles reveals the PINN
model’s ability to accurately reproduce the global temperature field, requiring only 6 sensor data points. Moving on to the examination of the
TError contour, the largest absolute errors (not presented as percentages)
are found to cluster near the source corner and the outlet. Despite this, the
44
Jerol Soibam
Chapter 4. Results and discussion
Figure 4.10: Case studies for different sensor placement to infer temperature
distribution in the enclosure domain
errors are fairly small in magnitude, suggesting the potential for further
accuracy enhancements through the addition of extra sensors.
Transfer learning
Training PINN models typically demands substantial computational power
due to the application of first-order optimisation techniques. However, using transfer learning, this training time can be significantly reduced. This
involves reusing the weights and biases from a previously trained network
for new configurations of a similar problem. For instance, weights and
biases from the case study of 6 sensor placements (Figure 4.9) have been
used as starting points for tuning a new network for different shapes and
two sources, utilising 8 sensor points on each source wall. The network
was then trained for 5000 iterations, with a learning rate of 1×10−6 . Compared to the previous case it showed an inferior predictive performance
with a maximum L2 relative error of 6.5%. It is noticeable that the transferred PINN model over-predicts around the source and under-predicts
between the two rectangular sources. Importantly, while training the inverse PINN from scratch took about 20 minutes on an nvidia RTX 3090
GPU. The use of transfer learning dramatically reduced the time needed
for fine-tuning parameters while retaining an acceptable accuracy.
Figure 4.11: Temperature distribution: Transfer learning case study for two
rectangular sources
Mälardalen University Press Dissertations
45
Machine Learning Techniques for Enhanced Heat Transfer Modelling
4.2.2
Mixed convection
The case study considered here is flow around a heated square cylinder.
Two cases have been studied, firstly, the heat is applied to the source using
constant heat flux at Re = 100, Pec = 100, and Ri = 0.4. In the second
case, constant temperature is imposed on the heated square for the flow
conditions Re = 50, Pec = 50, and Ri = 1.0.
QR sensor placement
The accuracy of inverse PINN simulations is strongly dependent on the
strategic placement of sensors within a given field. QR factorisation with
column pivoting is an effective method to optimise sensor positioning, as
it focuses on capturing the dominant modes of the field. The sensor location obtained from QR pivoting is then used to optimise the inverse PINN.
A performance comparison of PINN over time, utilising sensors derived
from both random selection and QR pivoting, is illustrated in Figure 4.12.
These figures indicate the L2 errors for u∗ and θ ∗ , thereby emphasising the
superior performance of PINN with sensors determined by QR pivoting,
compared to those selected randomly. This trend remains unaltered, despite the persistent changes in the flow’s temporal dynamics due to vortex
shedding from the square source.
The study investigates the relationship between the number of QR sensors used and the accuracy of PINN in reconstructing temperature distribution, as shown in Figure 4.13a. It demonstrates that as the number of
sensors increases, the L2 error decreases due to an enhanced ability to
capture detailed thermal characteristics of the flow. This increase in sensor count subsequently improves the performance of the PINN by offering more locations for optimising the energy and y-momentum equations,
which are vital for temperature field reconstruction. Nevertheless, a pattern of diminishing returns is observed, where further reduction in the L2
Figure 4.12: Comparision of L2 relative error between QR and randomly
selected sensors over time for the component u∗ and θ ∗
46
Jerol Soibam
Chapter 4. Results and discussion
(a) L2 relative error vs number of QR sensors
(b) L2 error for single and ensemble PINN
Figure 4.13: Sensitivity of QR sensors and uncertainty quantification of PINN
error tends to plateau beyond a certain sensor count. One of the primary
goals of this research is to pinpoint both the optimal and minimum necessary sensor locations for the PINN’s flow field reconstruction. Using
QR pivoting, 10 optimal sensors were chosen, offering a balance between
attaining the desired accuracy and constraining the number of sensors deployed.
Uncertainty quantification of PINN
This research compares single and ensemble PINN models. The single
model, trained on a minimal sensor dataset, is tested against the ensemble
model, which uses multiple models each starting from random initialisations. Both models use the same sensor parameters. The ensemble model,
although more computationally demanding, provides superior generalisability, and robust predictions by avoiding overfitting and underfitting as
shown in Figure 4.13b. In addition, it provides the mean predictions and
estimates the uncertainty, aiding in decision-making by indicating trustworthiness and ambiguity in the model’s predictions. Particularly in complex systems, these features are invaluable. The rest of the study concentrates on ensemble PINN predictions.
Constant heat flux
After training the PINN, it was then deployed to ascertain the temperature
profile surrounding the square, as depicted in Figure 4.14a. The graph
demonstrates a temperature profile from the PINN that aligns with the
trend of the values inferred from RANS around the square. The figure
shown was captured at a time of t ∗ = 40 and exhibits a peak L2 error of
8% on the right side where the flow interacts the least with the square.
Notably, the model’s uncertainty diminishes in areas with available sensor
Mälardalen University Press Dissertations
47
Machine Learning Techniques for Enhanced Heat Transfer Modelling
(a) Temperature profile around the square
cylinder
1
(b) Unknown parameters C1 = Ri and C2 = Pec
for
constant heat flux
Figure 4.14: Sensitivity of QR sensors and uncertainty quantification of PINN
data, which was expected. Yet, there is considerable uncertainty on the
left wall. A plausible reason for this could be an insufficiency of temperature data in that area for the PINN model. Likewise, there is notable
uncertainty on the right and bottom walls, potentially due to the impacts
of unstable flow and buoyancy.
During the training of the PINN, another assumption was that the nondimensional numbers, Richardson number (Ri) and Péclet number (Pec),
in the governing equation were unknown. Hence, the PINN was additionally tasked to predict these numbers during the training process. The
predicted values from the PINN and the actual values from RANS are pro1
vided in Figure 4.14b. Here, C1 stands for Ri and C2 represents Pec
. It’s
evident that at the start of training, both numbers are rather random and
exhibit substantial fluctuation and uncertainty. However, when the model
begins to stabilise around 100k epochs, the means of both C1 and C2 draw
closer to the actual values from RANS. On the flip side, the uncertainty
persists to be significant despite a consistent reduction. This can be ascribed to the uncertain thermal boundary condition in the square, given
both C1 and C2 play a role in the heat transfer in the governing equation.
Constant temperature
The analysis in this section focuses on a situation where the source wall’s
temperature remains constant. The insights for this case study have been
garnered from a DNS simulation, as indicated in (Fraigneau, Y., 2019).
Optimal sensor points were identified by applying QR pivoting based on
the temperature distribution. These sensors were then leveraged by the
PINN to reconstruct the flow field. The dynamics, as reconstructed by
the PINN at the 80th time-step, are shown in Figure 4.15. The left part
of the figure exhibits predictions made by the PINN, the centre shows the
48
Jerol Soibam
Chapter 4. Results and discussion
Figure 4.15: Comparison of flow field profiles for constant heat flux at t = 80, as
predicted by PINN and DNS, with corresponding L2 error for u∗ , v∗ ,
p∗ , θ ∗
outcomes from the DNS simulation, and the right side portrays the L2
relative error between the two sets of results. The figure demonstrates that
the error for u∗ , v∗ , and p∗ is below 3%. However, for the temperature θ ∗ ,
the error margin is around 10% close to the square wall, with the majority
of the error stemming from the square’s front side.
The PINN model’s predictive accuracy was evaluated at a random
wake-region point (x∗ =2.5, y∗ =0.1). This location was neither included
in the model’s training data nor was it a QR sensor point, making it a
reliable test of the model’s capacity to generalise from its training. As
depicted in Figure 4.16, the PINN model successfully predicted the components (u∗ , v∗ , p∗ , and θ ∗ ) of the flow dynamics at this point, and these
predictions exhibited strong alignment with the DNS data. This consistency with existing data reinforces the model’s predictive strength and reliability, demonstrating that the model performs well beyond its training
dataset. Despite its overall success, the model did show a higher level of
uncertainty in predicting the pressure component. Even with the increased
deviation in pressure prediction, the model’s uncertainty still manages to
encapsulate the range of the actual DNS values. This broad coverage provides an understanding of the model’s resilience and adaptability, despite
its fluctuating behaviour over time.
For the remaining components, the PINN model showed a notably low
level of uncertainty, highlighting its robustness and ability to maintain
Mälardalen University Press Dissertations
49
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Figure 4.16: Predictive performance of the PINN at random point which was not
a part of QR sensor nor training
accuracy. The model also demonstrated resistance to overfitting, which
means it does not rigidly depend on the specific data it was trained on
and can accurately predict data beyond its training set. The versatility of
this model had been previously established when it was able to successfully reconstruct the full flow field using the QR sensor location data, and
this additional test further emphasises that characteristic. For more details
please refer to Paper VI.
4.3
Discussion of the research questions
The advent of DL, in fluid mechanics and heat transfer has opened a transformative shift in research methodologies. This thesis assesses the applicability of ML methods to navigate the complexities of heat transfer
problems. Although the research questions posed are general in nature,
the conclusions drawn are based on the specific studies conducted in this
study. Within this framework, the potential of ML models to accurately
represent and predict these dynamics is explored. Additionally, the integration of established physical principles, the challenges of limited data,
and the importance of uncertainty quantification in ensuring model transparency are examined.
50
Jerol Soibam
Chapter 4. Results and discussion
RQ 1: How effectively can deep learning techniques capture the intricate nature of heat transfer based on data, and how successfully can these
techniques extrapolate to unseen scenarios? (Papers I & II)
In recent years, ML techniques have been applied to fluid mechanics and
heat transfer problems, driven by an influx of data from experiments and
simulations. The study in Papers I and II showcased data-driven approaches for subcooled boiling heat transfer. Numerical data was used
to train the deep learning models for predicting temperature and void
fraction within the minichannel. Three distinct deep learning methods —
DNN, MC dropout, and deep ensemble — were explored. After training,
the models were validated against a validation dataset and subsequently
tested using the evaluation dataset, as highlighted in section 4.1. These
trained models demonstrated adeptness in capturing the complexities of
subcooled boiling heat transfer. They proficiently predicted the temperature and void fraction in the minichannel, aligning well with the numerical
values. The models further demonstrated their ability to predict the quantities of interest with acceptable accuracy, even under conditions of a high
heat flux of q” = 40,000 W m−2 and an inlet velocity of u = 0.2 ms−1 .
Notably, these conditions were outside the training dataset, indicating that
the model used in this study can extrapolate unseen datasets.
Beyond accurate predictions, the models discerned intrinsic patterns
in the data, thereby providing insights into the physical process of the
subcooled boiling process. For instance, an increased inlet velocity led
to a delay in the onset of nucleate boiling, for high heat flux conditions.
Despite this shift in physics, the models adeptly predicted the onset of
nucleate boiling. This lag can be linked to the slower attainment of saturated temperature at increased inlet velocities, causing a subsequent delay
in bubble formation within the wall of minichannel. The MC dropout and
deep ensemble are based on a probabilistic approach, further highlighting
the models’ predictive performance. As the void fraction approaches a
value of 0.1, a surge in uncertainty was observed, possibly due to the nucleation of bubbles near the wall, prompting a phase transition from liquid
to vapour. Of all the models, the deep ensemble consistently delivered
superior predictive performance for both interpolation and extrapolation
datasets, with a relative error under 2%. The models examined in this
thesis proved their ability to detect shifts in physical states, such as phase
transitions and delays in the onset of nucleate boiling, underscoring the
intricate dynamics of subcooled boiling in the minichannel.
RQ 2: How to incorporate known physics of heat transfer in deep learning
models to improve the generalisability in estimating unknown parameters
Mälardalen University Press Dissertations
51
Machine Learning Techniques for Enhanced Heat Transfer Modelling
and system behaviours? (Papers IV & VI)
Integrating well-understood physics into deep learning models, especially
in the domain of heat transfer, offers substantial improvements in their
ability to estimate unknown parameters, such as thermal boundaries and
non-dimensional parameters. One of the main obstacles in heat transfer
simulations is often the lack of accurate thermal boundary conditions, rendering traditional methods inadequate for certain problems.
In response, the research presented in this thesis utilises the capabilities of inverse PINN. The study delves into both forced and mixed convection heat transfer within enclosures, focusing on scenarios with unknown
thermal boundaries. To address this, the Navier-Stokes and energy equations are enforced at random points in the domain, adhering to known
physical laws. The key findings of the research highlight the efficacy of
the PINN model. Even with a limited set of sensor data, it effectively represents the entire flow field inside the domain and aligns closely with temperature profiles from numerical studies. Additionally, the PINN model
accurately predicted the Péclet and Richardson numbers, which govern
the flow behaviour within the domain. A standout observation concerns
the crucial role of sensor placement. The model’s accuracy and reliability
often depend on this, and methods based on the energy equation’s residual and QR pivoting proved particularly effective. To enhance the robustness and reliability of predictions, the research introduced an ensemble
approach to PINN, resulting in improved accuracy and predictive confidence. In summary, the findings underscore the importance of strategic
sensor placement and ensemble methodologies in ensuring accurate and
reliable predictions, vital for the complexities of industrial heat transfer
systems.
RQ 3: How can machine learning methods secure reliable prediction outcomes in situations where heat transfer data are scarce? (Papers III, IV
& V)
ML models rely on data to make informed predictions. Their predictive
performance depends on the quality and amount of data. When data is
limited, the model’s performance can be significantly compromised, potentially affecting its predictive accuracy and decision-making abilities.
Therefore, this thesis emphasises selecting appropriate ML algorithms and
optimal sensor placements to address data limitations. The thesis also explores the use of transfer learning to enhance model performance and to
reuse trained models without relying on extensive datasets.
52
Jerol Soibam
Chapter 4. Results and discussion
Methods such as pGPR, CNN with transfer learning, and PINN were
adopted to rely on smaller datasets without deviating from the physics
or sacrificing model accuracy. In the case of pGPR, hypothetical data
points are considered rather than using the entire dataset during training
and prediction. This method allows for efficient data use while also indicating the prediction’s degree of confidence. With inverse-PINN, only a
few sensors’ data are needed to estimate the flow field and to predict the
temperature around the source wall. The PINN model was further coupled
with transfer learning to estimate the flow field in the enclosure domain
when the source shape and number of sources change. Transfer learning
was similarly used in the CNN architecture to track and segment bubbles in subcooled boiling flow. Results from Paper V suggest that transfer learning can significantly enhance model accuracy. Additionally, the
computational demand for training the algorithm can be notably reduced,
yielding the same or higher accuracy levels depending on the application.
For instance, in tracking the bubbles, the computational cost was halved
compared to training the model from scratch, while producing more reliable predictions.
RQ 4: How can uncertainty quantification contribute to enhancing the
robustness and reliability of machine learning models utilised to simulate
and predict heat transfer? (Papers III, II & VI)
ML models, particularly deep learning architectures, often function as
"black boxes", making their internal logic elusive. In fields like heat
transfer, where it’s essential to understand the underlying mechanisms,
such opacity poses challenges. This thesis emphasises the importance of
quantifying uncertainty within these models to enhance their predictive
capabilities, thereby offering a layer of transparency and deeper insight
into the model’s behaviour.
In the case study of subcooled boiling, two probabilistic models (MC
dropout and deep ensemble) were examined. The study revealed that probabilistic models outperform the deterministic DNN. For example, while
predicting the onset of nucleate boiling using a deterministic DNN model,
it showed an early nucleation site compared to the numerical value. In
contrast, predictions from the probabilistic model (DE) aligned more closely with the numerical value, signifying its robustness. Moreover, this
probabilistic model displayed a heightened uncertainty in specific regions,
possibly linked to the abrupt shift in physics from liquid to vapour (generation of many nucleation sites). Such indications of uncertainty empower
engineers to gauge the model’s reliability and understand its behaviour in
particular scenarios. In another instance, a pGPR model, rooted in the
Mälardalen University Press Dissertations
53
Machine Learning Techniques for Enhanced Heat Transfer Modelling
Bayesian approach, was employed to predict the CHF. The model managed to predict the CHF within a range of ±10%, while most empirical correlations deviated by about ±15%. Comparisons with experimental values
indicated minor deviations in the model’s predictions. Yet, a pervasive
uncertainty was observed across all predictions, likely due to the inherent
nonlinearity and experimental uncertainty present in the CHF dataset.
To address potential issues like overfitting and underfitting, an ensemblePINN was introduced, resulting in predictions that were more robust and
adaptable than those derived from a single-PINN. Assessing the uncertainty of a PINN is crucial for understanding its generalisability, especially
when adapting to changing flow dynamics or when the model is trained
with limited sensor data. The L2 relative error from the ensemble-PINN
consistently surpassed that of the single-PINN. Notably, the ensemblePINN maintained consistent predictions even when the dynamic changes
due to vortex shedding over time. Furthermore, the model’s uncertainty
accurately covered the range of actual numerical values. This ability to
quantify uncertainty brings transparency and supports informed decisionmaking. In such scenarios, gauging the potential outcomes’ range and
likelihood is as vital as the prediction itself.
Limitations
The incorporation of ML into heat transfer research, especially in areas
like subcooled boiling and convective heat transfer, has offered transformative insights. However, several challenges and limitations exist in the
methodologies and outcomes of the studies presented in this thesis.
The efficacy of these models is intrinsically linked to the quality, diversity, and availability of data. In the absence of adequate data, the models
may falter, particularly in depicting subcooled boiling heat transfer. An
in-depth understanding of subcooled boiling necessitates a thorough analysis of the number of input features. Furthermore, these models might
struggle to adapt to scenarios outside their training data.
In terms of bubble detection, while the current method is competent,
it has its hurdles. If the boiling process alters or produces larger bubbles
(bubbly flow), the algorithm needs revising. There’s also a pressing need
for better ways to manage and process the overlapping bubble images from
the mask produced by the CNN model. To ensure the model’s stability, a
detailed sensitivity analysis is crucial, especially when faced with varied
noise conditions and experiments.
Integrating sensor data, though advantageous, brings about uncertainties. Accurate sensor placement, underpinned by preliminary data, is crucial, especially when employing techniques like QR pivoting. Incorrect
54
Jerol Soibam
Chapter 4. Results and discussion
sensor placements or data collection errors can adversely impact predictions. Additionally, while the studies underscore the potential of techniques like transfer learning in situations with scarce data, the efficacy
of such techniques is still bound by the quality of the base model and
the similarities between the source and target domains. Other elements,
such as the enforcement of specific boundary conditions or the selection
of appropriate model hyperparameters, are decisive for the model’s accuracy. While the PINN model excels in portraying patterns like vortex
shedding and heat transfer mechanisms in the domain, certain aspects remain opaque, highlighting the need for further clarity. Finally, although
the models identify uncertainties, comprehensively capturing all types of
uncertainty, especially in noisy data scenarios, remains a priority.
Mälardalen University Press Dissertations
55
5
Conclusions and future work
This chapter presents the major conclusions of the thesis.
The collective conclusion in this thesis, drawn from the six papers,
highlights the successful application of deep learning techniques and statistical methods in improving the understanding, modelling, and prediction of various aspects of boiling and convective heat transfer processes.
The following overarching insights have been gleaned from this extensive
body of work:
• The implementation of supervised DNN models, yielded effective
results in simulating subcooled boiling heat transfer. Despite the
observed limitations in the extreme extrapolation of void fraction
fields, the model displayed an impressive level of accuracy, especially when predicting temperature fields. The comparison of deterministic and probabilistic deep learning models further illustrated
the advantages of the latter. Probabilistic models demonstrated superior performance in accurately predicting the quantities of interest, and in accounting for uncertainties in the model. The results
indicated the successful capture and reproduction of the underlying
physics of the boiling process, even under extreme extrapolation
conditions and addressed in RQ 1&4.
• The application of pGPR for predicting CHF resulted in improved
prediction accuracy compared to and empirical method and lookup table based approaches. By harnessing information available in
experimental data, the pGPR model provided insights into the sensitivity of CHF to various operating conditions. Furthermore, this
method demonstrated the little need for data to accurately predict
the CHF which addresses RQ 3&4.
• A CNN model was successfully used to understand local bubble dynamics and track bubbles even under less favourable imaging conditions. The model demonstrated robustness against noise and fluctuations and was adaptable to different flow regimes. In summary,
the CNN model employed in this study demonstrates the considerable potential for understanding bubble dynamics in boiling processes and can serve as a robust foundation for future research in
Mälardalen University Press Dissertations
57
Machine Learning Techniques for Enhanced Heat Transfer Modelling
this field. Moreover, the model’s flexibility to adapt to different flow
regimes through transfer learning enhances its applicability and this
addresses RQ 1&3.
• In response to RQ 2&3, the use of inverse PINN has proven to
be a crucial tool in reconstructing the unknown thermal boundary
conditions. These networks successfully reconstruct thermal distributions and flow fields across a variety of conditions and shapes.
By applying transfer learning techniques to PINNs, it was able to
enhance the computational efficiency while retaining an acceptable
accuracy. This approach not only provided insights into the complex thermal dynamics but also opened up opportunities for optimisation and predictive modelling in heat transfer applications.
• Finally, the effectiveness of PINNs was further emphasised through
the study of mixed convection flows. Utilising a QR pivoting strategy for optimal sensor placement significantly improved prediction
accuracy, with ensemble models outperforming single PINN model
despite higher computational requirements. The ensemble models
also introduced the important aspect of predictive uncertainty quantification and addressed RQ 2,3, &4.
In conclusion, the findings from these studies strongly suggest that
ML methods, specifically deep neural networks (fully connected / convolutional), Gaussian process regression, and physics-informed neural networks, provide promising tools for capturing the intricate physics involved,
predicting quantities of interest accurately, and accounting for the inherent
uncertainties.
Future work
Based on the findings and results obtained from this thesis, some relevant
future work are summarised below:
• To investigate deeper the quality and impact of input features specific to subcooled boiling heat transfer. Further, explore how these
models are influenced by data availability and refine them by integrating known physics principles.
• Refine the existing CNN model for bubble segmentation to accommodate a wider range of experimental and flow conditions. To delve
further into the dynamics of subcooled boiling to establish correlations that can be instrumental for numerical simulations.
58
Jerol Soibam
Chapter 5. Conclusions and future work
• To integrate aleatoric uncertainty into the models to enhance their
understanding and adaptability in managing noisy data, thereby establishing a comprehensive and robust uncertainty framework.
• Integration of QR sensor with energy residuals to better optimise
sensor locations, while iteratively embedding this within the PINN
framework. Furthermore, to incorporate sensor noise to assess the
model’s sensitivity.
• Expand the existing methodology of PINN to 3D transient simulations and investigate its applicability in engineering systems of
HVDC electronic cooling.
Mälardalen University Press Dissertations
59
Bibliography
Alayil, R. and Balaji, C. (2015). Conjugate heat transfer in latent heat
thermal storage system with cross plate fins. Journal of Heat Transfer,
137(10):102302.
Albawi, S., Mohammed, T. A., and Al-Zawi, S. (2017). Understanding of
a convolutional neural network. In 2017 International Conference on
Engineering and Technology (ICET), pages 1–6.
Alifanov, O. M. (2012). Inverse heat transfer problems. Springer Science
& Business Media.
Ardabili, S., Mosavi, A., and Felde, I. (2023). Machine learning in heat
transfer: Taxonomy, review and evaluation. In 2023 IEEE 17th International Symposium on Applied Computational Intelligence and Informatics (SACI), pages 000433–000442. IEEE.
Baraldi, P. and Mangili, Fand Zio, E. (2015). A prognostics approach
to nuclear component degradation modeling based on gaussian process
regression. Progress in Nuclear Energy, 78:141–154.
Benard, N., Pons-Prats, J., Periaux, J., Bugeda, G., Braud, P., Bonnet, J.,
and Moreau, E. (2016). Turbulent separated shear flow control by surface plasma actuator: Experimental optimization by genetic algorithm
approach. Experiments in Fluids, 57(2):22.
Bhamare, D. K., Saikia, P., Rathod, M. K., Rakshit, D., and Banerjee, J.
(2021). A machine learning and deep learning based approach to predict
the thermal performance of phase change material integrated building
envelope. Building and Environment, 199:107927.
Bolya, D., Zhou, C., Xiao, F., and Lee, Y. J. (2022). Yolact++ better realtime instance segmentation. IEEE Transactions on Pattern Analysis and
Machine Intelligence, 44(2):1108–1121.
Breiman, L. (1996). Bagging predictors. Machine learning, 24(2):123–
140.
Brunton, S. L. and Noack, B. R. (2015). Closed-loop turbulence control:
Progress and challenges. Applied Mechanics Reviews, 67(5).
Mälardalen University Press Dissertations
61
Brunton, S. L., Noack, B. R., and Koumoutsakos, P. (2020). Machine
learning for fluid mechanics. Annual Review of Fluid Mechanics,
52:477–508.
Brunton, S. L., Proctor, J. L., and Kutz, J. N. (2016). Discovering governing equations from data by sparse identification of nonlinear dynamical systems. Proceedings of the national academy of sciences,
113(15):3932–3937.
Cai, S., Wang, Z., Fuest, F., Jeon, Y. J., Gray, C., and Karniadakis, G. E.
(2021a). Flow over an espresso cup: inferring 3-d velocity and pressure
fields from tomographic background oriented schlieren via physicsinformed neural networks. Journal of Fluid Mechanics, 915.
Cai, S., Wang, Z., Wang, S., Perdikaris, P., and Karniadakis, G. E.
(2021b). Physics-informed neural networks for heat transfer problems.
Journal of Heat Transfer, 143(6).
Celik, N., Kurtbas, I., Yumusak, N., and Eren, H. (2009). Statistical regression and artificial neural network analyses of impinging jet experiments. Heat and mass transfer, 45(5):599–611.
Colaco, M. J., Orlande, H. R., and Dulikravich, G. S. (2006). Inverse and
optimization problems in heat transfer. Journal of the Brazilian Society
of Mechanical Sciences and Engineering, 28:1–24.
Colvert, B., Alsalman, M., and Kanso, E. (2018). Classifying vortex wakes using neural networks. Bioinspiration & biomimetics,
13(2):025003.
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. (2009).
Imagenet: A large-scale hierarchical image database. In 2009 IEEE
Conference on Computer Vision and Pattern Recognition, pages 248–
255.
Di Natale, L., Svetozarevic, B., Heer, P., and Jones, C. (2021). Deep reinforcement learning for room temperature control: a black-box pipeline
from data to policies. In Journal of Physics: Conference Series, volume
2042, page 012004. IOP Publishing.
Dracopoulos, D. C. (2013). Evolutionary learning algorithms for neural
adaptive control. Springer.
Drikakis, D. (2003). Advances in turbulent flow computations using highresolution methods. Progress in Aerospace Sciences, 39(6-7):405–424.
62
Jerol Soibam
Duraisamy, K., Iaccarino, G., and Xiao, H. (2019). Turbulence modeling
in the age of data. Annual Review of Fluid Mechanics, 51:357–377.
Fleming, P. J. and Purshouse, R. C. (2002). Evolutionary algorithms in
control systems engineering: a survey. Control engineering practice,
10(11):1223–1241.
Fraigneau, Y. (2019). 2d flow around a heated square-cylinder, re=50,
ra=5×106 . Available at: https://datasetmeca.lisn.upsaclay.fr. LISN
Université Paris-Saclay, Accessed: 2023-03-31.
Fu, Y. and Liu, Y. (2019). Bubgan: Bubble generative adversarial networks for synthesizing realistic bubbly flow images. Chemical Engineering Science, 204:35–47.
Fukami, K., Nabae, Y., Kawai, K., and Fukagata, K. (2019). Synthetic
turbulent inflow generator using machine learning. Physical Review
Fluids, 4(6):064603.
Gal, Y. and Ghahramani, Z. (2016). Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international
conference on machine learning, pages 1050–1059. ICML’16.
Geng, Z. and Wang, Y. (2020). Automated design of a convolutional neural network with multi-scale filters for cost-efficient seismic data classification. Nature communications, 11(1):1–11.
Goodfellow, I., Bengio, Y., Courville, A., and Bengio, Y. (2016). Deep
learning, volume 1. MIT press Cambridge.
Haas, T., Schubert, C., Eickhoff, M., and Pfeifer, H. (2020). Bubcnn: Bubble detection using faster rcnn and shape regression network. Chemical
Engineering Science, 216:115467.
He, K., Gkioxari, G., Dollár, P., and Girshick, R. (2017). Mask r-cnn.
In 2017 IEEE International Conference on Computer Vision (ICCV),
pages 2980–2988.
Hobold, G. M. and da Silva, A. K. (2019a). Automatic detection of the
onset of film boiling using convolutional neural networks and bayesian
statistics. International Journal of Heat and Mass Transfer, 134:262–
270.
Hobold, G. M. and da Silva, A. K. (2019b). Visualization-based nucleate
boiling heat flux quantification using machine learning. International
Journal of Heat and Mass Transfer, 134:511–520.
Mälardalen University Press Dissertations
63
Hughes, M. T., Kini, G., and Garimella, S. (2021). Status, challenges,
and potential for machine learning in understanding and applying heat
transfer phenomena. Journal of Heat Transfer, 143(12):120802.
Iten, R., Metger, T., Wilming, H., Del Rio, L., and Renner, R. (2020).
Discovering physical concepts with neural networks. Physical review
letters, 124(1):010508.
Jambunathan, K., Hartle, S., Ashforth-Frost, S., and Fontama, V. (1996).
Evaluating convective heat transfer coefficients using neural networks.
International Journal of Heat and Mass Transfer, 39(11):2329–2332.
Jiang, B., Ren, J., Hu, P., and Zhao, F. (2013). Prediction of critical heat
flux for water flow in vertical round tubes using support vector regression model. Progress in Nuclear Energy, 68:210–222.
Jiang, B., Zhou, J., Huang, X., and Wang, P. (2020). Prediction of critical
heat flux using gaussian process regression and ant colony optimization.
Annals of Nuclear Energy, 149:107765.
Jiménez, J. and Moin, P. (1991). The minimal flow unit in near-wall turbulence. Journal of Fluid Mechanics, 225:213–240.
Karniadakis, G. E., Kevrekidis, I. G., Lu, L., Perdikaris, P., Wang, S., and
Yang, L. (2021). Physics-informed machine learning. Nature Reviews
Physics, 3(6):422–440.
Kim, H. C., Baek, W., and Chang, S. H. (2000). Critical heat flux of water
in vertical round tubes at low pressure and low flow conditions. Nuclear
Engineering and design, 199(1-2):49–73.
Kim, Y. and Park, H. (2021). Deep learning-based automated and universal bubble detection and mask extraction in complex two-phase flows.
Scientific reports, 11(1):1–11, 8940.
Kingma, D. P. and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. Advances in neural
information processing systems, 25:1097–1105.
Lagaris, I. E., Likas, A., and Fotiadis, D. I. (1998). Artificial neural
networks for solving ordinary and partial differential equations. IEEE
transactions on neural networks, 9(5):987–1000.
64
Jerol Soibam
Lakshminarayanan, B., Pritzel, A., and Blundell, C. (2017). Simple and
scalable predictive uncertainty estimation using deep ensembles. In
Advances in neural information processing systems, pages 6402–6413.
Lee, C., Kim, J., Babcock, D., and Goodman, R. (1997). Application of
neural networks to turbulence control for drag reduction. Physics of
Fluids, 9(6):1740–1747.
Lee, H. and Kang, I. S. (1990). Neural algorithm for solving differential
equations. Journal of Computational Physics, 91(1):110–131.
Ling, J., Jones, R., and Templeton, J. (2016a). Machine learning strategies for systems with invariance properties. Journal of Computational
Physics, 318:22–35.
Ling, J., Kurzawski, A., and Templeton, J. (2016b). Reynolds averaged
turbulence modelling using deep neural networks with embedded invariance. Journal of Fluid Mechanics, 807:155–166.
Ling, J. and Templeton, J. (2015). Evaluation of machine learning algorithms for prediction of regions of high reynolds averaged navier stokes
uncertainty. Physics of Fluids, 27(8):085103.
Lipton, Z. C. (2018). The mythos of model interpretability: In machine
learning, the concept of interpretability is both important and slippery.
Queue, 16(3):31–57.
Lucor, D., Agrawal, A., and Sergent, A. (2022). Simple computational
strategies for more effective physics-informed neural networks modeling of turbulent natural convection. Journal of Computational Physics,
456:111022.
Lumley, J. (1970). Stochastic tools in turbulence, academic press, new
york.
Maulik, R., San, O., Rasheed, A., and Vedula, P. (2019). Subgrid modelling for two-dimensional turbulence using neural networks. Journal
of Fluid Mechanics, 858:122–144.
Milano, M. and Koumoutsakos, P. (2002). Neural network modeling for
near wall turbulent flow. Journal of Computational Physics, 182(1):1–
26.
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare,
M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G.,
et al. (2015). Human-level control through deep reinforcement learning.
nature, 518(7540):529–533.
Mälardalen University Press Dissertations
65
Molland, A. F. and Turnock, S. R. (2007). 6 - theoretical and numerical
methods. In Molland, A. F. and Turnock, S. R., editors, Marine Rudders and Control Surfaces, pages 233–311. Butterworth-Heinemann,
Oxford.
Murata, T., Fukami, K., and Fukagata, K. (2020). Nonlinear mode decomposition with convolutional neural networks for fluid dynamics. Journal
of Fluid Mechanics, 882.
Otter, D. W., Medina, J. R., and Kalita, J. K. (2021). A survey of the
usages of deep learning for natural language processing. IEEE Transactions on Neural Networks and Learning Systems, 32(2):604–624.
Parish, E. J. and Duraisamy, K. (2016). A paradigm for data-driven predictive modeling using field inversion and machine learning. Journal of
Computational Physics, 305:758–774.
PHAN, M. Q., JUANG, J.-N., and HYLAND, D. C. (1995). On neural
networks in identification and control of dynamic systems. In Wave
Motion, Intelligent Structures And Nonlinear Mechanics, pages 194–
225. World Scientific.
Piatek,
˛
W., Oleksiak, A., and Da Costa, G. (2015). Energy and thermal
models for simulation of workload and resource management in computing systems. Simulation Modelling Practice and Theory, 58:40–54.
Podvin, B., Fraigneau, Y., Jouanguy, J., and Laval, J.-P. (2010). On selfsimilarity in the inner wall layer of a turbulent channel flow. Journal of
fluids engineering, 132(4).
Pollard, A., Castillo, L., Danaila, L., and Glauser, M. (2016). Whither
turbulence and big data in the 21st century? Springer.
Proctor, J. L., Brunton, S. L., and Kutz, J. N. (2016). Dynamic mode
decomposition with control. SIAM Journal on Applied Dynamical Systems, 15(1):142–161.
Rabault, J., Kuchta, M., Jensen, A., Réglade, U., and Cerardi, N. (2019).
Artificial neural networks trained through deep reinforcement learning
discover control strategies for active flow control. Journal of fluid mechanics, 865:281–302.
Rabault, J. and Kuhnle, A. (2019). Accelerating deep reinforcement learning strategies of flow control through a multi-environment approach.
Physics of Fluids, 31(9):094105.
66
Jerol Soibam
Rabhi, A., Aslanidou, I., Kyprianidis, K., and Fdhila, R. B. (2021). Onset of nucleate boiling model for rectangular upward narrow channel:
Cfd based approach. International Journal of Heat and Mass Transfer,
165:120715.
Raissi, M., Babaee, H., and Karniadakis, G. E. (2019a). Parametric
gaussian process regression for big data. Computational Mechanics,
64(2):409–416.
Raissi, M., Perdikaris, P., and Karniadakis, G. E. (2019b). Physicsinformed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707.
Raissi, M., Wang, Z., Triantafyllou, M. S., and Karniadakis, G. E. (2019c).
Deep learning of vortex-induced vibrations. Journal of Fluid Mechanics, 861:119–137.
Raissi, M., Yazdani, A., and Karniadakis, G. E. (2020). Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations.
Science, 367(6481):1026–1030.
Rassoulinejad-Mousavi, S. M., Al-Hindawi, F., Soori, T., Rokoni, A.,
Yoon, H., Hu, H., Wu, T., and Sun, Y. (2021). Deep learning strategies for critical heat flux detection in pool boiling. Applied Thermal
Engineering, 190:116849.
Reddy, G., Celani, A., Sejnowski, T. J., and Vergassola, M. (2016). Learning to soar in turbulent environments. Proceedings of the National
Academy of Sciences, 113(33):E4877–E4884.
Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. (2016). You only
look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages
779–788, Las Vegas, NV, USA.
Rowley, C. W. and Dawson, S. T. (2017). Model reduction for flow analysis and control. Annual Review of Fluid Mechanics, 49:387–417.
Rowley, C. W., MEZI?, I., Bagheri, S., Schlatter, P., Henningson, D., et al.
(2009). Spectral analysis of nonlinear flows. Journal of fluid mechanics,
641(1):115–127.
San, O. and Maulik, R. (2018). Machine learning closures for model order
reduction of thermal fluids. Applied Mathematical Modelling, 60:681–
710.
Mälardalen University Press Dissertations
67
Sarghini, F., De Felice, G., and Santini, S. (2003). Neural networks based
subgrid scale modeling in large eddy simulations. Computers & fluids,
32(1):97–108.
Scheiff, V., Aslanidou, I., Kyprianidis, K., and Bel Fdhila, R. (2023). Experimental study of nucleate boiling dynamics in a rectangular minichannel set-up. In 8th Thermal and Fluids Engineering Conference
(TFEC).
Schmidt, M. and Lipson, H. (2009). Distilling free-form natural laws from
experimental data. science, 324(5923):81–85.
Serra, P. L., Masotti, P. H., Rocha, M. S., de Andrade, D. A., Torres,
W. M., and de Mesquita, R. N. (2020). Two-phase flow void fraction estimation based on bubble image segmentation using randomized
hough transform with neural network (rhtn). Progress in Nuclear Energy, 118:103133.
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez,
A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018). A
general reinforcement learning algorithm that masters chess, shogi, and
go through self-play. Science, 362(6419):1140–1144.
Singh, A. P., Medida, S., and Duraisamy, K. (2017). Machine-learningaugmented predictive modeling of turbulent separated flows over airfoils. AIAA journal, 55(7):2215–2227.
Sinha, K. N. R., Kumar, V., Kumar, N., Thakur, A., and Raj, R. (2021).
Deep learning the sound of boiling for advance prediction of boiling
crisis. Cell Reports Physical Science, 2(3):100382.
Suh, Y., Bostanabad, R., and Won, Y. (2021). Deep learning predicts
boiling heat transfer. Scientific reports, 11, 5622(1):1–10.
Sun, L., Gao, H., Pan, S., and Wang, J.-X. (2020). Surrogate modeling for
fluid flows based on physics-constrained deep learning without simulation data. Computer Methods in Applied Mechanics and Engineering,
361:112732.
Taira, K., Brunton, S. L., Dawson, S. T., Rowley, C. W., Colonius, T.,
McKeon, B. J., Schmidt, O. T., Gordeyev, S., Theofilis, V., and Ukeiley,
L. S. (2017). Modal analysis of fluid flows: An overview. AIAA Journal,
55(12):4013–4041.
68
Jerol Soibam
Taira, K., Hemati, M. S., Brunton, S. L., Sun, Y., Duraisamy, K., Bagheri,
S., Dawson, S. T., and Yeh, C.-A. (2020). Modal analysis of fluid flows:
Applications and outlook. AIAA journal, 58(3):998–1022.
Traverso, T., Coletti, F., Magri, L., Karayiannis, T. G., and Matar, O. K.
(2023). A machine learning approach to the prediction of heat-transfer
coefficients in micro-channels. arXiv preprint arXiv:2305.18406.
Wang, C.-Y., Bochkovskiy, A., and Liao, H.-Y. M. (2022). Yolov7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696.
Wang, Q., Li, X., Xu, C., Yan, T., and Li, Y. (2021). Bubble recognizing and tracking in a plate heat exchanger by using image processing
and convolutional neural network. International Journal of Multiphase
Flow, 138:103593.
Yang, L., Zhang, D., and Karniadakis, G. E. (2020). Physics-informed
generative adversarial networks for stochastic differential equations.
SIAM Journal on Scientific Computing, 42(1):A292–A317.
Zhang, D., Lu, L., Guo, L., and Karniadakis, G. E. (2019). Quantifying total uncertainty in physics-informed neural networks for solving
forward and inverse stochastic problems. Journal of Computational
Physics, 397:108850.
Zhu, Y., Zabaras, N., Koutsourelakis, P.-S., and Perdikaris, P. (2019).
Physics-constrained deep learning for high-dimensional surrogate modeling and uncertainty quantification without labeled data. Journal of
Computational Physics, 394:56–81.
Mälardalen University Press Dissertations
69