DEGREE PROJECT IN MATERIALS SCIENCE AND ENGINEERING,
SECOND CYCLE, 30 CREDITS
STOCKHOLM, SWEDEN 2020
Combined CALPHAD and
Machine Learning for
Property Modelling
KYLE PAULUS
KTH ROYAL INSTITUTE OF TECHNOLOGY
SCHOOL OF INDUSTRIAL ENGINEERING AND MANAGEMENT
Combined CALPHAD and
Machine Learning for
Property Modelling
KYLE PAULUS
TTMVM Materials Design
Date: July 7, 2020
Supervisor: Johan Jeppsson
Examiner: Malin Selleby
Engineering Materials Science
Host company: Thermo-Calc Software AB
Abstract
Techniques to improve the speed at which materials are researched and developed has been conducted by investigating the machine learning methodology.
These techniques offer solutions to connect the length scales of material properties from atomistic and chemical features using materials databases generated from collected data. In this assessment, two material informatics methodologies are used to predict material properties in steels and nickel based superalloys using this approach. Martensite start temperature and sigma phase
amount as a function of input composition has been modelled with the use of
machine learning algorithms. The experimental methodology had a collection
of over 2000 unique experimental martensite start temperature points. This
yielded important information on higher order interactions for the martensite
start temperature, and a root mean square error (rmse) of 29 Kelvin using ensemble tree based algorithms. The metamodel was designed using an artificial
neural network from TensorFlow’s library to predict sigma phase fraction and
its composition. The methodology for building, calculating, and using data
from TC-Python will be laid out. This generates a model that would generalize sigma phase fraction 97.9 % of Thermo-Calc’s equilibrium model in 7.1
seconds compared to 227 hours needed in the simulation to calculate the same
amount of material property data.
iii
Sammanfattning
Tekniker för att förbättra hastigheten med material som forskas och utvecklas har genomförts genom att undersöka metodik för maskininlärning. Dessa
tekniker erbjuder lösningar för att ansluta längdskalorna för materialegenskaper från atomistiska och kemiska egenskaper med hjälp av materialdatabaser
genererade från insamlade data. I denna bedömning används två materialinformatikmetoder för att förutsäga materialegenskaper i stål och nickelbaserade superlegeringar med denna metod. Martensite-starttemperatur och sigmafasmängd som en funktion av ingångssammansättningen har modellerats med
användning av maskininlärningsalgoritmer. Den experimentella metoden hade en samling av över 2000 unika experimentella starttemperaturpunkter för
martensit. Detta gav viktig information om interaktioner med högre ordning
för martensit-starttemperaturen och ett root-medelvärde-kvadratfel (rmse) på
29 Kelvin med användning av ensemble-trädbaserade algoritmer. Metamodellen designades med hjälp av ett artificiellt neuralt nätverk från TensorFlows
bibliotek för att förutsäga sigma-fasfraktion och dess sammansättning. Metoden för att bygga, beräkna och använda data från TC-Python kommer att anges.
Detta genererar en modell som skulle generalisera sigma-fasfraktion 97,9 % av
Thermo-Calcs jämviktsmodell på 7,1 sekunder jämfört med 227 timmar som
behövs i simuleringen för att beräkna samma mängd materialegenskapsdata.
iv
Preface
To Mom: sacrificing every penny you had to put me through school. Thank
you.
I would like to thank all the people at Thermo-Calc.
-Kyle Paulus 04/07/2020
v
Contents
1
Introduction
1.1 Research Question . . . . . . . . . . . . . . . . . . . . . . .
1
1
2
Background
2.1 Materials Informatics . . . . . . . . . . . . . . . . . . . . .
2.1.1 Why is it Important? - Social, Ethical, and Environmental Considerations . . . . . . . . . . . . . . . .
2.1.2 The Data . . . . . . . . . . . . . . . . . . . . . . .
2.1.3 Machine Learning Tool Box . . . . . . . . . . . . .
2.1.4 Machine Learning Methodology . . . . . . . . . . .
2.1.5 Connecting the Length Scales . . . . . . . . . . . .
2.2 Ni-Based Alloys . . . . . . . . . . . . . . . . . . . . . . . .
2.2.1 Metamodel for Sigma Phase Amount . . . . . . . .
2.3 Martensite in Fe-Based Alloys . . . . . . . . . . . . . . . .
2.3.1 Experimental Model for Martensite . . . . . . . . .
.
2
2
.
.
.
.
.
.
.
.
.
2
4
5
5
6
7
7
8
9
Martensite
3.1 Methodology . . . . . . . . . . . . . . . . . . . . .
3.1.1 Gathering the Data . . . . . . . . . . . . . .
3.1.2 Recognising the Machine Learning Problem .
3.2 Ensemble Methods . . . . . . . . . . . . . . . . . .
3.2.1 Entropy . . . . . . . . . . . . . . . . . . . .
3.2.2 Random Forest . . . . . . . . . . . . . . . .
3.2.3 Adaboost . . . . . . . . . . . . . . . . . . .
3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . .
3.3.1 Initial benchmark of Uncleaned Dataset . . .
3.3.2 Cleaning the Dataset . . . . . . . . . . . . .
3.3.3 Feature Importance . . . . . . . . . . . . . .
3.3.4 Feature Selection and Generation . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
11
11
11
12
12
12
13
13
14
14
14
24
25
3
vi
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
CONTENTS
3.3.5
3.3.6
3.3.7
3.3.8
4
Cleaned Dataset Results . . . . . . . . . .
Comparison to the Thermodynamic Model
Other Machine Learning Platforms . . . .
Discussion . . . . . . . . . . . . . . . . .
Ni based Alloys
4.1 Metamodelling . . . . . . . . . . .
4.2 Artificial Neural Network . . . . . .
4.2.1 Structure . . . . . . . . . .
4.2.2 Features . . . . . . . . . . .
4.3 Metamodelling Methodology . . . .
4.3.1 Creating data . . . . . . . .
4.3.2 Calculating the Data . . . .
4.3.3 Using the Data . . . . . . .
4.4 Results . . . . . . . . . . . . . . . .
4.4.1 Benchmark Results . . . . .
4.4.2 Distribution Significance . .
4.4.3 Amount of Data Significance
4.4.4 Features Significance . . . .
4.4.5 Multioutput Example . . . .
4.4.6 Tuning the Hyperparameters
4.4.7 Best Result . . . . . . . . .
4.4.8 Comparison Test . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
25
27
29
32
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
34
34
35
35
36
36
36
38
39
40
40
42
44
44
46
47
48
49
5
Conclusions
52
5.1 Metamodel of Sigma . . . . . . . . . . . . . . . . . . . . . . 52
5.2 Experimental Model of Ms Start Temperature . . . . . . . . . 53
6
Future Works
6.1 Increase the Complexity of Neural Network Architecture
6.2 Hyperparameter Optimization . . . . . . . . . . . . . .
6.3 Materials Informatics Cloud Based Database . . . . . . .
6.4 Sample Around Equilibrium . . . . . . . . . . . . . . .
6.5 Metamodel Entire Database . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
54
54
54
55
55
56
7
Acknowledgements
57
8
References
58
vii
Chapter 1
Introduction
An average computer can perform 1.8E10 calculations per second. This type
of calculation power is significant for the discovery and optimization of new
materials. Computational materials design is based off of using this tool as a
way of working backwards analysing the performance of a material, its properties, micro-structure, and processing parameters. In these distinct individual
steps one can find computer science techniques used to compute and couple the
scales [1]. In this work, powerful, open source statistical algorithms are employed to investigate material properties. Metamodelling, or computer based
simplifications are used with machine learning as a tool to predict a targeted
property. It is here where the investigation of the methodology to obtain an
estimate of the solution provides a framework to using the machine learning
techniques to aid in discovery of material properties and how these same techniques can be used in the understanding of properties. As the periodic table
is the standard for chemistry, data science is a fundamental tool in computing
the correlation between a number of complex variables in high dimensional
space. This can be used to identify unknown patterns that fill gaps in our understand of materials science. These tools not only process vast amount of data
in seconds, but also can produce visualizations of the correlated data reduced
to transparent dimensional space, creating opportunities to study metallurgical
systems without having the laboratory scaled experiments.
1.1
Research Question
How can material informatics aid the development and understanding of materials?
1
Chapter 2
Background
2.1
Materials Informatics
There is a demand for materials design where the bottleneck in manufacturing
is the development of a new materials. Materials informatics offers an approach to revolutionize this design process by storing and using a vast amount
of information about the processing, microstructure, properties, and performance features. The time to develop new materials can be decreased from
20 years to 10 [2]. This puts a new spotlight on the possibilities of engineering because systems can be designed without the bounded restrictions of
conventional materials. A materials genome, not unlike the human genome,
can be a source of information that will store the sequence for new materials.
A database of this magnitude would be the entire collection of our materials
knowledge at this point in time.
2.1.1
Why is it Important? - Social, Ethical, and Environmental Considerations
A deeper understanding that the world is made of limited resources is making
way for computer aided design of new materials. This has created a common
theme as a methodology to reduce the cost and time of creating new materials.
The rapid prototype of materials have surpassed traditional methods because
a trial and error way of materials design is no longer the standard as the bulk
of the hard work can be done in computational space [3]. In the informatics of
biomaterials (European Bioinformatics Institute) and nuclear material informatics (European Organization of Nuclear Research CERN) together contain
250 petabytes of materials information [2]. The field of computational materi-
2
CHAPTER 2. BACKGROUND
als design is rapidly approaching the scale of big data analytics. A major issue
with this big data scale for materials is that it is not entirely clean, has misleading and conflicting information. A database consisting of accurately collected
length scaled data would attract highly profitable investments. It has been estimated that a new materials design economy could generate $123- $270 billion
in the United States alone [4].
A model of a metallurgical system in computational space reflects positively on the influence this particular industry has on the environment. This
provides an alternative to harmful experimental practices that involve the use
of valuable resources. With the techniques described by material informatics, more understanding of the atomic scale can be obtained by using less of
these valuable raw materials. For example, experimenting with real minerals
requires casting in high temperature furnaces, creating toxic by-products like
CO2 and other noxious gases that are released into the environment. The reasons for considering computational design methods over traditional trial and
error can be further strengthened by reflecting on the long term effects our
engineering systems have on the environment. The motivation for creating
better alloys can be used to increase efficiency in our harmful combustion gas
turbine engines. The solutions offered in this text can significantly reduce the
output of CO2 created by the aerospace industry by increasing the strength
of alloys modelled with the computational techniques [5]. It is also of great
interest to these industries to obtain a model that can better be used to predict material properties. A small increase (0.1 %) in combustion efficiency
could save billions of dollars in fuel costs [6], reducing potentially hundreds
of thousands metric tons of CO2 emitted every year. A quality, up-to-date academic source on the precise figure of the global aviation industry CO2 is not
available. Reports indicate that the emissions rate has increased 85 % from
1990 to 2004 [7]. The industries that are producing harmful effects to our
environment are forced to comply with regulatory efforts to reduce the harm
caused by their products with the aerospace industry being included European
Emissions Trading Scheme [8]. Currently the aerospace industry alone makes
up 3 % of the global emission of CO2 with this number expected to reduce
because of regulatory efforts and an increase in technology [8]. The specific
technology that could improve efficiency could come from materials design
techniques with informatics using artificial intelligence.
Artificial intelligence also comes with an added level of ethical considerations, obtaining solutions for materials science with these techniques should
be met with a level of skepticism as the solutions obtained with these architectures are not fully understood. Therefore to maintain a high level of trust
3
CHAPTER 2. BACKGROUND
in the scientific method, elegant and transparent solutions should be considered first before brute force methods such as artificial neural networks. These
particular details will be discussed later. It is the social, ethical, and environmental responsibilities all scientists should share equally in presenting work.
The following text will attempt to complete the materials informatics argument
for predicting properties with this in mind, laying out the fundamentals of the
subject while maintaining a central theme of providing an alternative framework to reduce costly experimental practices while aiding in the discovery of
new materials.
2.1.2
The Data
It is the increased computational power in personal computers that has made it
possible to optimize materials this way. The online community, Stack-Overflow,
offers a free forum that contains libraries of fixed bugs allowing a quick "google
search" to solve most coding issues. This dense online collection of computer
programmers, and user-friendly application programming interface (API’s)
are creating a multidisciplinary industry of computer scientists. Sci-Kit-Learn
[9], and TensorFlow [10] offer professional machine learning API libraries for
free, as well as Silicon Valley start-ups like Citrine informatics [11] that are
specifically in this domain of informatics for materials science. These sources
make up only a part of the growing community of data science for materials
with increased industrial, academic, and government influence contributing to
this growth.
This combination of knowledge from different industries to be used for
materials science are connecting the length scales in materials design. Computer vision has emerged as a tool to classify microstructural features predicting creep characteristics from micrographs obtained from a SEM [12].
As well as initiatives to understand particle physics, with European Organization of Nuclear Research (CERN) [13] contributing to a new length scale.
The CALPHAD approach to understanding the thermodynamics have matured
[14]. Atomistic modelling has blossomed into open source databases with the
Open Quantum Materials Database [15]. As well government initiatives like
Materials Genome Initiative facilitate the collecting and sharing of this information [4]. Computer aided discovery of materials is growing denser with
these new technologies/initiatives every year. A way of incorporating all these
length scales is needed [16]. Information learned at each of these length scales
can be stored in a database to be used to design or predict new material properties.
4
CHAPTER 2. BACKGROUND
2.1.3
Machine Learning Tool Box
Materials science data has been documented since the advent of writing 12001500 years B.C see figure 2.1 [17]. A single author can produce a series of
papers with reliable experimental documentation and data. Academic journals
are the source of experimental data that will be used in the machine learning
algorithms.
Figure 2.1: Egyptian hieroglyphic showing metal working instructions for bronze
doors from Nicholson, Shaw, et al. [17]. The written, peer reviewed, published scientific work would be a source of data to be used for predicting experimental material
properties.
This assessment of machine learning is following popular trends with treebased algorithms and artificial neural networks (ANN). Recent advancements
in graphical processing units (GPU) and advanced microprocessor architectures have allowed neural networks to outperform tree-based models in image
recognition competitions [18]. Materials science has not been as densely populated with information, yet. Experimental data sets can be considered large
with two thousand or more points. However, this is changing with calculationbased API software’s like Thermo-Calc’s TC-Python [19]. This has essentially
removed the barrier to large data sets where millions of data points are now
possible because of a data-driven approach to materials science being captured
with this platform. This completely changes the approach to materials design
with respect to algorithm choice and methodology. Tree-based models used
to outperform ANN because of the use of low variance high bias predictors,
an analogy to the wisdom of crowds. The artificial neural network is better
suited to metamodelling, as it automates the learning. This automation is the
feature selection [18]. More on this will be discussed later.
2.1.4
Machine Learning Methodology
There are two approaches demonstrated in this assessment of the materials
science machine learning methodology, metamodelling and experimental data
5
CHAPTER 2. BACKGROUND
modelling. The difference in these methodologies lies in how the data is collected. Metamodelling is precisely used for the benefits of computational
speed, by sampling and calculating with software or simulations (CALPHAD,
precipitation models, diffusion simulations, density functional theory (DFT),
finite element analysis (FEA), etc..) [20]. Experimental databases with uses
of machine learning are different: Referred by Butler et al. [21] as an older
generation of AI materials properties prediction. Experimental data is more
prone to human error. Thermo-Calc’s simulation software are built off of
physical based models describing Gibbs energy. In this methodology using
machine learning both experimental and metamodelling will be used to aid
in the prediction of both Ni-based superalloys used in the energy sector, and
steels. Methods in this assessment using data retrieval, cleaning, and organizing into a transparent understanding, will influence the computational speed
and contribute to more understanding of the physics based simulations. This
methodology will also cover fundamental materials theory and background
where evaluating the physical limits of materials are the first source of predictive power. Therefore, to show that machine learning is never a starting point
to materials discovery. These powerful tools and mature industrial software
are creating new opportunities with modelling of material behaviour.
2.1.5
Connecting the Length Scales
The length scales describe the different functional scales of materials design:
Elementary particle, atomic, nano, micro, macro where the goal is to develop
a model that is continuum, or able to connect all these length scales to a final
engineered product. However today, each of these steps are usually a discrete
form of design, meaning a single model is only valid for one working scale.
As one travels through the different length scales it is necessary to provide assumptions to apply what is learned to another scale. Machine learning offers
a way to use information at different length scales simultaneously. Matminer
feature generation connects large databases together to be used for such a goal
[22]. It is here where the immediate impact of the materials genome initiative and its associated databases can be used to affect computational materials
design. The composition of an alloy can be used to generate new descriptive
features. Some of these, to name a few: magnetic moment, electron vacancy
amount, fraction of transition metals, as well as features from crystallographic
structure, and density of states. This can have significant benefits to the machine learning model, connecting this information to predict material properties.
6
CHAPTER 2. BACKGROUND
The materials informatics argument has been laid out with the various tools
for collecting data, and the methodologies with their uses for materials design. A demonstration of these tools has been selected for two problems, a
metamodel for Ni-based super alloy predicting sigma phase amount. And an
experimental model to predict martensite start temperature in steels. An explanation of the importance of these two systems to aid the argument of materials
informatics for uses in research and industry will now be presented.
2.2
Ni-Based Alloys
The energy industry has benefited greatly by the advent of the Ni based alloy. The heat resistant material can withstand operating temperatures near
its melting point while maintaining its mechanical properties. The almost
perfect lattice misfit between the gamma and gamma prime phase creates a
microstructure that is balanced between toughness and strength [23]. Their
function is always in the most vital areas of the combustion chamber. The
location of an inefficient process responsible for a low net-work output in turbines. A small improvement in the Ni based alloy in this area would prove to
be massive in terms of saving resources and reducing emissions [24]. Other
applications of these alloys can be found in our energy production plants, providing the same functional role. Materials design has allowed the properties
to be tuned to fit the specific requirements of these industries. This one alloy
system has sparked research in development in many sub-fields of materials
science allowing academic research to branch into industry. Aluminium and
large castings dominated the field of aerospace 40 years ago, today it is composites and additive manufacturing representing an affinity to innovation [5].
2.2.1
Metamodel for Sigma Phase Amount
Computational materials design offers a wider range of possibilities for high
value, large scale industries. Rolls-Royce spends 1.1£ billion a year on research and development of their widebody engines [25]. With the vast amounts
of data published every year, data science techniques are now an interdisciplinary standard being used to augment the cost of research and development
[3].
New alloys are a complex system of give and take due to the vast composition space. To search this entire space is an impossible cognitive task. For
Ni based alloys there exists a fine balance between multiple properties such
as density, solid solutioning strength, oxidation resistance, creep resistance,
7
CHAPTER 2. BACKGROUND
and lattice misfit that all need to be optimised together. This is because optimising one property may in fact lead to detrimental properties in another
[26]. This optimisation problem can be handled with modern computation
techniques that can search an endless continuous composition space for the
optimum composition. It is here that better predictions of the space can be
done with the use of artificial intelligence models, reducing the need for having an experimental scaled model.
The sigma phase is a detrimental phase that forms in the operating temperature of most turbine combustion chambers. Dislocation, coble, and Nabarro
creep are the main areas of research to improve the operating life of Ni based
alloys [27]. To increase the dislocation glide resistance, rare earth elements
such as Re are added. An element known to control the precipitation rate of
the gamma prime phase, retarding the growth and limiting sigma phase formation [28]. The high order system is commonly designed in conjunction with
CALPHAD type calculations [14]. Here the focus of the computational materials design argument takes shape, allowing further freedom in composition
selection by analysing potentially millions of combinations of alloys in a single
model. This is not practical to do experimentally.
To produce data for the search of this computational space is limited by
computational power. CALPHAD calculations for this complex system are
very time consuming. Therefore, a metamodel will be used to train an artificial neural network to predict sigma phase fraction, demonstrating the speed,
accuracy, and flexibility of a machine learning model. This model will showcase how the same data can be obtained with comparable accuracy’s to their
computationally heavy, physics-based calculation counter-parts that are thousands of times quicker.
2.3
Martensite in Fe-Based Alloys
Martensite is highly dependent on composition and cooling rate, characterized
by a BCT microstructure that contains high levels of hardness due to interstitial strengthening elements that act as a barrier to dislocation. Morozov et al.
[29] studied much of this transformation with regards to the effect of the binary/ternary Fe-X alloys. In this evaluation the authors’ developed a unique
hydraulic valve used to control the cooling rate at a very precise level ± 40
K/s in terms of accuracy [30]. The cooling rate control was fundamental in
the assessment of the three morphologies martensite can exist in: plate, lathe,
and epsilon, see figure 2.2. This collection of academic papers by Mirzayev
and his colleagues would be constantly referenced with regards to martensite
8
CHAPTER 2. BACKGROUND
through to present publishing and would prove to be a major source of reliable
data. This methodology of understanding the experimental conditions, are key
to selecting reliable experimental data.
Figure 2.2: Cooling nozzle used in Morozov et al. [29] experiments to determine
martensite start temperature. In their studies they labelled these morphologies as
plateaus where over a min/max cooling threshold one could obtain a martensite morphology. In their assessment this cooling nozzle could accurately control cooling rate
± 40 K/s
2.3.1
Experimental Model for Martensite
The current thermodynamic model is based off the work of Stormvinter et al.
[31] where the martensite start temperature is modelled as the energy at which
martensite structure (BCT) has the same Gibbs energy as austenite (FCC). This
is the Ms temperature when the thermodynamic barrier T0 is 0; increasing the
barrier pushes the start temperature down. This model relies on a thermodynamic description of FCC, BCC, and HCP phases. Stormvinter’s model was
based off the work of Borgenstam and Hillert [32] for calculating the driving
force for binaries described by the rapid cooling rate data provided by Morozov
et al. [29] and a series of papers by Mirzayev et al. [33]. A clear thermodynamic description of the martensite start temperature was investigated using
all the available data to completely describe the barrier with a single mathematical expression (see equation 2.1 regarding lathe martensite). Hence, an
equation that will be able to generalize the problem to multicomposition alloys.
The assessment is referred to as semi-empirical fitting to experimental data.
The bulk of the model is largely due to a good description of unary, binary,
9
CHAPTER 2. BACKGROUND
and ternary interactions in the thermodynamic database TCFE10 [34]. These
higher order terms in the description of the barrier (combination of non-ideal
Gibbs energy contributions of important elements ex. Fe-C-Cr, Fe-Mo-Ni interactions) are modelling the deviation from any ideal behavior. All of the
combinations are possible to model, but need experimental data. Therefore,
only the terms with reliable data are included in the final thermodynamic description. It is precisely here that an investigation using machine learning can
be used to understand which higher order terms are useful.
Equation 2.1 represents the current model for calculating the thermodynamic barrier for lathe martensite ∆G∗γ→α
m(III) (plateau III) developed by Stormvinter et al. [31]. Xm represents the mole-fraction of an element, and Ms the
theoretical start temperature. From Stormvinter, to be able to use this equation
to predict Ms temperature: "it is necessary to find the temperature where the
available driving force for the alloy composition equals the required driving
∗γ→α
force, i.e., the transformation barrier [31]." In other words, when ∆Gm(III)
=
0
∆G∗γ→α
m(III) =3640 − 2.92Ms + 346400
16430xC − 785.5xCr +
x2C
−
(1 − xC )
(2.1)
7119xMn − 4306xNi +
xCr
350600xC
(J/mol)
1 − xC
The present work is to demonstrate and build off of the published results of
Rahaman et al. [35]. Tree based models were trained from experimental data
on martensite start temperature. Rahaman’s methodology approach cleaned
the dataset of outliers with materials science fundamentals; a fully austenitic
region. The same approach was used in this assessment aided by the Ms model
from Thermo-Calc software [36], a solubility limit rejecting alloy compositions above a threshold of carbide and nitride forming phases in the austenitic
region, and only including stable martensite morphologies in the model. This
presentation will also demonstrate the same powerful use of ensemble methods with accurate predictions on a dataset similar in size to the one used in Rahaman et al. [35]. Much of the data was referenced from the same sources, as
well as previously collected data from [37], and [32]. The data-set composed
of composition (weight/mass fraction percent), and martensite start temperature. The techniques added to this assessment included data mining for extra
features and to build on the result of Rahaman et al. [35].
10
Chapter 3
Martensite
3.1
Methodology
The work for an experimental database of martensite start temperature is based
off of the collection of data from a previous works of Hanumantharaju and Kumar [37]. The following text will include a machine learning model using the
same data to predict martensite start temperature as a function of composition.
The methodology, results, and discussion are presented.
3.1.1
Gathering the Data
The martensite data was collected in a so-called .exp file format by Hanumantharaju and Kumar [37] containing binary, ternary, and multicomposition
alloys. Such a format is used to plot experimental data in Thermo-Calc’s software. For machine learning, the data had to be stored in an Excel spreadsheet
with .csv file format. With each experimental file of collected data, the reference to that data was given and an assessment of the data was done by comparing it to the binary and ternary diagrams produced by Hanumantharaju and
Kumar [37]. This would also be an opportunity to review the experimental
practices in the references that were found. It should be mentioned that not
all sources could be located from Hanumantharaju and Kumar [37] as some
of these sources dated back to the 1930’s and some of the sources were only
available through journal databases by print that were not accessible through
KTH library.
11
CHAPTER 3. MARTENSITE
3.1.2
Recognising the Machine Learning Problem
The Machine learning problem can be defined as: a supervised learning approach to regression with a sparse dataset. Each alloy contained, in its data set,
the composition (inputs) and its corresponding start temperature (the target).
By analyzing the data as a whole, there are certain areas of the composition
space that are highly populated with points while other sections would have
no points in them. This is considered a sparse data set and is a problem for
machine learning that learns best by examples of information in the entire prediction space. The algorithms of choice will be tree-based structures using
ensemble algorithms to approach the small sparse dataset.
3.2
3.2.1
Ensemble Methods
Entropy
Decision trees select features at nodes by calculating the entropy equation 3.1.
This technique of selecting the best split recursively is the feature selection
method for tree based algorithms. This allows the entire structure of the tree
to be built automatically where each split is the best possible split; the greedy
approach. pi is the probability of a single outcome [38].
X
Entropy =
−pi log2 pi
(3.1)
i
Each split can be determined best by how much information that particular
node yields or a question that splits the greatest number of data points, given
by equation 3.2. S represents the entire data set, A is an attribute of that data
set and SV is a subset of data according to the value of A. In simple language:
information gain is represented by the entropy of the entire data set minus the
entropy of one attribute. The attribute with the lowest entropy gives the highest
information gain [38].
Inf ormation Gain = Ent(S) −
| {z }
bef ore
|SV |
Ent(S)
|S| | {z }
v∈V alues(A)
|
{z
} af ter
X
weighted sum
12
(3.2)
CHAPTER 3. MARTENSITE
3.2.2
Random Forest
Random forest is built on the assumption that “crowds have wisdom." This is
a collection of knowledge from high variance low bias learners that are combined to produce a collection of trees that are more powerful than any single
tree-based machine learning model. This technique’s foundation is regarded
for its ability to train many small trees in parallel. The result is obtained by
averaging the predictions of all parallel trees (bagging). This combination produces a predictor that is robust, lowers the variance (closer to the actual value),
and reduces overfitting. The feature selection works as described with information gain but the random in random forests contains two important properties; feature selection at each node and the way which the data is sampled
[39].
3.2.3
Adaboost
This algorithm keeps with the ensemble theme by making use of weak learners and random sampling/feature selection. This was developed by Freund
and Shapire [40] in 97 and is highly regarded solution for small data sets. This
algorithm is similar to bagging method used in random forests, except for its
use of an indicator function which is responsible for adding weight to misclassified points, reducing weight if it’s correct. The weighted samples that
are misclassified will in turn have a larger penalty. Hence, more attention will
spent on classifying these points correctly in the next iteration (boosting). The
trees are built in series, recursively, based off the prediction of the previously
generated tree. This is distinctly different from the random forest algorithm
that builds many trees in parallel. At the end of a set amount iterations what
is left a tree-based structure that has a collection of the trees that minimized
the error [39]. It can be summarized as follows in 4 steps.
4 Step Process
Step 1
Train weak classifier using the data and weights, selecting the one that minimizes the training error. is the error , xj is the input(s), yj is the target, wj
is the weight, t is the iteration number, α is reliability coefficient and ht is the
predicted target value.
=
m
X
(t)
wj Ind(yj 6= ht (xj ))
j=1
13
(3.3)
CHAPTER 3. MARTENSITE
Step 2
Compute the reliability coefficient
1 − t
)
t
t must be less than 0.5 and break loop if t ≈0.5
αt = loge (
(3.4)
(t)
(3.5)
Step 3
Update the weights
(t+1)
wj
= wj exp(−αt yj ht (xj ))
Step 4
Normalize the weights so that they sum to 1
3.3
3.3.1
Results
Initial benchmark of Uncleaned Dataset
3600 rows of data (points) had been collected with 18 inputs of composition
and a target martensite start temperature. The algorithms to be chosen for
this benchmark from Sci-Kit Learn [9]: linear regression, random forest and
adaboost. The initial benchmark from the uncleaned dataset is ± 58 K, with
random forest having the lowest root mean squared error of 54.39 K, see figure
3.1. This would give a starting point to the evaluation. The next sections will
describe the process of cleaning the data investigated by using both literature
and experimental practices with Thermo-Calc’s TC-Python modules: martensite, property diagrams, and batch calculations of the thermodynamic barrier
[19].
3.3.2
Cleaning the Dataset
The motivation for cleaning a dataset is to provide the algorithm with the best
possible examples of martensite start temperature as a function of composition. To do this would be aided by software, calculating all the points of
dataset with the TC-Python’s martensite module developed by Thermo-Calc
(currently still in development) using the TCFE10 database [34] [36]. This
would act as a comparison to identify outliers in the data by also using the
thermodynamic barrier as seen in the bottom of figure 3.2. These points are all
labelled by their respective morphology: red-plate, blue-lathe, green-epsilon
14
CHAPTER 3. MARTENSITE
(a) Linear regression
(b) Random forest
(c) Adaboost
Figure 3.1: Initial benchmark of uncleaned data set with an average rmse of ± 58
K between the three algorithms (a) linear regression, b) random forest, c) adaboost)
using Sci-Kit Learn with the algorithm with the lowest rmse value obtained with
15 the root mean squared error.
random forest. The red dotted lines represent
CHAPTER 3. MARTENSITE
with their lines representing the theoretical start temperature. The black points
are the labels with no recorded morphology from the source. By cross referencing the sources given by these points unique id and the assistance of this
visualization, further assumptions could be made about the morphology of
these unclassified points. The methodology to lay out the cleaning would be
further strengthened by: property models calculated from the same data, targeting austenite stability and a threshold of carbide/nitride forming elements
and finally, a dataset composed only of stable martensite morphology’s as a
function of composition and temperature. These steps in the methodology of
applying experimental data to machine learning would attempt to asses the
quality of every point. Their specific details are discussed in the following
paragraphs.
16
CHAPTER 3. MARTENSITE
Figure 3.2: An example of a nickel binary section calculated with TC-Python’s
martensite module [36]. The theoretical Ms start temperature for each morphology
is displayed with the labeled experimental data overlayed on top. The uncleaned data
set is presented here with the top section represents martensite start (Ms) temperature
[K] as a function of composition and the bottom calculated barrier [Gibbs energy] as
a function of composition Ni.
17
CHAPTER 3. MARTENSITE
Figure 3.3: Representing the same nickel binary section as figure 3.2 but with a
cleaned dataset. The alloys that are in this binary section of nickel are ones that have
completely stable austenization range and exist above the thermodynamic barrier T0 .
Notice the collection of points that fail to meet the outlined criteria towards the tail
end of the composition space. These were purposely left here even though they did
not meet all the criteria. From the experimentally collected data there is a clear trend
here that the thermodynamic model is not able to capture well.
18
CHAPTER 3. MARTENSITE
The significance of these plots would provide visuals of the trends in the
data and more importantly, identify the sections of the thermodynamic model
that were not being captured well. To identify these trends would require some
intuitive assumptions to be made. For example, in the Ni binary figure 3.3 at
higher concentrations, there was a clear trend in the data that was not being
captured, indicating sharp decrease due to a magnetic transition after the Néel
temperature beyond 17 at % [37]. These areas that were not being captured
well would be maintained with points populated in this area, especially if they
were not being correctly modelled. The goal here to keep examples in this
uncertain area to help capture this trend in the machine learning model. The
question arise during cleaning: should we keep or discard this data? Is it
displaying outlier behavior? Or is the thermodynamic model not capturing
the true behavior? Such is the problem when working with experimental data
as not all of these questions had answers.
The theoretical barrier added to the bottom of the plot would provide an additional screening method to visualize outliers. This barrier would help show
clear outlier behavior, displaying points that were not meeting this fundamental limit. Any point that did not have a greater Gibbs energy then the barrier
was removed classified as "clear outlier behaviour."
Another assumption was comprised of recognizing the points that had reliable austenite stability. By introducing a step calculation (800-2200 K) to
a property model defined in T-C Python [41], property diagrams for each of
the 3600 alloys could be calculated. This would maximize the confidence in
the experimental data, providing alloy compositions that would have the highest probability of forming one of the three martensite morphologies because
martensitic transformation is a γ →α or transformation. The workload of
this was carried out via external desktop to handle the calculations. 99.9%
austenite phase and a range of at least 100 degrees was required in order infer
that the alloy composition had a reliable austenite range [35]. Unstable alloys
that contained a large amount of carbides were removed figure 3.4a and 3.4b.
19
CHAPTER 3. MARTENSITE
(a) Accepted Alloy
(b) Rejected Alloy
Figure 3.4: An example of the acceptance criteria laid out by calculation of a property
diagram using TC-Python’s property model module [41]. This alloy had a fully stable
austenite region and at least 100K of 99.9 % FCC Al labelled in green. A rejected
alloy that contained no fully austenetic region due to nitrides and carbides formed.
Thermo-Calc’s martensite model is a single thermodynamic physics based
calculation that covers three morphologies. The software predicts a morphology based on the nearest stability. This stability is the morphology with the
highest start temperature. It can be visualized in figure 3.5. Each morphology
has its unique behavior features, depending on which area in the composition
space the alloy exists will contain a different stable morphology. From the
work of Mirzayev et al. [33] clear data was produced for these three plateaus.
However, the entire range of the composition space cannot be accurately covered with the current thermodynamic model. 15-25% range in Ni binary plot
figure 3.2 needs a better description of the deviation from ideal behavior. To
attempt to capture this, a stable morphology column was added to the dataset
that would only contain points with the highest start temperature of the three
available (lathe, plate, epsilon). This would vary for different sections of binary and ternary as the morphology transitions normally from lathe to plate to
epsilon (roughly in that order) as one increases the alloy concentration.
20
CHAPTER 3. MARTENSITE
Figure 3.5: Stable morphology of an iron-nickel section computed with TC-Python
martensite module [36]. A stable morphology is represented by the highest martensite
start temperature (top) and the lowest energy barrier (bottom). Notice how the plate
martensite points (that would be labelled in red) have now been removed. These
would represent metastable morphology, where under high enough cooling rates this
morphology could be obtained. As the Ni content increases the stable morphology
switches from a mixture of plate/lathe to plate, so there will be red points in the 20-30
mole-percent region of this section.
21
CHAPTER 3. MARTENSITE
The collection at the end of this assessment contained 2863 unique clean
alloys composed of binary, ternary, and multicomposition (cleaned metastable
points were kept to include in feature generation and selection section). The final dataset used to train and test with ensemble M-L methods contained 2015
points. Rejected alloys contained: metastable morphologies, no stable FCC
range, duplicates, and clear outlier behavior. The exercise was to predict stable
martensite start temperature to try and capture non ideal behaviour in higher
composition ranges where the thermodynamic model was not able to do precisely do this. Figure 3.6, a plot of all sections theoretical vs calculated barrier
reiterates this subsection of text, showing how the same methodology applied
to all 3600 rows of collected alloys could be used to clean an experimental data
set to be used in a machine learning model. The assessment so far has shown
that the majority of effort to build a model with experimental data is used to
clean the data of outlier behaviour. The next section of feature generation and
selection leads to new information gained that can contribute to a more accurate thermodynamic model of martensite. This is done by creating meta-data
generated from the current set of experimental data to aid in the understanding
of higher order interactions, or the deviation from non-ideal behaviour.
22
CHAPTER 3. MARTENSITE
Figure 3.6: A section of the entire cleaned data set, visualized by plotting the experimental martensite barrier to TC-Python’s martensite model [36]. The corresponding
morphology is represented by the color. This plot would consist of all the binary,
ternary, and multi-composition cleaned stable sections. These would all be filtered
out the same way for each section individually, by assessing visually how the plots
compared to Thermo-Calc’s martensite module, and rejecting alloys with no stable
austentic region from property diagrams and finally removing metastable points that
occurred below the highest start temperature.
23
CHAPTER 3. MARTENSITE
3.3.3
Feature Importance
The automatic generation of trees by the greedy approach naturally creates
a list of important features. These can readily be called from Sci-Kit Learn
tree models as feature importance where a score of information ranks the individual features by their position in the tree. The results are based on the
algorithm of choice and the quality of the data. The results in figure 3.7 are a
benchmark of the feature importance’s. This is the effect the pure interactions
have on the martensite start temperature ranked from most significant to least.
This data does not provide any new information, as these interactions are already described well in the model. Rather, this was used as a starting point to
investigate non-ideal behaviour with tree based models.
(b) Random Forest
(a) Adaboost
Figure 3.7: Feature importance’s produced by decision trees. These are produced
naturally, where the most important decision will represent the first split in the tree.
This is a piece of information that could be useful to future works. The feature generation and selection added to the tree based algorithms would be able to produce
these plots with much more information by adding the generated features to the input
dataset for training. Here a demonstration of the unary elements and their importance
to the Ms start temperature.
To try and capture the deviation from non-ideal behavior in the martensite
model it is important to understand the relationship between how each individual composition effects the martensite start temperature as this provides a
reference to known information about the pure interactions. More advanced
techniques were used with feature generation to build a set of new features.
Feature generation is a technique to add extra composition columns; i.e, a way
of providing more data to the model, by generating it from data already col-
24
CHAPTER 3. MARTENSITE
lected. This is done before machine learning takes place.
3.3.4
Feature Selection and Generation
More complexity can be added to the features by generating higher order interaction terms. This was performed using polynomial feature generation from
Sci-Kit-Learn [9], combined with a feature importance selector; K-best selector with mutual information regression. These new pieces of information
would be ranked in order of “how significant does one feature have in knowing
the martensite start temperature.” This algorithm would score parameters significance to the target. Mutual information selection was designed for sparse
data as it makes clever assumptions about the composition space, measuring its
dependence between variables and a target. This list of feature importance’s
would provide clues to which of the higher order terms were significant for
modelling. Hence, could provide a significant contribution to the excess Gibbs
energy terms in the thermodynamic model.
3.3.5
Cleaned Dataset Results
Knowledge about these features was performed on a clean dataset by filtering
the data by specific morphology alpha (lathe and plate combined) and epsilon.
The results of important interactions for a specific morphology can be shown
in figure 3.8a and 3.8b. It is interesting to see the difference in these terms.
It is known that increased levels of manganese contribute to the formation of
epsilon and the relationship between carbon and nickel for alpha morphology.
Again, there is nothing new is obtained from this information about the pure
and binary interactions rather, verifying that the important features are believable. This set of new features are supplying additional information about the
higher order interactions that are not possible to model. This is especially
valuable information gained as these terms that do not find their way into the
thermodynamic assessment because there is no experimental data available
to describe them. Therefore, this technique with cleaned data and generated
features could be a powerful tool for the thermodynamic simulation. This
is currently just a belief in feature generation and selection; maintaining the
integrity of the scientific method, further investigation of these features will
need to be studied closer. It cannot be said clearer than this: statistics does
not equal understanding [18]. The understanding can be done by looking to
materials science foundations for published material on why this set of feature
importance terms are this way. The best way of assessing this data currently
25
CHAPTER 3. MARTENSITE
is experimental practice, but with none available, added information from different length scales could increase the probability that this data is reliable. For
this model, feature generation and selection using added length scales will be
evaluated and discussed by using open source platforms for materials informatics. More on this later.
(a) Alpha martensite higher order interaction features
(b) Epsilon martensite higher order interaction features
Figure 3.8: The dataset was filtered based on morphology, where feature importance
using K-select with mutual information regression. The relationship between composition and Ms start temperature filtered by morphology is presented. It is important
to note here that this feature selection is performed with the cleaned data set where
the confidence of the examples fed into the model are the highest. The significance
of these features could add value to the thermodynamic model where experimental
information on the higher order interactions is not currently available.
26
CHAPTER 3. MARTENSITE
3.3.6
Comparison to the Thermodynamic Model
Hyperparameter optimization was carried out using Sci-Kit Learn’s grid search.
This performs 5-fold cross validation for every combination of set hyperparameters, see table 3.1. In simple language, this adjusts the settings of the
machine learning algorithm to best fit the model to the experimental data, preventing overfitting that contains a balance between complexity and accuracy.
The details of the hyperparameters and the tuning of them for a specific dataset
would require a lengthy discussion. For this simple assessment, using a baseline of tree-based algorithms, further investigation of these tuning parameters
could be leveraged for higher prediction accuracy with more complex architectures. To move the conversation of the materials informatics methodology
forward, the hyperparmeter algorithm developed with grid search would take
approximately 120 minutes with Thermo-Calc’s multicore desktop to produce
the current model. The specific details are omitted. When the best model was
obtained, it is saved as a pickle file format. This would allow the model to be
loaded and a theoretical Fe-X binary data set was tested. This would consist of
running this loaded model on a theoretical data set that only contained binary
data for the section investigated. Its’ results, in comparison to Thermo Calc’s
martensite model are presented in figure 3.9. Note that this model would be
the same model used to predict all binary behaviour and its results show accuracies ± 50K with adaboost and random forest having very similar results. It
is also interesting to see how these points do not follow the stable morphology
line, but predict somewhere in the middle. The data surely contained points
for only this section with very specific morphologies, but this model is a collection of all data, including multi-composition alloys.
Table 3.1: Hyperparameters used to obtain the best train model. This process was
done with Sci-Kit’s grid search and the spefics of these hyperarmeters can be found
in their documentation [9] for hyperparameter optimisation of random forest.
Hyperparameter
ccp alpha (post-pruning)
max depth
min samples leaf
min weight fraction leaf
min impurity decrease
number estimators
Value
0
20
2
0.0
0.75
1000
27
CHAPTER 3. MARTENSITE
(c) Fe-Ni
(d) Fe-Mn
Figure 3.9: Binary predictions comparing adaboost and random forest predictions
to Thermo-Calc Ms model [36]. The experimental data is also plotted. The data
was generated with a uniform variation of composition labeled on the x axis in the
sections. The series of vertical and horizontal lines are indicative of the tree-based
structures the algorithms were built on. The target performance of these algorithms
were to capture non-ideal behaviour in sections of the plots that the thermodynamic
model was lacking. Their results are presented with random forest (purple), and adaboost (yellow) where a single model produced with the complete dataset of binary,
ternary, and multicomposition alloys.
28
CHAPTER 3. MARTENSITE
The plots produced from models are series of vertical and horizontal lines.
This behavior is expected due to the tree based structure. The prediction looks
this way because a tree based structure is built with an adjacency matrix that
is just a boolean matrix of 0 and 1’s. The decision tree is showing some comparison with the theoretical martensite model for binary elements demonstrating the robust nature of the prediction which is a collection of many binary,
ternary, and multicomposition alloys.
3.3.7
Other Machine Learning Platforms
Mast-Ml and Citrine informatics are cloud based API software specifically
built for materials informatics [42] [11]. They can either be used with a python
API, or through an online graphical user interface. Citrine informatics is private and Mast-Ml is an open source platform. The Citrine informatics platform
offers free limited use to students. This platform automates the selection of
the model, the hyperparameters, feature generation and selection. This feature
generation and selection is a powerful tool for creating new features based off
the experimental data paired with dense libraries of data collected at different
length scales, see figure 3.10. To do this the user needs to add a composition
column with the element and its fraction of composition in either mole fraction/percent or weigh fraction/percent for each alloy (Ex: Fe0.90C0.01Ni0.09
in mole %). This includes chemical, first principle, and crystallography data.
Citrine informatics platforms contains their own cloud based collection of
length scaled data and Mast-Ml, automates the generation of features from
magpie feature generation that is similar in content [43]. For experimental
modelling of properties, this experience in machine learning is very powerful.
The best result rmse values were obtained using these software ± 29 & 33
rmse figures 3.11 and 3.12.
29
CHAPTER 3. MARTENSITE
Figure 3.10: The top feature importance represented in the Citrine platform by the
significance in percent [11]. One can see the added features based on the label "...
for formula" these are generated features based off the individual point’s composition. These features are added from a database of stored information from DFT
calculations, crystallography data (XRD) and information specific for each element
(molecular weight, s, p, f shells etc..) inside the cloud based platform. It is showing
the features’ correlation to predicting martensite start temperature as a function of
molecular weight, composition (C, Mn, Ni, N), magnetic moment calculated from
the orbital spins (electron structure), and melting temperature to name the top eight
features.
30
CHAPTER 3. MARTENSITE
Figure 3.11: The results from Citrine informatics [11]. This assessment was performed with their online cloud based computational platform. An extra column labelled composition was added with element and mole fraction in composition. It is
interesting to observe from these plots, that compared to the Sci-Kit Learn model they
show little outlier behaviour. These platforms, internally, clean datasets from outlier
behaviour. The results shown here have an rmse of ± 33 K.
31
CHAPTER 3. MARTENSITE
Figure 3.12: The results using Mast-Ml open source materials informatics platform
with a random forest regressor. This is the best rmse value of ± 29 K,
3.3.8
Discussion
These platforms share techniques to connect length scales to experimental data
with generated features and professionally developed algorithms that combine
to predict the Ms temperature 15-20K better rmse then the Sci-Kit ensemble models. This is surely do to a combination of the techniques outlined
in this methodology because data ingesting/parsing, cleaning, feature generation/selection, algorithm choice, hyperparameter selection is done automatically. The platforms are performing better, but without knowing what is happening in the background it is difficult to understand how each parameter is
affecting the model. Therefore, it is suggested that this increase predictive
power could be a combination of the many factors that are automated. The way
Citrine informatics’ platform collects, stores, and uses data to generate new
columns is an example of how the length scales can be connected to provide
understanding to material properties. Perhaps, this information is pointing to
valuable clues needed to make new discoveries in materials. Machine learning is a method for pattern recognition. It is a brute force method leveraging
powerful computation to sift through data. This tool has been used to produce
an interesting combination of features pertaining to the martensite start temperature that can be seen in 3.10. Investigation into connecting these length
scales is still needed and it should be met with some guarded questions. It is
not enough to put the trust into this set of features. Here, there is no presenting
32
CHAPTER 3. MARTENSITE
text on the proving that these features are significant for this particular material
property investigated. There is only intuition at this point in the assessment of
this system.
The best literature sources from Morozov et al. [29] with a unique experimental method of cooling, could only experimentally determine the cooling
rate ± 40 K/s. These plateaus of martensite are independent of cooling rate,
to a certain point, and the experimental data is flawed, to a certain level, based
on the confidence of experimental technology available at the time of publication. If this is the highest reported accuracy, what can be said about the rest
of the experimental data used for this assessment? How significant are these
results? And what is preventing the description of the Ms temperature to be
exactly predicted?
To answer these questions, this methodology of feature generation could
be used when increased levels of focus can be placed on the results of the
machine learned models. As the experimental machine learning methodology
increases in complexity, more ways of analyzing the results are necessary. Experimental data can be misleading due to random and systematic error, but is
currently still the best way of proving theoretical results. However, with tools
like Thermo-Calc, it has been demonstrated how to clean this data without potentially harmful experimental practices. The original experiment started with
3600 alloys. Many were removed due to increasing assumptions in an attempt
to produce the highest quality dataset. The majority of the methodology was
laying out techniques to do this. What If data could be generated, i.e. metadata? Where materials scientist had more freedom in composition choice and
access to highly accurate data at a fraction of the time? This is the motivation
for the next section, metamodelling.
33
Chapter 4
Ni based Alloys
4.1
Metamodelling
A metamodel has significant advantages when it comes to computational speed
as adding more complexity scales linearly in terms of computational power.
The simulations they are attempting to predict do not scale this way. Example,
moving a 2D asymmetric model to 3D in a fluid dynamics simulation. A metamodel is just a model of a model. Another abstraction that is specifically built
for savings of computation power. This technique is solving optimization problems, using data generated by optimization solutions with accuracy’s 98-99 %
as good as their computationally expensive simulations. An artificial neural
network (ANN) can be used for this purpose of metamodelling. The machine
learning algorithm is fed solutions from the simulation that filters and outputs
the solution. This scales to multioutput solutions, creating an architecture that
can predict multiple properties with one model. This methodology could have
significant impact on materials design, by cutting the computational time by
multiple factors.
Nyshadham et al. [44] created a multioutput ANN metamodel for DFT calculations predicting formation enthalpies of 10 different binary elements, each
with 3 crystal structures, fcc, bcc, hcp, with an error less than 1mev. All of
this with the use of 1 ANN. DFT calculations must be run on multicore processors for calculation with multiple atoms other than a unit cell as an increase
in the number of atoms in the unit cell, the amount of possible structures can
be thousands figure 4.1. This limits the amount of atoms that can be simulated as realistic models contain large sets of individual atoms. An alternative
metamodel for representative elementary volume simulations (REV, or multilevel finite element analysis) are known to run 120 hours for a single predictor.
34
CHAPTER 4. NI BASED ALLOYS
A Metamodel of this system performed by Lu et al. [45] to predict electrical
conductivity properties in graphene/polymer nanocomposites was able to cut
the computation time for the same calculation to 267 seconds with an ANN.
This model was only 1 % off the simulation that was over a thousand times
quicker.
Figure 4.1: DFT structure data showing the correlation between the amount of atoms
in a unit cell and the possible structures it contains.
4.2
4.2.1
Artificial Neural Network
Structure
The main structure of the ANN is its three fundamental layers, input, hidden, and output that act as filters. These filters capture representations as data
and store it in nodes. This is called the forward propagation step. Each layer
stores its representation in terms of bias (node) and weight (synapse). The
learning happens due to a minimization of the error. This minimization is
done through backpropagation, a statistical approach to function approximation which measures the difference between the prediction and the true values
[46]. The weights adjust to provide the layer to the right of it a 0 error by
calculating the amount to move the weight and in what direction. The error
rarely ever reaches 0 rather, it is a functional approximation similar to that of
a Fourier transform or a Taylor series used to approximate a solution. The
difference the artificial neural network has over these approximation methods
is it calculates the error implicitly, meaning examples are used as a comparison to the true value that interpolates the weights/neurons, bias/layers [46].
The backpropagation and the adjustment of weights happens in series, for each
layer, until it ends at the input. Then this is done over and over called epochs
where 1 epoch would mean 1 forward and 1 back propagation step [47]. The
result (a prediction) of a neural network is therefore just a summation of the
layers with the bias multiplied by the weights.
35
CHAPTER 4. NI BASED ALLOYS
4.2.2
Features
The features used to generate representations are given by loss, metrics, and
optimizer functions. The evaluation of the loss categorized by a metric, is done
with an optimizer. This optimizer has significant impacts on the learning as it
is the function that locates the minimum of the error. In neural networks this
is done with gradient decent, visualized by a bowl, where the optimum lies at
the lowest point. Gradient decent takes steps back and forth in order to arrive
at this minimum (the learning rate). In non-linear solutions, this is not a trivial
task as there can be multiple local minimum. Therefore, careful consideration
is gone into selecting and calibrating the right optimizer function [47].
It is confusing when authors speak about neural networks and their ability
to automate feature selection. This separates it from other machine learning
algorithms that use discrete feature selection. For example: kernels as in support vector machines, information gain from ensemble methods, and distance
measured between points as in linear regression. However, in the neural network architecture these features are not defined this way. A representation
mentioned earlier is saying what it is without identifying it precisely, an abstract feature. This entire concept is vague and needs more research, outside
the scope of this demonstration. Therefore the reader is referred to chapter
1.1.3 "Learning Representations from Data" in Chollet [18], for a simplified
way of looking at the problem, and chapter 10 "The Complexity of Learning" from Rojas [46]. Specifically, "Kolmogorov’s theorem", which draws the
author to conclude that the artificial neural network problem of function approximation is considered NP-complete, a term from computational complex
theory that states in simple language: no algebraic polynomial can be used
to precisely measure the computation time needed to identify a solution. The
NP-complete proposal for this solution to artificial neural networks is ad-hoc,
Rojas [46] claiming there may not exist an algebraic solution to solving polynomials of more than seven orders. The solutions using this method should be
met with very serious regard for their consequences on the integrity of materials science.
4.3
4.3.1
Metamodelling Methodology
Creating data
It was the idea to investigate a distribution that was able to generalize the formation of sigma where a single distribution would be able to perform well on
36
CHAPTER 4. NI BASED ALLOYS
Table 4.1: Composition range used to sample the original data set in mole % and
temperature in Celsius from Rettig et al. [26].
Composition
Range
Ni
balance
Al
10-15
Co
0-15
Cr
6-12
Mo
0-2
Re
0-3
Ta
0-3.5
Ti
0-4.0
W
0-3
Temperature 500-1500 [◦ C]
different data sets. A general composition space was referenced from Rettig
et al. [26] in table 4.1. Three statistical variations were sampled from this
space: uniform, multivariate normal, and latin hyper cube with the compositions spread around the mean. The latin hyper cube can be visualized in figure
4.2 where the entire space between Al and Co has some cluster of points in it.
Its properties consist of unique arrays of alloys that have no overlapping values,
meaning no two compositions are ever used. This is particularly important as
one goes to higher dimensional space where clusters of like constituents get
further apart [48].
37
CHAPTER 4. NI BASED ALLOYS
Figure 4.2: Latin hypercube distribution for Al and Co (mole %) vs frequency. This is
demonstrating the way this distribution populates the space evenly around the mean.
It looks similar to a Gaussian distribution. The latin hypercube is commonly used
with an optimisation criteria where optimizing for a property, or range of criteria
produces a dataset sampled around the ideal value. In this demonstration the latin
hypercube is sampled around the mean corners of the cube, providing a general data
set for estimating sigma phase amount in Ni based alloy.
Sampling can be described as selecting a smaller number of data points
that are used to describe the population as a whole. The aim of a distribution
is a decrease in samples that is motivated by the increase in sample quality.
A low number of points that are of a high quality would mean being able to
describe a population with minimal effort.
4.3.2
Calculating the Data
For each alloy composition, a mole fraction of sigma and mole-fraction of
composition was calculated. This was performed with Thermo-Calc’s TCPython API using a single equilibrium calculation [49] with the TCNI9 database
[50]. An Intel i9-7920X @ 2.90 Ghz, 12 core external desktop was used to
process the equilibrium and with a multithreaded calculation using all 12 cores
38
CHAPTER 4. NI BASED ALLOYS
overclocked to 3.7 GHz. A 4000 row, 10 element system was calculated. With
the use of cache folders, for reusing later, this calculation took 83 minutes. The
data was saved to an Excel spreadsheet and Pandas as an input data frame for
the build of a three-layer sequential neural network with one hidden layer using
Tensorflow/Keras with Python programming language [19] [51] [52] [10].
4.3.3
Using the Data
Figure 4.3: Feature importance measured with K best using mutual information
showing the relationship between which term contributes the most information in
predicting sigma phase amount.
Before the model is built, it is necessary to first look at the data. A clearer
picture of the sigma phase calculated by Thermo-Calc could be obtained by
using plots for visualization and a bar graph of feature importance, see figure
4.3. This provided some intuition to general trends. Figure 4.4 shows two
opposite trends found where an increase in aluminium content decreases the
amount of sigma, and the opposite effect for Cr. This plot also verifies the
relationship between formation temperature and amount as temperature is the
4th most important term. The important features are ranked by how one can
have an impact on knowing the amount of sigma phase.
39
CHAPTER 4. NI BASED ALLOYS
Figure 4.4: 3D heat plot of amount of sigma, and composition vs temperature from
equilibrium calculation. Importance is on the significance of the Al and Cr plots
showing a clear trend of how increasing the amount of Al decreases the formation of
sigma and the opposite effect for Cr.
4.4
4.4.1
Results
Benchmark Results
These parameters are stock parameters from table 4.2, trained on all data sets.
The initial parameters chosen were from TensorFlow’s library of supervised
learning regression examples. The output layer’s nodes for determining the
target was adjusted based on how many targets were to be predicted (1 & 10
targets).
These benchmarks would be used to evaluate how the models improved followed by evaluating the effect of sampling. The data sets would be increased
by a factor of 10, to verify the effect of amount of data. New composition
features were generated from Matminer [22], comparing to the benchmark.
A demonstration of multiple outputs, predicting sigma phase amount and its
40
CHAPTER 4. NI BASED ALLOYS
Table 4.2: Parameters chosen for the supervised learning regression ANN. There
specific details can be found in TensorFlow documentation [10].
Parameter
Activation function
Optimizer
Learning rate
Nodes per layer
Metrics
Layer type
Tuneable parameters
Number epochs
Test/validation split
Description
Relu
RMSprop
0.001
64
Mean squared error, mean absolute error
Dense
531457
1000
20 %
Table 4.3: Input and single target parameters used to train the ANN
Inputs[mole-fraction]
Target[mole-fraction]
[Ni, Al, Co, Cr, Mo, Re, Ta, Ti, W, temp (C)]
Amount sigma
composition. Finally the last section will be showcasing how to tune the hyperparameters of the model, comparing the best results to Thermo-Calc’s thermodynamic model. Their specific details will now be presented in the following
paragraphs.
The goal of this assessment is to generate a model that would be able to
most accurately predict the amount of sigma phase in a Ni based alloy. The
initial benchmark produced a root mean square error (rmse) of +- 3 sigma
weight percent with input/target parameters taken from table 4.3. The outliers
in this case were gathered, labelled as red points in figure 4.5. These points
were statistically evaluated to deduce where the model was not capturing these
areas well. This initial test of the neural network would give the starting point
of the evaluation and the effect of sampling will be compared.
41
CHAPTER 4. NI BASED ALLOYS
Figure 4.5: Multivariate normal distribution trained with 4000 points split 20 % for
validation. The blue points represent prediction that are within the rmse value showed
in the plot. The red points represent outlier behavior.
4.4.2
Distribution Significance
The ideal dataset would be a distribution that describes sigma in the most general way. An investigation was carried out to understand if a trained model
using one distribution would be able to generalize the others. How the data
is sampled has an effect on the model’s ability to generalize a problem. The
rmse values were similar between uniform, multivariate normal, and Latin hyper cube. (in that order: 2.6-2.8 %) However, they did not all share the same
ability to predict each other. It can be seen in figure 4.6 that the latin hypercube is able to predict the uniform distribution 0.4 rmse weight percent better
than the uniform distributions ability to predict the Latin hypercube.
42
CHAPTER 4. NI BASED ALLOYS
(a) Uniform
(b) Latin hypercube
Figure 4.6: A comparison between how the two distributions can effectively predict each other. This was performed by saving the trained model in a Tensorflow
model.save file format and testing each 4000 point dataset with the saved model. The
latin hypercube has 0.4 mole % better predictive accuracy.
43
CHAPTER 4. NI BASED ALLOYS
4.4.3
Amount of Data Significance
A supervised learning approach to Thermo-Calc’s behaviour of sigma phase
is naturally increasing in understanding with more examples. Increasing the
amount of data would have the most significant impact on the accuracy. By
using a data set that was 10 times larger at 40000 points, a 60 % decrease in
rmse value was obtained, see figure 4.7, with stock parameters table 4.2. At
this size of data, it is still quite practical in handling on a personal computer but
the computational power to calculate this dataset has already become impractical. The use of the multicore processors was key in obtaining these larger
data sets. Approximately 14 hours was needed in computation time to obtain
this data set from TC-Python [49].
Figure 4.7: Latin hypercube distribution trained with the stock parameters and 40000
points split 20 % for validation. The rmse is ±1.6 weight %
4.4.4
Features Significance
99 new columns of information were added to the original dataset. These were
generated with Matminer table 4.4 and descriptors were calculated based off of
the alloys composition table 4.1. To calculate these columns from Matminer,
44
CHAPTER 4. NI BASED ALLOYS
Table 4.4: Featurizers from matminer’s list of featurizers for composition used to
generate 120 new columns. These featurizers are generated from the alloys unique
composition. 99 unique features were ultimately used to train and test.
Featurizer
Miedema
Yang solid solutioning
Atomic packing efficiency
Tmetal fraction
Valence Orbitals
Meredig
Description
Formation enthalpies from intermetallic compounds
Mixing thermochemistry and size
mismatch terms
Packing efficiency based on geometric theory of amorphous packing
Calculates the fraction of magnetic
transition metals in a composition
Attributes of valence orbital shells
Features generated by DFT data
approximately 16 hours was needed for the entire 40000 point dataset. The
rmse value does not alter when using these new parameters, but interestingly,
does not increase the error. The feature importance, are completely different
figure 4.8 showing a correlation between the electronic structure, its attributes,
and its ability to predict phase formation of sigma at an atomic scale.
45
CHAPTER 4. NI BASED ALLOYS
Figure 4.8: Top 20 feature importance with K-best mutual information for the matminer set of new features. Notice how one of the featurizers listed in table 4.4 produces many columns of new data (MagpieData). This contains composition specific
information about the electron structure of the data generated from Thermo-Calc software.
4.4.5
Multioutput Example
A major advantage of using a neural network is its ability to produce multiple
outputs simultaneously with one model. The predicted outputs however do
come with a cost. This usually provides a lower output accuracy and increases
the computational time needed to produce the results (17 minutes to train the
ANN). It can be demonstrated in figure 4.9 that the ANN is able to predict
sigma phase and its corresponding compositions with a mean absolute error of
1.6 %. This technique can be scaled up to larger systems. With a multioutput
model, it’s easy to extend this to phases. Instead of predicting one phase,
multiple phases and the composition of their phases can be predicted.
46
CHAPTER 4. NI BASED ALLOYS
Figure 4.9: Single ANN model used to predict 10 outputs, sigma fraction and its
composition in mole fraction with a mean absolute error of 1.6 %. This model takes
approximately 17 minutes to train with the stock parameters and the 40000 point
dataset.
4.4.6
Tuning the Hyperparameters
It is straight forward to produce an ANN that performs well on training data
but not on the testing data. This is known as overfitting figure 4.10 b) where
the testing error increases as epochs increase. Dropout, weight regularization
and adjusting the learning rate are techniques used to prevent this, called regularization. The idea is to produce a model that has very similar training and
testing errors figure 4.10 a). The learning rate demonstrated with the regularization parameters in table 4.5 were achieved through experimentation as
this subject is highly deliberated; there is not one right way to perform this
task. With the hyperparameters for the 40000 points dataset and single target
of sigma phase amount, it take approximately 15 minutes to train the model.
47
CHAPTER 4. NI BASED ALLOYS
Table 4.5: Hyperparameters used to obtain the best train model. This process was
done with experimentation, moving one hyperparameter at a time and observing the
accuracy. The specific details of the hyperparameters can be obtained from TensorFlow’s documentation [10].
Hyperparameter
Weight regularization
Dropout
Inverse time decay
Optimizer
Value
L2 1E-5
0.01
Learning rate = 0.0001
Decay steps = 10000
Decay rate = 0.125
Adam
(a) Regularized learning curve
(b) Overfitting
Figure 4.10: The learning curve demonstrating how regularization effects the overfitting where a) is tuned, and b) is overfitting. Regularization has a significant impact
of the way the model can predict information is has not seen before. Notice how the
testing data and the training data is very close to the same values in a) and how this
opposite trend is observed in b).
4.4.7
Best Result
When a 40000-point dataset is ran through a trained ANN it computes the entire dataset’s target values in 7.1 seconds on a personal computer. The trained
model is just a saved matrix of weights computed by the earlier neural network. So, when provided with new data, only one forward propagation step
48
CHAPTER 4. NI BASED ALLOYS
Figure 4.11: Best model to predict the amount of sigma in mole fraction using new hyperparameters from table 4.5 with inputs of only composition on the latin hypercube 40000 point data set.
The rmse value
±1.5 weight percent sigma and 0.7% mean absolute error.
through the network can sort out all of the predictions. This is what makes it
so fast. This value is a comparison between a calculation done with ThermoCalc on a multicore processor vs a metamodel being computed on a personal
computer. An estimation of the time to calculate the same data set with a personal desktop using Thermo-Calc is 227 hours (2.6 GHZ dual-core intel i5
with 8Gb 1600 DDR3). It is reasonable to claim that the metamodel is magnitudes faster with respectable accuracy of mean absolute error of 0.7 % off
Thermo-Calc’s thermodynamics model, see figure 4.11.
4.4.8
Comparison Test
A predictor of composition behaviour in a multicomposition dataset was investigated by comparison with Thermo-Calc model and the best trained ANN
figure 4.12 and 4.13. It can be shown the effects of tuning the hyperparameters. In these data sets a single element was varied uniformly while the other
compositions would be held at their average. It can be seen that the ANN is not
able to predict the behavior with precision (2.2% mean absolute error). However, its ability to be improved with the techniques described show potential.
49
CHAPTER 4. NI BASED ALLOYS
It is not a key piece of information to be able to predict this behaviour, rather
this demonstration shows how tuning the parameters of an ANN can produce
a model that is robust where a dataset based off an extreme distribution can be
predicted with some idea of accuracy. With all of data points held constant exactly at the mean, guarantees that the ANN had no examples of this dataset. As
the latin hypercube does not have reused composition or a uniform distribution
of any composition. This is indicated by the behaviour of the old model 4.12.
The ANN has very strange predictions for data it has not seen before. This can
also be observed in the multioutput model see figure, 4.9 where predictions
around the extreme composition space are very inaccurate (Ex: Ti).
(a) Co
(b) Mo
Figure 4.12: Comparison test by creating a uniform data set and calculating the values with TC-Python [41]. The composition are held at their mean composition from
4.1 and the dependent element is varied uniformly. The results obtained here with a
trained ANN with tuned hyperparameters saved as a model.
50
CHAPTER 4. NI BASED ALLOYS
(c) Uniform
(d) Multivariate normal
Figure 4.13: The final results of the tuned latin hypercube model predicting the 40000
point, multivariate normal and uniform data sets in 7.1 second with an average mean
absolute error of 2.1 % and rmse value between 3.2 - 3.4 sigma mole %. This is
considered the accuracy of this assessment as it is important to be able to predict behaviour of the sigma phase in multiple distributions
that could mimic the continuously
51
changing composition space research is done with Ni-based super alloys.
Chapter 5
Conclusions
5.1
Metamodel of Sigma
The ANN networks can be trained on a 40000 point data set in approximately
15 minutes and can compute all target values once trained in 7.1 seconds. The
metamodelling methodology has shown an alternative to the computationally
expensive simulations for materials design. Current techniques to calculate
properties from compositions using thermodynamic simulations are incredibly time consuming. A meta model can find its place in the materials design
process for initially selecting promising candidates. Optimization strategies
involving millions of potential alloys to identify multicriteria performance features can be shortened. The metamodel can act as an additional filter for the
computationally heavy simulations. This technique is flexible, providing multiple outputs for a single model. Ultimately, the question is, is it okay designing
new alloys with an error of 2.1 %? Materials design is a highly selective process. This tool places speed ahead of precision saving the heavy simulations
for performing precise calculations from a reduced set of potential solutions
generated by the metamodel. It is difficult to ignore the improvement of speed
and the comparable accuracy this model has showcased. More research is
needed as this brute force method of obtaining the pattern in thermodynamic
simulations with artificial neural networks is not fully understood. Guarded
progress of these models will need to be placed ahead of the ambition to reduce
computation time.
52
CHAPTER 5. CONCLUSIONS
5.2
Experimental Model of Ms Start Temperature
The results presented were an example of the experimental methodology of
applying machine learning techniques to predict martensite start temperature.
The steps taken to achieve this provided useful techniques in the overall structure of combining material and computer science. The automated way of recognizing patterns with many variables at once is something that has provided
the most useful services in this goal where progress has been made in understanding the application to experimental data. The methods include: automated ways of sorting and cleaning material data, generating intuitions based
on thermodynamic theory applying it to the dataset, and identifying interesting features to provide clues to new higher order interactions. In the overall
computational materials design method using Cohen’s reciprocity [53], machine learning offers a powerful tool that could bridge many gaps by connecting relationships between length scales and material properties. This is not a
technique that replaces experimental practices, as these provide the scientific
proof that this assessment is lacking.
53
Chapter 6
Future Works
6.1
Increase the Complexity of Neural Network Architecture
Data flow’s through an artificial neural network by a path of least resistance.
This path can be improved with different ANN architectures. Advanced techniques such as: convolution-neural networks (NN), recursive-NN, generative
deep learning-NN, and ensemble methods. A promising candidate for this is a
multimodal model using convolution neural networks and ensemble models.
Convolutional neural networks have the ability of incorporating multiple data
sources into one metamodel, opening up for the possibility of adding image
data from SEM, TEM, and XRD crystallography data. By using ensemble
methods in a neural network, the amount of data to produce accurate results
could be reduced, motivated by the demonstration of this method with treebased algorithms in martensite experimental methodology. The attempt to
link the length scales together increases the complexity. A broad library of
techniques should be researched to increase the transparency when this eventually happens.
6.2
Hyperparameter Optimization
This work has demonstrated the effect of tuning the hyperparameters. More
efficient processes of accomplishing this task are available starting with visualizing the neural network better. Tensorflow offers a very well-done API TensorBoard that works specifically with visualization for their neural networks.
It provides statistics and graphs that show how the network is performing. This
54
CHAPTER 6. FUTURE WORKS
can greatly influence the strategy for tuning a hyperparameter. Experimentally
altering one parameter at a time, grid search, and random search is an inefficient method of tuning parameters. An open source library Hyperas [54] is
specifically designed for the use of tuning the hyperparameters for Keras. This
provides a way of automating the tuning the hyperparameters, by predicting
which parameters would perform well.
6.3
Materials Informatics Cloud Based Database
A materials database containing information at all the length scales is the
future of materials informatics. Elementary particle, DFT, thermodynamic,
FEA, and image data from X-ray diffraction (XRD), and scanning electron
microscopy (SEM). Connection of the length scales into a single data set will
help decrease the gap between what is not fully understood about materials
theory. With the expertise and recourses available to Thermo-Calc this could
be a powerful feature to software. A module for predicting properties based on
the entire length of current information could be implemented with the help
of machine learning. Users could design the next generation materials with a
user-friendly API, providing another tool for materials science.
6.4
Sample Around Equilibrium
The way one samples a dataset has an effect on its ability to generalise a problem. By improving the sample quality, a more general metamodel could be
built. I suggest sampling around the equilibrium expression of a particular
phase. Gibbs energy could be optimized by a use of Monte Carlo simulation
by using an acceptance/rejection criterion for calculating the equilibrium of
a phase. Monte Carlo simulations contain properties (Markov Chains) that
guarantee convergence on the equilibrium with the higher number of sampled
points. The samples used to converge on free energy based on composition,
pressure and temperature will become the first dataset. This can be imagined
by looking at a phase diagram with only one phase where the Monte Carlo
optimization simulation would calculate this line (driving force). This would
contain a collection of points concentrated around the equilibrium. Then the
composition, temperature, and pressure will be used as inputs to TC-Python
to calculate the data set used to train the neural network. This would provide
a metamodel with a highly biased set of points, focused around the behaviour
of one specific phase.
55
CHAPTER 6. FUTURE WORKS
6.5
Metamodel Entire Database
Using the techniques previously described, a metamodel of an entire database
is possible where all 680 phases in Thermo-Calc’s TCNI9 database [50] could
be modelled. This would be the beginning of a materials design module, using a series of individually calculated data sets for each phase, their calculated
properties at different length scales, options for selecting relevant information from this data, and/or predictions from feature importance that should be
included. All of the predictions from this are wrapped up into a single metamodel. Massive databases would need to be constructed. With the current
understanding 40000 points per phase. This would need to be coupled with
cloud based, big data techniques and a large investment in hardware. With
expertise in software and materials science, the foundation to accomplish this
is already there for Thermo-Calc.
56
Chapter 7
Acknowledgements
Thank you to PhD Johan Jeppsson at Thermo-Calc AB for giving me so much
of your time when I asked for it and allowing me the resources and freedom
to pursue my own ideas.
Thank you PhD Masoomeh Ghasemi and PhD Giancarlo Trimarchi also at
Thermo-Calc AB for introducing me to feature engineering and selection.
Thank you to my examiner, Malin Selleby professor at department of Materials Science and Engineering, KTH Royal Institute of Technology for pushing
me to produce a thesis better then I could have ever done alone.
57
Chapter 8
References
[1] Richard LeSar. “Materials informatics: An emerging technology for
materials development”. In: Statistical Analysis and Data Mining: The
ASA Data Science Journal 1.6 (2009), pp. 372–374.
[2] Seeram Ramakrishna, Tong-Yi Zhang, Wen-Cong Lu, Quan Qian, Jonathan
Sze Choong Low, Jeremy Heiarii Ronald Yune, Daren Zong Loong Tan,
Stéphane Bressan, Stefano Sanvito, and Surya R Kalidindi. “Materials informatics”. In: Journal of Intelligent Manufacturing 30.6 (2019),
pp. 2307–2326.
[3] Lauri Himanen, Amber Geurts, Adam Stuart Foster, and Patrick Rinke.
“Data-Driven Materials Science: Status, Challenges, and Perspectives”.
In: Advanced Science 6.21 (2019), p. 1900808.
[4] Paul Hernandez. “Materials Genome Initiative”. In: (2017).
[5] Aerospace Materials- Past, present, and future. https : / / www .
aerospacemanufacturinganddesign.com/article/amd0814materials-aerospace-manufacturing. Accessed: 2020-1705.
[6] Platts S P global Jet fuel price. https://www.spglobal.com/
platts/en. Accessed: 2020-11-06.
[7] European Environment Agency, European Topic Centre on Air, and Climate Change. Greenhouse gas emission trends and projections in Europe. Vol. 2007. European Environment Agency, 2007.
58
CHAPTER 8. REFERENCES
[8] Annela Anger. “Including aviation in the European emissions trading
scheme: Impacts on the industry, CO2 emissions and macroeconomic
activity in the EU”. In: Journal of Air Transport Management 16.2
(2010). Selected Papers from the Air Transport Research Society Conference Athens, 2008, pp. 100–105. issn: 0969-6997. doi: https :
//doi.org/10.1016/j.jairtraman.2009.10.009. url:
http://www.sciencedirect.com/science/article/
pii/S096969970900088X.
[9] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O.
Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. “Scikit-learn: Machine Learning in Python”. In: Journal of Machine Learning Research 12 (2011), pp. 2825–2830.
[10] Martın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng
Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu
Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dandelion Mané, Rajat Monga, Sherry
Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens,
Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng.
TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems.
Software available from tensorflow.org. 2015. url: https://www.
tensorflow.org/.
[11] Citrin Informatics. https : / / citrination . com/. Accessed:
20200-17-03.
[12] LA Dobrzański, M Sroka, and J Dobrzański. “Application of neural
networks to classification of internal damages in steels working in creep
service”. In: Journal of Achievements in Materials and Manufacturing
Engineering 20.1-2 (2007), pp. 303–305.
[13] CERN ATLAS, Roberto Bonino, Allan Geoffrey Clark, Xin Wu, Christian Couyoumtzelis, and Hisanori Kambara. Letter of intent for a generalpurpose pp experiment at the Large Hadron Collider at CERN. CERN,
1992.
59
CHAPTER 8. REFERENCES
[14] Hans Leo Lukas, Suzana G Fries, Bo Sundman, et al. Computational
thermodynamics: the Calphad method. Vol. 131. Cambridge university
press Cambridge, 2007.
[15] Scott Kirklin, James E Saal, Bryce Meredig, Alex Thompson, Jeff W
Doak, Muratahan Aykol, Stephan Rühl, and Chris Wolverton. “The Open
Quantum Materials Database (OQMD): assessing the accuracy of DFT
formation energies”. In: npj Computational Materials 1.1 (2015), pp. 1–
15.
[16] Dorota Koziej, Alessandro Lauria, and Markus Niederberger. “25th anniversary article: metal oxide particles in materials science: addressing
all length scales”. In: Advanced materials 26.2 (2014), pp. 235–257.
[17] Paul T Nicholson, Ian Shaw, et al. “Ancient Egyptian materials and
technology”. In: (2000).
[18] Francois Chollet. Deep Learning mit Python und Keras: Das PraxisHandbuch vom Entwickler der Keras-Bibliothek. MITP-Verlags GmbH
& Co. KG, 2018.
[19]
TC-python documentation 2020a. https://www.thermocalc.
com / products - services / software / tc - python/. Accessed: 20200-17-03.
[20] Rajesh Jha, Nirupam Chakraborti, David R Diercks, Aaron P Stebner,
and Cristian V Ciobanu. “Combined machine learning and CALPHAD
approach for discovering processing-structure relationships in soft magnetic alloys”. In: Computational Materials Science 150 (2018), pp. 202–
211.
[21] Keith T Butler, Daniel W Davies, Hugh Cartwright, Olexandr Isayev,
and Aron Walsh. “Machine learning for molecular and materials science”. In: Nature 559.7715 (2018), pp. 547–555.
[22] Logan Ward, Alexander Dunn, Alireza Faghaninia, Nils ER Zimmermann, Saurabh Bajaj, Qi Wang, Joseph Montoya, Jiming Chen, Kyle
Bystrom, Maxwell Dylla, et al. “Matminer: An open source toolkit for
materials data mining”. In: Computational Materials Science 152 (2018),
pp. 60–69.
[23] Andrzej Nowotnik. “Nickel-Based Superalloys”. In: (2016).
60
CHAPTER 8. REFERENCES
[24] Raffi Babikian, Stephen P Lukachko, and Ian A Waitz. “The historical
fuel efficiency characteristics of regional aircraft from technological,
operational, and cost perspectives”. In: Journal of Air Transport Management 8.6 (2002), pp. 389–400.
[25] Rolls Royce press release. https://www.rolls-royce.com/
media/press-releases/2020/28-02-2020-rr-holdingsplc- 2019- full- year- results.aspx. Accessed: 2020-1705.
[26] Ralf Rettig, Nils C Ritter, Harald E Helmer, Steffen Neumeier, and
Robert F Singer. “Single-crystal nickel-based superalloys developed by
numerical multi-criteria optimization techniques: design based on thermodynamic calculations and experimental validation”. In: Modelling
and Simulation in Materials Science and Engineering 23.3 (2015), p. 035004.
[27] Stefan Jonsson. Mechanical Properties of Metals and dislocation Theory from an Engineer’s Perspective. Department of Materials Science
and Engineering, Royal Institute of Technology, 2007.
[28] JH Weber, YE Khalfalla, and KY Benyounis. “Nickel-Based Superalloys: Alloying Methods and Thermomechanical Processing”. In: (2016).
[29] O.P. Morozov, D.A. Mirzayev, and M.M. Shteynberg. “SOME REGULARITIES OF TRANSFORMATIONS IN IRON COOLED AT HIGH
RATES.” In: Physics of Metals and Metallography 32.6 (1971). cited
By 14, pp. 170–177. url: https://www.scopus.com/inward/
record.uri?eid=2-s2.0-0015198005&partnerID=40&
md5=efc5bd4837962402d7b721898cd3d96e.
[30] Gautam Ghosh and Gregory B Olson. “Computational thermodynamics
and the kinetics of martensitic transformation”. In: Journal of Phase
Equilibria 22.3 (2001), p. 199.
[31] Albin Stormvinter, Annika Borgenstam, and John Ågren. “Thermodynamically based prediction of the martensite start temperature for commercial steels”. In: Metallurgical and Materials Transactions A 43.10
(2012), pp. 3870–3879.
[32] Annika Borgenstam and Mats Hillert. “Driving force for fcc→ bcc martensites in Fe X alloys”. In: Acta materialia 45.5 (1997), pp. 2079–2091.
61
CHAPTER 8. REFERENCES
[33] D.A. Mirzayev, M.M. Steynberg, T.N. Ponomareva, and V.M. Schastilivtsev. “INFLUENCE OF COOLING RATE ON THE POSITION OF
MARTENSITIC TRANSFORMATION POINTS. CARBON STEELS.”
In: Physics of Metals and Metallography 47.1 (1979). cited By 17, pp. 102–
111. url: https : / / www . scopus . com / inward / record .
uri ? eid = 2 - s2 . 0 - 0018682734 & partnerID = 40 & md5 =
d439f532ed51383eafa4621372c5f452.
[34] Thermo-Calc Software TCFE Steels/Fe-alloys database version 10. accessed 06 June 2020.
[35] Moshiour Rahaman, Wangzhong Mu, Joakim Odqvist, and Peter Hedström. “Machine Learning to Predict the Martensite Start Temperature
in Steels”. In: Metallurgical and Materials Transactions A 50.5 (2019),
pp. 2081–2091.
[36] TC-Python martensite module (still in development stage). https://
www.thermocalc.com/products-services/software/
tc-python/. Accessed: 20200-17-03.
[37] Gulapura Hanumantharaju and Arun Kumar. Thermodynamic modelling
ofmartensite start temperature in commercial steels. 2018.
[38] Atsuto Maki. Lecture 2 Decision Trees. 2020.
[39] Atsuto Maki. Lecture 10 Ensemble learning. 2020.
[40] Y Freund and R Shapire. “13th International Conference on Machine
Learning”. In: Experiments with a new boosting algorithm (1996), pp. 148–
156.
[41] TC-Python property diagram module TC-python documentation 2020a.
https://www.thermocalc.com/products- services/
software/tc-python/. Accessed: 20200-17-03.
[42] Ryan Jacobs, Tam Mayeshiba, Ben Afflerbach, Luke Miles, Max Williams,
Matthew Turner, Raphael Finkel, and Dane Morgan. “The Materials
Simulation Toolkit for Machine Learning (MAST-ML): an automated
open source toolkit to accelerate data-driven materials research”. In:
Computational Materials Science 176 (2020), p. 109544.
[43] Logan Ward, Ankit Agrawal, Alok Choudhary, and Christopher Wolverton. “A general-purpose machine learning framework for predicting properties of inorganic materials”. In: npj Computational Materials 2 (2016),
p. 16028.
62
CHAPTER 8. REFERENCES
[44] Chandramouli Nyshadham, Matthias Rupp, Brayden Bekker, Alexander V Shapeev, Tim Mueller, Conrad W Rosenbrock, Gábor Csányi,
David W Wingate, and Gus LW Hart. “Machine-learned multi-system
surrogate models for materials prediction”. In: npj Computational Materials 5.1 (2019), pp. 1–6.
[45] Xiaoxin Lu, Dimitris G Giovanis, Julien Yvonnet, Vissarion Papadopoulos, Fabrice Detrez, and Jinbo Bai. “A data-driven computational homogenization method based on neural networks for the nonlinear anisotropic
electrical response of graphene/polymer nanocomposites”. In: Computational Mechanics 64.2 (2019), pp. 307–321.
[46] Raúl Rojas. Neural networks: a systematic introduction. Springer Science & Business Media, 2013.
[47] Andrew Trask. Grokking deep learning. Manning Publications Co., 2019.
[48] Atsuto Maki. Lecture 3 Challenges in Machine learning. 2020.
[49] TC-Python property single equilibrium calculation module TC-python
documentation 2020a. https://www.thermocalc.com/productsservices/software/tc-python/. Accessed: 20200-17-03.
[50] Thermo-Calc Software TCNi Ni-alloys database version 9. accessed 06
June 2020.
[51] Guido Van Rossum and Fred L. Drake. Python 3 Reference Manual.
Scotts Valley, CA: CreateSpace, 2009. isbn: 1441412697.
[52] Wes McKinney et al. “Data structures for statistical computing in python”.
In: Proceedings of the 9th Python in Science Conference. Vol. 445.
Austin, TX. 2010, pp. 51–56.
[53] Gregory B Olson. “Computational design of hierarchically structured
materials”. In: Science 277.5330 (1997), pp. 1237–1242.
[54] Max Pumperla. Hyperas. 2019.
63
TRITA ITM-EX 2020:436
www.kth.se