Supplementary Materials for the Manuscript
Disorder identification in hysteresis data: recognition analysis
of random-bond random-field Ising model
O. S. Ovchinnikov,1 S. Jesse,2 P. Bintacchit,3 S. Trolier-McKinstry,3 and S.V. Kalinin1
1
Department of Physics and Astronomy, University of Tennessee, Knoxville, TN
2
Oak Ridge National Laboratory, Oak Ridge, TN 37831
3
Department of Materials Science and Engineering and Materials Research Institute,
Pennsylvania State University, University Park, PA 16802
I. Recognition analysis of frustrated spin glass
The system evolution is described using Glauber dynamics. On each time step, the spin is
i
  J ij S j  H  hi  , i.e. for
assumed to flip if it is antiparallel to the local field hloc
j
i
i
hloc
S i  0 . For hloc
S i  0 the spin is stable. At each field value, H , the spin values are
updated randomly until 100 iterations of the average magnetization S i do not change.
The resulting hysteresis loop S H  S i H  defines the field dependence of average
magnetization averaged over 40 realizations of disorder.
As an example of recognition analysis for a 2 parameter Ising model, frustrated
spin glass model with J 0 = 0 and varying J and h was studied. Overall, 363 curves
1
Corresponding author, sergei2@ornl.gov
1
corresponding to J = 0, .., 5 and h = 0, .., 16 with the 0.5 step for both parameters
have been generated.
Characteristic loop shapes for spin glass model are shown in Fig. S.1.
1.0
Magnetization, a.u.
Magnetization
1.0
0.5
0.0
-0.5
h
J
0.0
-0.5
-1.0
-1.0
-40
(a)
0.5
-20
0
20
Magnetic Field, a.u.
-40
40
(b)
-20
0
20
Magnetic field, a.u.
40
Fig. S.1. Hysteresis loops for frustrated spin glass, J 0 = 0. (a) Hysteresis loops for J =
3 and h = 0, 4, 8, 12, 16. (b) Hysteresis loops for h = 0 and J = 1, 2, 3, 4.
For the spin glass, the relatively smooth shape of the hysteresis loops is reflected in a
regular shape of the PCA eigenvectors, as shown in Fig. S.2. The structure of the
principal components is illustrated in Fig. S.2 (c), illustrating the relative magnitude of
the first 12 PCA components in the parameter space h vs. J . The relative intensity of
colors in the i-th PCA map corresponds to the relative contribution of the i-th eigenvector
to a hysteresis loop shape for the corresponding parameter value. The structure of the
surface allows determination of the origins of variability and noise in the analysis. The
presence of relatively smooth surfaces gradually evolving for higher component numbers
is indicative of the well-behaved decomposition and suggests a reliable recognition
2
analysis. The presence of Gaussian noise in the data leads to a random component map if
the intensity of noise exceeds the corresponding critical value for a given component.
Finally, the noise associated with a finite model size (100 spins) and large jumps in the
response results in the “random looking” PCA maps such as those for 7th and 12th
components in Fig. S.2 (c). Thus, the structure of principal component surfaces in the
parameter space of the system can be used to determine the statistical reliability of the
training data set, and determine an upper limit for the number of inputs to a neural
network.
3
4
Eigenvector, a.u.
6
Ln(eigenvalue)
4
2
0
0
-2
-4
-2
0
(a)
2
10
20
30
40
Eigenvalue number
-40
50
(b)
-20
0
20
40
Field, a.u.
(c)
Fig. S.2. Principal component analysis of data set for the frustrated spin glass model. (a)
scree plot (eigenvalues plotted from largest to smallest). (b) First four PCA eigenvectors
(divided by 1/4, 1/9, and 1/16 for 2nd, 3rd, and 4th eigenvector). (c) First 12 PCA
components in the parameter space as a function of h (horizontal axis) and J (vertical
axis).
4
The principal components were used to train a (10, 24, 2) feed-forward neural
network with a tansig transfer function in the hidden layer and a linear transfer function
in the output layer.1 For optimal fitting, the overall error decreased by ~1.5 – 2 orders of
magnitude compared to a random input. After training, the validity of network training
was checked using the simulated array, and a comparison of the network output and
model parameters is shown in Fig. S.3.
6
5
4
dh_sim
J_sim
3
2
1
0
0
1
2
3
J_real
4
-10.0
-6.0
-2.0
2.0
6.0
10.0
15
dh_real
10
5
0
(c)
5
0
1
2
3
4
(b)
4
6 8 10 12 14 16 18
dh_real
-10.0
-6.0
-2.0
2.0
6.0
10.0
10
5
0
5
dJ_real
2
15
dh_real
-1
(a)
18
16
14
12
10
8
6
4
2
0
-2
-2 0
0
(d)
1
2
3
dJ_real
4
5
Fig. S.3. Neural network reconstruction of a 2–parameter random field Ising model. (a)
Reconstructed J vs. real value for h = 0.5, 1, ..5. (b) Reconstructed h vs. real value
for J 0 = 0, 0.5, .., 5. (c) Error map (in %) for J reconstruction and (d) Error map for h
reconstruction (for a normalized date set) within the parameter space of the system as a
function of the model J and h .
5
The correspondence between the recognized and actual values of random bond,
J , and random field, h , disorders illustrate excellent reconstruction. Note that the
vertical extent of the response points in Figs. S.3 (a) represents a full set of reconstructed
J values for all h values, for the overall error of several percent. The corresponding
distribution for h is larger, but still remains within ~10%.
The structure of corresponding error surfaces, is shown in Fig. S.3 (c,d). Here, to
improve the recognition procedure, the hysteresis loops were normalized to have the
same value of coercive field, i.e. field H c for which S H   0 and +(-) refers to forward
(reverse) branches of the hysteresis loops. The horizontal axis was normalized using a
parameter dependent scaling factor,  , so that H c  1 . The error surface for J
illustrates that the error is below 2-3% for the full parameter space, except in the vicinity
of J = 0. In other words, the recognition error is constant, rather than linear in J , as
can be observed directly from the data scatter in Fig. 2 (a). The error surface for h is
shown in Fig. S.3 (d) and illustrates that the recognition error is less than ~5% in the full
range of parameters excluding the region for which h < J (in absolute units). This
behavior can be understood from direct visual inspection of the corresponding hysteresis
loops, which are almost indistinguishable for large J and small h values (i.e. one type
of disorder dominates).
2. Random bond Ising model
As a second example of a 2-parameter model, we performed recognition analysis
of the random bond Ising model, i.e. varying J 0 and J in Eq. (1). The hysteresis loops
6
for several parameter values are shown in Fig. S.4. Overall, 77 curves corresponding to
J 0 = 0, .. 3 and J = 0, .., 5 with the 0.5 step in parameter values were generated. The
loops for J 0 = 0 show a characteristically elongated shape. The loops exhibit
characteristic transitions from square-like to elongated ones with an increase of the
Magnetization, M/Mmax
1.0
0.5
0.0
-0.5
-1.0
-40
(a)
-20
0
Field, a.u.
20
40
Normalized magnetization, M/Mmax
J J 0 ratio, as shown in Fig. S.4 (a).
1.0
0.5
0.0
-0.5
-1.0
-40
-20
(b)
0
20
40
Field, H
5
3
2
3
dJ_sim
J_sim
4
1
2
1
0
0
-1
0.0 0.5 1.0 1.5 2.0 2.5 3.0
J_real
(c)
0
1
2
3
4
5
dJ_real
(d)
Fig. S.4. Hysteresis loops for random the bond model, h = 0. for J 0 = 2 and J = 1, 2,
3, 4, 5. (b) Reconstructed hysteresis loop using the 1st 6 eigenvalues for J 0 , J  = 1,1
(red), 1.5,3 (green), and 3,5 (blue). Shown are the original hysteresis loop (solid) and
the PCA reconstruction one(dashed). (c) Reconstructed J 0 vs. real value for J = 0.5, 1,
7
..5. (d) Reconstructed J vs. real value for J 0 = 0, 0.5, 1, ..3. Black lines correspond to
ideal value.
The data set was analyzed using principal component decomposition and the
eigenvalue plot shows that ~99% of information is concentrated in the first 6 eigenvalues.
Hence, the full information contained in the hysteresis loop samples at 900 points can be
well represented by the first 6 components. The eigenvalue plot does not saturate, as
expected in the absence of Gaussian noise. The first 6 principal components allow for
very good reproduction of the hysteresis loops, as shown in Fig. S.4 (d). The noise effect
on PCA behavior is analyzed in detail elsewhere,2 and the presence of uncorrelated white
noise can be detected as the saturation of the eigenvalue plot above a certain critical
value, and corresponding eigenvectors become essentially random. Note that in this
analysis the hysteresis loops are not scaled with respect to the coercive field, i.e. both J 0
and J are free parameters.
The use of PCA allows for the decorrelation of the data, reducing it to a linear
combination of orthonormal vector components, Eq. (2). Thus, the obtained 6 PCA
components were used as inputs for (6,12,2) feed forward neural network, with a tansig
transfer function in the hidden layer and linear transfer function in the output layer. The
J 0 , J 
values were used as examples for supervised network training. On repetitive
runs, the network error converged to ~0.1 from an initial ~3, leading to an estimated ~3%
error. Note that in all cases the reconstruction error is of the order of ~3%, suggesting the
validity of the process.
8
3. Random Field Ising model
Finally, the recognition was analyzed for a random field Ising model, i.e. varying
J 0 and h .
6
15
5
dh_real
dh_real
4
3
2
10
5
1
0
0
(a) -1
0
1
2
3
4
0
(b)
5
5
J_real
14
12
dh_real
10
8
6
12
10
8
6
4
2
2
0
(c)
1
2
3
J_real
4
-10.0
-6.0
-2.0
2.0
6.0
10.0
14
4
0
15
16
dh_real
-10.0
-6.0
-2.0
2.0
6.0
10.0
16
0
10
J_real
5
0
(d)
1
2
3
J_real
4
5
Fig. S.5. Neural network reconstruction of a 2 – parameter Ising model for widthnormalized data. (a) Reconstructed J vs. real value for h = 0.5, 1, ..5. (b)
Reconstructed h vs. real value for J 0 = 0, 0.5, .., 5. (c) Error map (in %) for J
reconstruction. (d) Error map for h reconstruction. The vertical scale corresponds to
error (in %)
4. Effect of normalization on reconstruction
9
(a)
(b)
Fig. S.6. PCA components for random-field Ising model (a) without and (b) with
normalization.
One notable aspect of the analysis is the effect of normalization, as shown in Fig.
S.6. Here, the PCA components for non-normalized data set develop a “checkerboard”
structure, indicative of highly non-uniform behavior of principal components in the
parameter space. Normalization precludes this behavior, and also illustrates that the
10
number of significant components is limited to ~6. The effect of normalization on the
reconstruction is shown in Figs. S.7,8. The overall scatter of reconstructed values is
improved by a factor of ~2, and the distribution of reconstruction error becomes
significantly more uniform after normalization.
6
5
15
4
dh_sim
J_sim
3
2
10
5
1
0
(a)
0
0
1
2
3
4
5
J_real
-10.0
-6.0
-2.0
2.0
6.0
10.0
16
14
12
dh_real
10
8
6
4
-10.0
-6.0
-2.0
2.0
6.0
10.0
10
8
6
2
3
8 10 12 14 16
12
4
2
6
14
2
1
4
dh_real
4
0
2
16
0
0
5
J_real
(c)
0
(b)
dh_real
-1
0
(d)
1
2
3
J_real
4
5
Fig. S.7. Neural network reconstruction of 2 – parameter Ising model. (a) Reconstructed
J vs. real value for h = 0.5, 1, ..5. (b) Reconstructed h vs. real value for J 0 = 0, 0.5,
.., 5. (c) Error map (in %) for J reconstruction. (d) Error map for h reconstruction.
11
6
15
5
dh_real
4
dh_real
3
2
10
5
1
0
(a)
0
-1
0
1
2
3
4
0
(b)
5
5
16
14
12
dh_real
10
8
6
16
12
10
8
6
4
4
2
2
0
(c)
1
2
3
J_real
4
-10.0
-6.0
-2.0
2.0
6.0
10.0
14
dh_real
-10.0
-6.0
-2.0
2.0
6.0
10.0
0
15
J_real
J_real
0
10
5
0
1
(d)
2
3
J_real
4
5
Fig. S.8. Neural network reconstruction of 2 – parameter Ising model for widthnormalized data. (a) Reconstructed J vs. real value for h = 0.5, 1, ..5. (b)
Reconstructed h vs. real value for J 0 = 0, 0.5, .., 5. (c) Error map (in %) for J
reconstruction. (d) Error map for h reconstruction.
References
1
S. Samarasinghe, Neural Networks for Applied Sciences and Engineering: From
Fundamentals to Complex Pattern Recognition, Auerbach Publications (2006).
2
S. Jesse and S.V. Kalinin, Nanotechnology 20, 085714 (2009).
12