See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/338589208
Seismic Impedance Inversion Using Fully Convolutional Residual Network and
Transfer Learning
Article in IEEE Geoscience and Remote Sensing Letters · January 2020
DOI: 10.1109/LGRS.2019.2963106
CITATIONS
READS
148
1,542
5 authors, including:
Bangyu Wu
Naihao Liu
Xi'an Jiaotong University
Xi'an Jiaotong University
101 PUBLICATIONS 1,683 CITATIONS
127 PUBLICATIONS 2,586 CITATIONS
SEE PROFILE
Ying Wang
Norges geologiske undersøkelse
21 PUBLICATIONS 198 CITATIONS
SEE PROFILE
All content following this page was uploaded by Bangyu Wu on 18 January 2020.
The user has requested enhancement of the downloaded file.
SEE PROFILE
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS
1
Seismic Impedance Inversion Using Fully
Convolutional Residual Network
and Transfer Learning
Bangyu Wu , Delin Meng, Lingling Wang , Naihao Liu , Member, IEEE, and Ying Wang
Abstract— In this letter, we use a fully convolutional residual
network (FCRN) for seismic impedance inversion. After training with appropriate data, the FCRN can effectively predict
impedance with high accuracy, and have good robustness against
noise and phase difference. However, it cannot give acceptable
results in training and predicting models with different geological
features. Transfer learning is later introduced to ease this
problem. Marmousi2 and Overthrust models are used to verify
the effectiveness of the proposed method. Tests show that after
fine-tuned by five traces of Overthrust model, the FCRN trained
on the Marmousi2 model can give a comparable result similarly
predicted by the FCRN trained purely on the Overthrust model.
Index Terms— Fully convolutional residual network (FCRN),
impedance inversion, transfer learning.
I. I NTRODUCTION
EISMIC impedance is a critical property used for stratigraphic interpretation, and reservoir prediction [1]. Seismic impedance inversion is a procedure to convert seismic
data into rock property. Techniques for impedance inversion
have received much attention over the last 40 years, evolving
from direct inversion to model-based inversion, from linear
inversion to nonlinear inversion, and from poststack inversion
to prestack inversion, and so on [2], [3]. Impedance inversion
is essentially an optimization process with diverse constraints,
which is usually solved by conventional optimization methods.
During the boom of machine learning in recent decades,
deep learning is a popular branch that has developed
significantly [4]. In 1998, LeCun et al. [5] proposed
the convolutional neural networks (CNNs) deep learning
S
Manuscript received August 20, 2019; revised December 2, 2019; accepted
December 24, 2019. This work was supported in part by the National
Natural Science Foundation of China under Grant 41674123, Grant 41874154,
Grant 41404107, and Grant 41904102, in part by National Postdoctoral
Program under Grant 2016M600780 and Grant BX201900279, and in part
by Fundamental Research Funds for the Central Universities under Grant
xjj2018260 and Grant xjh012019030. (Corresponding author: Lingling Wang.)
Bangyu Wu and Delin Meng are with the School of Mathematics
and Statistics, Xi’an Jiaotong University, Xi’an 710049, China (e-mail:
bangyuwu@xjtu.edu.cn; mengdelin@stu.xjtu.edu.cn).
Lingling Wang is with the Institute of Geophysics and Geomatics, China University of Geosciences, Wuhan 430074, China (e-mail:
wangll@cug.edu.cn).
Naihao Liu is with the National Engineering Laboratory for Offshore Oil
Exploration, Xi’an Jiaotong University, Xi’an 710049, China, and also with
the School of Information and Communications Engineering, Xi’an Jiaotong
University, Xi’an 710049, China (e-mail: naihao_liu@mail.xjtu.edu.cn).
Ying Wang is with Geological survey of Norway (NGU), 7491 Trondheim,
Norway (e-mail: ying.wang@ngu.no).
Color versions of one or more of the figures in this letter are available
online at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/LGRS.2019.2963106
architecture and since then CNNs have achieved outstanding
performances in different areas such as computer vision [6],
and speech recognition [7]. Due to its excellent capability
in extracting and learning complex features, CNNs have
drawn more and more attention with many flourishing
variants. Fully convolutional networks (FCNs) are one such
representative where fully connected layers are replaced by
locally connected, i.e., convolutional layers in the network.
FCNs have been widely used for semantic segmentation [8].
CNNs and FCNs have been applied to tackle some geophysical problems, such as seismic waveform classification and
first-break picking [9], [10], seismic data interpolation [11],
and fault detection [12]. Inversion by machine learning is also
a topic drawing attentions from both academia, and industry.
Das et al. [13] used a two-layer FCN to obtain impedance
from seismogram and systematically tested the generalization
ability of the FCN. Puzyrev et al. [14] used a CNN, a FCN,
and a recurrent neural network (RNN), respectively, to predict
velocity models from seismic data and proved the applicability
of deep neural networks in seismic inversion. Xu et al. [15]
proposed to pretrain a U-net (a special FCN) by a rough
prediction result obtained by traditional method, and then to
fine-tune the pretrained U-net by well logging data for field
data prediction. Guo et al. [16] used a bi-directional long shortterm memory RNN for seismic impedance inversion. In their
study, training samples of the network were generated from the
logging data of 150 wells. However, in practice, the number of
available well logs is usually much smaller due to cost. The
lack of labeled data (“the ground truth,” from well logs for
instance) for network training is a common problem for the
application of deep learning methods to geophysical problems.
In this letter, we use a fully convolutional residual network
(FCRN) for the acoustic impedance inversion. In Section II,
we first introduce the two models used for our impedance
inversion demonstration, the Marmousi2, and the Overthrust;
then, in Section III, the network architecture and the details of
network training are introduced. Transfer learning is evoked to
use as small as five-traces from the Overthrust model to finetune the networks trained on the Marmousi2 model, resulting
an improved prediction of the Overthrust model impedance.
Conclusions are given in Section V.
II. S YNTHETIC S EISMIC DATA S ET
Two synthetic data sets are used in this letter, which
are denoted as the Marmousi2 model (Marmousi2), and the
Overthrust model (Overthrust), respectively.
1545-598X © 2020 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
2
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS
Fig. 1. Marmousi2 model data set. (a) Impedance and (b) synthetic seismic
data generated by 30-Hz 0◦ phase Ricker wavelet.
A. Marmousi2
The first model is produced from the Marmousi2 P-wave
velocity model where density is assumed to be constant. The
profile of the impedance is shown in Fig. 1(a). The size of the
profile is 13 601 traces with 2800 time points, and the time
interval is 1 ms. Then reflectivity is calculated by
Z i+1 − Z i
ri =
(1)
Z i+1 + Z i
where ri is the i th sampling point of reflectivity r (t), and
Z i is the i th sampling point of impedance Z (t). For the
training data set, we use various Ricker wavelets with a
combination of dominant frequencies (DFs) at 30, 40, 50, and
60 Hz and phases of 0◦ , 10◦ , 20◦ , 30◦, and 45◦ , respectively,
to generate a set of various synthetic seismic data. Specifically,
we convolve the reflectivity with the aforementioned Ricker
wavelets separately, and obtain 20 different synthetic seismic
data profiles in total. An example profile generated using
Ricker wavelet with 30-Hz DF, and 0◦ phase is demonstrated
in Fig. 1(b).
B. Overthrust
The second model is generated from a 2-D profile that
is spliced by three profiles extracted from the Overthrust
3-D velocity model, and density is also set to constant. The
impedance profile is shown in Fig. 2(a), and the size of the
profile is 2800 × 401. A Ricker wavelet with 30-Hz DF and
0◦ phase is convolved with the reflectivity calculated from the
impedance to generate the synthetic seismic data [Fig. 2(b)].
From the Overthrust model, we have also extracted five
traces (trace number 50, 100, 150, 200, and 300), which is less
than 1.25% of the entire model, as an analog of the sparsely
available well logs in practice. Here, these five traces can be
regarded as pseudowells. Cubic convolution interpolation technique is utilized to generate 20 new impedance and seismic
data traces between every two adjacent pseudowells [17]. The
interpolated 80 traces of impedance and synthetic seismic data
together with the original five traces [Fig. 2(c) and (d)] are
used to fine-tune the network pretrained on Marmousi2. The
positions of the five pseudowells are shown as vertical dashed
lines in Fig. 2(b).
III. M ETHODOLOGY
A. Network Architecture
FCNs are a type of CNNs without the fully connected
layers, which can reduce the parameters of the network and are
Fig. 2. Overthrust model and the interpolated data set for transfer learning.
(a) Impedance (b) and its corresponding synthetic seismic data generated
with 30-Hz 0◦ phase Ricker wavelet. (c) Interpolated impedance and (d) its
corresponding synthetic seismic data generated with 30-Hz 0◦ phase Ricker
wavelet for transfer learning. The five vertical dashed lines in (b) denotes the
position of pseudowells.
able to predict from the inputs of arbitrary sizes [8]. With this
advantage, FCNs can be used to solve inversion problems [18].
Deeper neural networks can enrich the extraction of features. However, the degradation problem is often observed
while training deeper neural networks. He et al. [19] proposed
the residual networks (ResNets) which are easier to optimize,
and can gain accuracy from considerably increased depth.
Other than directly learning the target mapping by stacked
layers, the deep ResNets stack the residual blocks and fit a
residual mapping for each residual block. A Lot of experiments
show that the deep ResNets are easy to train, and better
performances are observed when increasing the depth over
plain networks [19].
In this letter, we take the advantage of the FCNs and
ResNets to design a FCRN, which can improve prediction
accuracy while ease the network training problem. The architecture of the FCRN is shown in Fig. 3, which has stacked
three residual blocks between two 1-D convolution layers.
To better capture the low frequency features of the seismic
data and inspired by the work of Das et al. [13], the first
convolution layer of the FCRN has 16 kernels of size 300 × 1;
each residual block consists of two convolution layers, where
the first layer has 16 kernels of size 300 × 1, followed by
another layer with 16 kernels of size 3 × 1; the last convolution
layer has 1 kernel of size 3 × 1. The stride is set to one,
and zero-padding is used in all convolution layers, which
keep the size of the input and output of each convolution
layer the same. In order to improve the network’s nonlinear
expressiveness and accelerate the convergence, we choose the
rectified linear unit (ReLU) to activate the outputs of the first
and the last convolution layer, respectively, as well as that
of the first layer of each residual block, and of each residual
block [6], [20], [21]. In addition, we apply batch normalization
(BN) to the output of all the convolution layers except for the
last layer to speed up network training [22].
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
WU et al.: SEISMIC IMPEDANCE INVERSION USING FCRN AND TRANSFER LEARNING
Fig. 3.
3
Architecture of the FCRN.
B. Network Training
The networks are trained in a trace-to-trace fashion.
Specifically, let s denote a seismic data trace and use it as
the input of the network. The output denoted by (s) is
the corresponding impedance trace predicted by the network,
where denotes the trained network, and represents the
parameters including the elements in each convolution kernel
(bias is set to none in the network).
For the Marmousi2 model, we randomly sample
10 601 traces from each profile with 0◦ phase, and different
DFs (30, 40, 50, and 60 Hz) for training. Two groups
of 1500 traces from each profile are used for validation and
testing, respectively.
For the Overthrust model, we randomly sample 351 traces
for training, and two groups of 25 traces are used for validation
and testing, respectively.
The loss function of the networks is mean squared error
(mse), defined as
Loss = (s) − I(s)22
(2)
where I(s) denotes the labeled impedance trace. In all the
tests, the batch size is set to ten since small-batch method
leads to strong generalization of the network [23]. In addition,
Adam algorithm [24] is used to optimize the parameters in
the network, and the parameter weight decay is set to 10−7 .
The number of epoch is set to 50. For the first five epochs,
the learning rate is set to 0.01 and then decreases to 0.001.
To avoid overfitting, the network training is stopped when the
loss of validation begins to grow.
C. Transfer Learning
Many machine learning methods follow a crucial hypothesis
that the data for training and for prediction must be in the same
feature space, and same distribution [25]. In our case, Marmousi2 and Overthrust are not with same geological features.
Numerical tests show that the networks trained from one model
cannot give acceptable predictions of the other. To address this
issue, we use transfer learning to adjust the pretrained network
on Marmousi2 to adapt to the Overthrust model prediction.
This experiment also tries to reflect a practical situation when
the ground truth label can only be obtained from a limited
number of well logs. Therefore, we only use five traces (mimic
five wells in practice) from the Overthrust model for transfer
learning. We copy the architecture and all parameters from
the FCRN trained on Marmousi2 as an initialization of a new
FCRN. Then we train the initialized FCRN on the data set
generated with these five traces from the Overthrust model.
Fig. 4. Predictions of the FCRN and FCN when the input is generated using
0◦ phase wavelet with DFs of (a) 30, (b) 40, (c) 50, and (d) 60 Hz.
Here, the learning rate is set to 0.001 while the other training
settings are the same as those for the Marmousi2 test.
IV. E XPERIMENTS
A. Experiment on Marmousi2 Model
In this experiment, we train and test the proposed FCRN on
the Marmousi2 model. As comparison, FCN by Das et al. [13]
is also trained and tested on the same data set.
We first compare the capability of FCRN and FCN for training/predicting from seismic data with same Ricker wavelets.
Fig. 4 shows impedance at same location from seismic data
generated by Ricker wavelets of 30 [Fig. 4(a)], 40 [Fig. 4(b)],
50 [Fig. 4(c)], and 60 Hz [Fig. 4(d)]. The red, green, and
blue lines are for true, FCN, and FCRN impedance. It can be
seen that the result predicted by FCRN is more consistent
with the true impedance than that by FCN. Fig. 5 shows
the profiles predicted by the FCN [Fig. 5(a)–(d)] and FCRN
[Fig. 5(e)–(h)], respectively. Both networks are able to predict
the impedance, while the profiles predicted by the FCRN have
shown better lateral continuity, in general, and also impedance
details than those predicted by FCN. To be precise, the mse
of the profiles predicted by the FCN and FCRN are reported
in Table I, which explicitly demonstrates that the FCRN
outperforms FCN.
Later we test the tolerance to wavelet phase error of the
FCN, and the FCRN. We took the traces of the seismic data
generated using Ricker wavelets with 30-Hz DF and 10◦ , 20◦ ,
30◦, and 45◦ phase rotations, which are not contained in the
training data (0◦ phase data), as the inputs of both networks.
Predictions show that both networks have some degree of
tolerance to the Ricker wavelet phase difference of training,
and predicting (Fig. 6). It shows that the impedance predicted
by the FCRN matches the true impedance better than that by
the FCN. However, the predictions from both networks are not
as good as that of training/predicting with Ricker wavelets of
same phase.
We also test the FCRN’s tolerance to noise. We add five
different levels of Gaussian noise with signal-to-noise ratio
(SNR) of 45, 35, 25, 15, and 5 dB, while the training data is
clean. The mses between the prediction of different SNR data
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
4
IEEE GEOSCIENCE AND REMOTE SENSING LETTERS
Fig. 5. Profiles predicted by FCN (first row) and FCRN (second row). The inputs are generated using 0◦ phase wavelet with DFs of (a) and (e) 30 Hz,
(b) and (f) 40 Hz, (c) and (g) 50 Hz, and (d) and (h) 60 Hz.
TABLE I
MSE OF THE P REDICTION P ROFILES V ERSUS DF
OF THE S OURCE WAVELET
TABLE II
MSE S OF THE P REDICTIONS ON S EISMIC W ITH D IFFERENT SNR
FOR D IFFERENT DF S OF THE S OURCE WAVELET
Fig. 6. Predictions of the FCRN and FCN when the input is generated using
Ricker wavelets with 30-Hz DF and (a) 10◦ , (b) 20◦ , (c) 30◦ , and (d) 45◦
phase rotations.
and the real impedance are presented in Table II. Comparing
Tables I and II, it shows that FCRN is more accurate than FCN
predicting by data without noise when the SNR is above 25.
For prediction results by SNR of 15 and 5 dB with the mses
between 0.0797 and 0.2913, the main structures can still be
identified from the FCRN-predicted impedance profiles.
B. Experiment on Overthrust Model
In this experiment, we denote the FCRN trained on
Marmousi2 model as the pretrained network and use transfer
learning to fine-tune the pretrained FCRN (fine-tuned FCRN)
for impedance prediction on the Overthrust model.
We compare the impedance predicting performance among:
1) the fine-tuned FCRN using only five traces from the
Overthrust model; 2) the pretrained FCRN using none of the
data from the Overthrust model; and 3) the FCRN trained
from scratch using samples from the full set of the Overthrust
model. The three sets of results are illustrated in Fig. 7 together
with the true impedance as benchmark. Impedance traces at
four locations are demonstrated for a closer inspection. It can
be seen that the pretrained FCRN from Marmousi2 model
predicts the high-frequency components of the impedance
but fails to give the low-frequency components (orange line
in Fig. 7), which is caused by the different geological features
of the training data (Marmousi2), and the predicting data
(Overthrust model) [25]. The fine-tuned FCRN uses only five
traces that give comparable results with the FCRN training
from scratch on the Overthrust model. Fig. 8 shows the predicted impedance profiles by the FCRN training from scratch
[Fig. 8(a)], and the fine-tuned FCRN [Fig. 8(b)]. Fig. 8(a)
shows great lateral continuity and is highly consistent with
the true impedance. The whole predicted impedance profile
by the fine-tuned FCRN is comparable to that by the FCRN
training from Overthrust model, which verifies the transfer
learning strategy.
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.
WU et al.: SEISMIC IMPEDANCE INVERSION USING FCRN AND TRANSFER LEARNING
Fig. 7. Impedance traces from Overthrust model with trace number (a) 120,
(b) 220, (c) 320, and (d) 350. Predictions of the pretrained FCRN by the
Marmousi2 model (orange), the fine-tuned FCRN (blue), and the FCRN
training from scratch using the Overthrust model (green). The red line is
the true impedance.
Fig. 8. Profiles predicted by (a) FCRN training from scratch and fine-tuned
FCRN by (b) five traces of Overthrust model.
V. C ONCLUSION
In this letter, we apply a FCRN combined with a transfer
learning strategy for seismic impedance inversion. Numerical
tests show that the FCRN can effectively predict the impedance
with high accuracy, and have good robustness against noise
and phase difference. Due to different geological features,
the network trained from one model cannot give acceptable
predictions on the other. Transfer learning can greatly mitigate
this issue by using only a few traces from the new model.
Tests on Marmousi2 and Overthrust models validate transfer
learning on impedance inversion. This offers a potential solution to the impedance inversion on field data. The network
can be trained on a data set with sufficient well logs while
using only a few well logs from the target area for prediction.
Furthermore, it is also possible to train the network on a
synthetic model and perform the transfer learning with a few
well logs for field data impedance prediction.
R EFERENCES
[1] S. Yuan, S. Wang, C. Luo, and Y. He, “Simultaneous multitrace
impedance inversion with transform-domain sparsity promotion,” Geophysics, vol. 80, no. 2, pp. R71–R80, Mar. 2015.
[2] S. Yuan, Y. Liu, Z. Zhang, C. Luo, and S. Wang, “Prestack stochastic
frequency-dependent velocity inversion with rock-physics constraints
and statistical associated hydrocarbon attributes,” IEEE Geosci. Remote
Sens. Lett., vol. 16, no. 1, pp. 140–144, Jan. 2019.
View publication stats
5
[3] S. Yuan, S. Wang, Y. Luo, W. Wei, and G. Wang, “Impedance
inversion by using the low-frequency full-waveform inversion result
as an a priori model,” Geophysics, vol. 84, no. 2, pp. R149–R164,
Mar. 2019.
[4] I. Goodfellow, Y. Bengio, A. Courville, Deep Learning. Cambridge, MA,
USA: MIT Press, 2017, pp. 1–2.
[5] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proc. IEEE, vol. 86, no. 11,
pp. 2278–2324, Nov. 1998.
[6] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “ImageNet classification
with deep convolutional neural networks,” in Proc. Adv. Neural Inf.
Process. Syst., 2012, pp. 1106–1114.
[7] O. Abdel-Hamid, A.-R. Mohamed, H. Jiang, and G. Penn, “Applying
convolutional neural networks concepts to hybrid NN-HMM model for
speech recognition,” in Proc. IEEE Int. Conf. Acoust., Speech Signal
Process. (ICASSP), Mar. 2012, pp. 4277–4280.
[8] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks
for semantic segmentation,” IEEE Trans. Pattern Anal. Mach. Intell.,
vol. 39, no. 4, pp. 640–651, Apr. 2017.
[9] S. Yuan, J. Liu, S. Wang, T. Wang, and P. Shi, “Seismic waveform classification and first-break picking using convolution neural
networks,” IEEE Geosci. Remote Sens. Lett., vol. 15, no. 2, pp. 272–276,
Feb. 2018.
[10] H. Wu, B. Zhang, F. Li, and N. Liu, “Semiautomatic first-arrival picking
of microseismic events by using the pixel-wise convolutional image
segmentation method,” Geophysics, vol. 84, no. 3, pp. V143–V155,
May 2019.
[11] B. Wang, N. Zhang, W. Lu, and J. Wang, “Deep-learning-based seismic
data interpolation: A preliminary result,” Geophysics, vol. 84, no. 1,
pp. V11–V20, Jan. 2019.
[12] X. Wu, L. Liang, Y. Shi, and S. Fomel, “FaultSeg3D: Using synthetic
data sets to train an end-to-end convolutional neural network for 3D
seismic fault segmentation,” Geophysics, vol. 84, no. 3, pp. IM35–IM45,
May 2019.
[13] V. Das, A. Pollack, U. Wollner, and T. Mukerji, “Convolutional neural
network for seismic impedance inversion,” Geophysics, vol. 84, no. 6,
pp. R869–R880, Nov. 2019.
[14] V. Puzyrev, A. Egorov, A. Pirogova, C. Elders, and C. Otto, “Seismic
inversion with deep neural networks: A feasibility analysis,” presented
at the 81st EAGE Conf. Exhib., London, U.K., Jun. 2019.
[15] P. Xu, W. Lu, J. Tang, and L. Chen, “High-resolution reservoir prediction
using convolutional neural networks,” presented at the 81st EAGE Conf.
Exhib., London, U.K., Jun. 2019.
[16] R. Guo, J. J. Zhang, D. Liu, Y. B. Zhang, and D. W. Zhang, “Application
of bi-directional long short-term memory recurrent neural network for
seismic impedance inversion,” presented at the 81st EAGE Conf. Exhib.,
London, U.K., Jun. 2019.
[17] R. Keys, “Cubic convolution interpolation for digital image processing,”
IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-29, no. 6,
pp. 1153–1160, Dec. 1981.
[18] V. Puzyrev, “Deep learning electromagnetic inversion with convolutional
neural networks,” Geophys. J. Int., vol. 218, no. 2, pp. 817–832,
Aug. 2019.
[19] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for
image recognition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.
(CVPR), Jun. 2016, pp. 770–778.
[20] V. Nair and G. E. Hinton, “Rectified linear units improve restricted
Boltzmann machines,” in Proc. Int. Conf. Mach. Learn., 2010,
pp. 807–814.
[21] B. Xu, N. Wang, T. Chen, and M. Li, “Empirical evaluation of rectified activations in convolutional network,” 2015, arXiv:1505.00853.
[Online]. Available: https://arxiv.org/abs/1505.00853
[22] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep
network training by reducing internal covariate shift,” in Proc. Int. Conf.
Mach. Learn., 2015, pp. 448–456.
[23] N. S. Keskar, D. Mudigere, J. Nocedal, and M. Smelyanskiy, “On largebatch training for deep learning: Generalization gap and sharp minima,”
in Proc. Int. Conf. Learn. Representations, Apr. 2017.
[24] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,”Dec. 2014, arXiv:1412.6980. [Online]. Available:
https://arxiv.org/abs/1412.6980
[25] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE
Trans. Knowl. Data Eng., vol. 22, no. 10, pp. 1345–1359,
Oct. 2010.