M.Sc. Thesis Proposal
Classification of Autism Spectrum Disorder Using Multi-Modal
Data and Transfer Learning Based Approach
Presented By:
Adnan Ferdous Ashrafi
Student ID: 171041014
Supervised By:
Prof. Dr. Md. Hasanul Kabir
Department of Computer Science and Engineering
Islamic University of Technology
September 9, 2022
Department of CSE, IUT
Classification of ASD
1 / 13
Overview
Motivation
Literature Review
Research Scope
Research Objectives
Possible Outcomes
Outline of Research Methodology
References
Department of CSE, IUT
Classification of ASD
2 / 13
Motivation
According to Centers for Disease Control and Prevention (CDC) [1] in the
USA,
▶ the number of children diagnosed with Autism Spectrum Disorder (ASD)
increased by 340% within 18 years
▶ ASD can be reliably diagnosed by age 2, but children may be diagnosed at
earlier ages.
▶ Early identification of ASD helps children get the services they need.
There is no cure for ASD!
Department of CSE, IUT
Classification of ASD
3 / 13
Motivation (cont.)
This field of research still remains challenging due to the facts that
▶ The outcomes must be medically verified
▶ Various modality of data available for research
1. Autism Brain Imaging Data Exchange (ABIDE) [2] - Functional Magnetic
Resonance Imaging (f-MRI) dataset
2. Saliency4ASD grand challenge [3] - Eye Tracking Scan Paths dataset
3. Autistic Spectrum Disorder Screening Data for Children [4] - Autism
Spectrum Quotient (AQ-10) questionnaire dataset
4. Facial images of children with and without Autism [5] - Facial image
Dataset
▶ Similarity in symptoms of syndromic developmental disorders (SDD)
▶ Socio-cultural differences contribute heavily in early detection
Department of CSE, IUT
Classification of ASD
4 / 13
Literature Review
1. T. Akter et al., “Improved transfer-learning-based facial recognition framework to detect
autistic children at an early stage,” Brain Sciences 2021, Vol. 11, Page 734, vol. 11, p. 734, May
2021. [6]
▶ Justified the use of an improved MobileNet-V1 model on the facial images dataset
▶ Achieved an accuracy of 90.67% and 9.33% of fall-out and miss rate.
▶ Carried out benchmark experiments against several machine learning (ML) classifiers and
improved convolutional neural network (CNN) models
Figure 1: Improved MobileNet-V1 Transfer Learning Model
Department of CSE, IUT
Classification of ASD
5 / 13
Literature Review (cont.)
2. N. Dominic et al., “Transfer learning using inception-resnet-v2 model to the augmented
neuroimages data for autism spectrum disorder classification,” Commun. Math. Biol. Neurosci.,
vol. 2021, p. Article ID 39, 2021. [7]
▶ Justified the use of data augmentation techniques by utilizing a InceptionResNetV2 model.
▶ Achieved an accuracy of 57.6% on the 2 dimensional f-MRI images dataset.
▶ Carried out benchmark experiments for patients of only one site of 17 possible sites of the
original dataset
Figure 2: InceptionResNetV2 architecture
Department of CSE, IUT
Classification of ASD
6 / 13
Literature Review (cont.)
3. M. I. Al-Hiyali et al. “Classification of bold fmri signals using wavelet transform and transfer
learning for detection of autism spectrum disorder,” Proceedings - 2020 IEEE EMBS Conference
on Biomedical Engineering and Sciences, IECBES 2020, pp. 94–98, March 2021 [8]
▶ Extracted blood-oxygen-level-dependent (BOLD) signals and converted to scalogram images
▶ Achieved an accuracy of 85.9% on a sample size of only 82 patients.
▶ Carried out experiments using pre-trained Googlenet, DenseNet201, Resnet18, and Resnet101
Figure 3: General methodology for classification of resting-state BOLD fMRI signals
using wavelet transform and pre-trained CNNs
Department of CSE, IUT
Classification of ASD
7 / 13
Literature Review (cont.)
4. J. Han et al., “A multimodal approach for identifying autism spectrum disorders in children,”
IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 2003–2011,
2022. [9]
▶ Used multi-modal data containing electroencephalogram (EEG) and eye-tracking (ET)
features
▶ Justified the use of a two-step multimodal feature learning and fusion model, multimodal
stacked denoising autoencoder (MMSDAE)
▶ Carried out experiments for different modalities separately and fused together
Figure 4: Multimodal identification framework of children with ASD by fusion of
multimodal EEG and ET data
Department of CSE, IUT
Classification of ASD
8 / 13
Research Scope
After reviewing the state-of-art literature carefully, a handful of scopes of
research can be identified. The potential research scopes are as follows:
▶ None of the approaches have yet explored the possibility of using
multi-modal data with transfer leaning based approaches.
▶ Transfer learning can take advantage of the generalization of a previous
task modeled with one modality of the dataset.
▶ Eventually, while modeling the next modality this generalization can be
used to our advantage.
Hence, multi-modal transfer learning based approach might be a very good
choice for classification of ASD.
Department of CSE, IUT
Classification of ASD
9 / 13
Research Objectives
The objectives with specific aims of this research are as follows:
▶ To develop a robust and efficient method for exploiting generalization in
different models with transfer learning for ASD detection.
▶ To explore the benefits of using multi-modal data.
▶ To analyze the comparative performance against the different existing
standard methods with proposed methodology.
Department of CSE, IUT
Classification of ASD
10 / 13
Possible Outcomes
The possible outcomes of this research can be coined as below:
▶ The proposed model would contribute in early detection of ASD using an
improvised transfer learning based approach
▶ Starting an early intervention/treatment plan for them, would become
more convenient for both parents and care-givers
Department of CSE, IUT
Classification of ASD
11 / 13
Outline of Research Methodology
1. Multi-modal Dataset Acquisition
2. Model Design with a transfer learning based approach
3. Pre-training, Re-Use and Fine Tuning model
4. Prediction and Performance Evaluation
Department of CSE, IUT
Classification of ASD
12 / 13
References
[1] “Data & statistics on autism spectrum disorder,” March 2022. [Online]. Available:
https://www.cdc.gov/ncbddd/autism/data.html. [Accessed on: 04/03/2022]
[2] C. Cameron, B. Yassine, C. Carlton, C. Francois, E. Alan, J. András, K. Budhachandra, L. John, L. Qingyang,
M. Michael, Y. Chaogan, and B. Pierre, “The neuro bureau preprocessing initiative: open sharing of
preprocessed neuroimaging data and derivatives,” Frontiers in Neuroinformatics, vol. 7, 2013.
[3] J. Gutiérrez, Z. Che, G. Zhai, and P. L. Callet, “Saliency4asd: Challenge, dataset and tools for visual attention
modeling for autism spectrum disorder,” Signal Processing: Image Communication, vol. 92, p. 116092, March
2021.
[4] T. F. Fayez, “Autistic spectrum disorder screening data for children data set,” 2017. [Online]. Available:
https://www.kaggle.com/datasets/fabdelja/autism-screening-for-toddlers. [Accessed on: 04/03/2022]
[5] G. Piosenka, “Detect autism from a facial image,” January 2020. [Online]. Available:
https://www.kaggle.com/discussions/general/123978#707551. [Accessed on: 04/03/2022]
[6] T. Akter, M. H. Ali, M. I. Khan, M. S. Satu, M. J. Uddin, S. A. Alyami, S. Ali, A. K. Azad, and M. A. Moni,
“Improved transfer-learning-based facial recognition framework to detect autistic children at an early stage,”
Brain Sciences 2021, Vol. 11, Page 734, vol. 11, p. 734, May 2021. [Online]. Available:
https://www.mdpi.com/2076-3425/11/6/734
[7] N. Dominic, Daniel, T. W. Cenggoro, A. Budiarto, and B. Pardamean, “Transfer learning using
inception-resnet-v2 model to the augmented neuroimages data for autism spectrum disorder classification,”
Commun. Math. Biol. Neurosci., vol. 2021, p. Article ID 39, 2021. [Online]. Available:
http://www.scik.org/index.php/cmbn/article/view/5565
[8] M. I. Al-Hiyali, N. Yahya, I. Faye, Z. Khan, and K. A. Laboratoire, “Classification of bold fmri signals using
wavelet transform and transfer learning for detection of autism spectrum disorder,” Proceedings - 2020 IEEE
EMBS Conference on Biomedical Engineering and Sciences, IECBES 2020, pp. 94–98, March 2021.
[9] J. Han, G. Jiang, G. Ouyang, and X. Li, “A multimodal approach for identifying autism spectrum disorders in
children,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 2003–2011, 2022.
[Online]. Available: https://ieeexplore.ieee.org/document/9832930/
Department of CSE, IUT
Classification of ASD
13 / 13