Evolution of Deep Neural Networks
Dr. Subrajeet Mohapatra
Assistant Professor
Department of Computer Science & Engineering,
BIT Mesra, Ranchi
Jharkhand
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Automation
•
•
Automation is the use of machines/technology to accomplish a task
with as little human interaction as possible.
Implementation of automation technologies improve the efficiency,
reliability, and/or speed of many tasks that were previously
performed by humans.
Manual Process
Computer Aided Process
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Traditional Computing
Computation is any type of calculation that includes both
arithmetical and non-arithmetical steps which follows a
well defined model. e.g. Algorithm of a well defined
problem.
Input
Output
Characteristics of an algorithm
Finiteness
Definiteness
Unambiguous
Traditional computing can be of three types : Serial
Computing,
Parallel
Computing
and
Distributed
Computing.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Traditional Computing/Computer Automation
Data
Computation
Result
Algorithm
•
•
Refers to applications, platforms, or computer programs that automate
simple, rule-based, repetitive tasks.
Examples : Tablet Manufacturing, Banking, Insurance, Railway
Reservation, Mechanized assembly (Conventional Approaches)
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Intelligent Computing/Cognitive Automation
• Intelligent Computing refers to the ability of a computer/machines
to learn specific task from data or experimental observations.
Data
Computation
Outcome
Knowledge
•
Incorporates Cognitive Computing, Artificial Intelligence (AI), Machine
Learning, Natural Language Processing, Computer Vision in the
automation process (Intelligent Approaches).
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Traditional vs. Intelligent Computing
• Traditional computing (programming) refers to any manually
created program that uses input data and runs on a computer to
produce the output.
• It is a manual process—meaning a person (programmer) creates
the program.
• Without manually formulating or code rules (programming) the
logic cannot be updated.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Traditional vs. Intelligent Computing
• In intelligent computing (Machine Learning) the input data and
output are fed to an algorithm to create a program which can be
used to predict future outcomes.
• In machine learning (intelligent computing), on the other hand, the
algorithm automatically formulates the rules from the data.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Trends in Intelligent Computing
•
•
•
•
•
•
Artificial Intelligence
Soft Computing
Machine Learning
Deep Learning
Data Science
Cloud Computing
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Trends in Intelligent Computing
• Artificial Intelligence (AI) :• The study of the modeling of human mental functions by
computer programs.
• Any code, technique or algorithm that enable machines to
mimic, develop or demonstrate the human cognition or
behavior is AI.
• Machine Learning (ML) :• Machine learning is the science of getting computers to
act without being explicitly programmed.
• ML is a subset of AI which uses statistical methods to
enable machines to improve with experience.
• Algorithms based on ML are designed in such a way that
they can learn and improve over time when exposed to
new data.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Trends in Intelligent Computing
• Deep Learning (DL) :• DL is a subfield of machine learning concerned with
algorithms inspired by the structure and function of the
brain called artificial neural networks.
• Data Science (DS) :• Data science is a concept used to tackle big data and
includes data cleansing, preparation, and analysis.
•
A data scientist gathers data from multiple sources and
applies machine learning, predictive analytics, and
sentiment analysis to extract critical information from the
collected data sets.
• They understand data from a business point of view and
can provide accurate predictions and insights that can be
used to power critical business decisions.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Cousins of Artificial Intelligence
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Artificial Intelligence vs. Machine Learning
• AI and ML are often used interchangeably, especially in the
realm of big data (But these aren’t the same thing).
• AI is a broader concept than ML, which addresses the use of
computers to mimic the cognitive functions of humans.
• When machines carry out tasks based on algorithms in an
“intelligent” manner, that is AI.
• Whereas ML is a subset of AI and focuses on the ability of
machines to receive a set of data and learn for themselves,
adapting algorithms as they learn more about the information
they are processing.
• Training computers to think like humans is achieved partly
through the use of artificial neural networks along with
learning algorithms.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning vs. Deep Learning
• DL goes yet another level deeper and can be considered a
subset of machine learning.
13
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning vs. Deep Learning
• It is not a secret that deep learning outperforms other
machine learning algorithms.
• Insights from the graph
Conclusion
Conclusion
Conclusion
Conclusion
0:
1:
2:
3:
AI products need data.
the more data we have—the smarter AI will be.
industry giants have much more data than others.
quality gap in AI products is defined by amount of data.
• Neural network architecture can strongly influence the
performance of an AI system, yet the amount of training data has
biggest impact on performance.
• Companies which focused on data gathering provide better AI
products and are hugely successful.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning vs. Deep Learning
Common mistake: AI is all about building neural nets.
15
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning vs. Deep Learning
• When people think about AI they think about the coding of
DL algorithms, but they should also think about the data.
• Algorithms are free: Google and other giants tend to share
their state-of-the-art research platform with the world, but
what they don’t--they don’t share data.
• Lots of people have jumped on AI hype train and created
awesome tools to build and train neural networks, but very
few focus on training data.
• When companies try to apply AI they have all the tools to
train neural networks but no tools to develop training data.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning vs. Deep Learning
Andrew Ng Says Enough Papers, Let’s Build AI Now!
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Evolution of Deep Neural Network
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Evolution of Deep Neural Network
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Introduction to Machine Learning
• Machine learning uses data and produces a model to perform a
task
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning : A problem specific
overview
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning : A problem specific
overview
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning : A problem specific
overview
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Data Collection
• The performance of predictive model depends upon
• Amount of data
• Quality of data
• Learning algorithm
• Sources of machine learning dataset sources
•
•
•
•
•
Google’s datasets search engine
Government (.gov) datasets
Kaggle Project
Registry of Open Data on AWS
UCI Machine Learning Repository
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Data Preparation/ Data Preprocessing
• Process of transforming raw data for making it suitable for
applying machine learning model
• The preprocessing process incorporates :
• Feature Engineering
• Data Cleaning
• Data Integration
• Data Reduction
• Data Transformation.
•
It also includes
Split data into training
and validation sets, and
keep some data
available for testing the
model
• Dealing with missing values
• Dealing with outliers
• Dealing with imbalanced data
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Choosing a Predictive Model
Selecting a suitable learning algorithm/ ML model is essential for
problem solving
• The choice of predictive model is based on the
• Type of problem (Classification, Clustering and Regression)
• Nature, type and size of data
•
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Factors to consider for choosing a ML model
• The toughest part of solving a machine learning problem can be
finding the right estimator for the job.
• There can be several options for the selection of a predictive
model, and can be some of the following
•
How does your target variable look like?
•
Is computational performance an issue
•
Does my dataset fit into memory?
•
Is my data linearly separable?
•
Finding a good bias variance threshold.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Factors to consider for choosing a ML model
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Training/Learning of the Model
• The objective of training (learning) is to find a model and its
corresponding parameters such that the resulting model perform
well on unseen data
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Supervised Learning of the Model
• The learning here is performed with the help of a teacher (domain
expert).
• Supervised learning is feasible if there exists statistical data (InputOutput mapping).
X
(Input)
Neural
Network
(Weight W)
(D-Y)
Error Signal
Y
(Actual Output)
Error
Signal
Generator
Application of Machine Learning in Drug Discovery & Development
D
(Desired
Output)
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Unsupervised Learning of the Model
• The learning here is performed without the help of a teacher
(domain expert).
• Unsupervised learning is performed if statistical data (Input-Output
mapping) is unavailable.
• The input vectors of similar type are grouped without the use of
training data.
X
(Input)
Neural
Network
(Weight W)
Application of Machine Learning in Drug Discovery & Development
Y
(Actual
Output)
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Evaluation/Testing the Model
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Hyper parameter Tuning
• Hyper parameter is a parameter whose value is used to control
the learning process in the predictive model
• The problem of choosing a set of optimal hyper parameters for a
learning algorithm is known as hyper parameter tuning.
• Its an experimental process (hit and trial approach)
• Art rather than science
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Hyper parameter Tuning
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Steps in Machine Learning
Prediction
• At this stage we have our trained model, right model parameters
• We can use trained model for predicting unseen points
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Traditional ML vs. Deep Learning
• Data dependencies
• When the data is small, deep learning algorithms don’t perform
that well.
• Traditional ML algorithms with their handcrafted rules prevail
in this scenario.
• Hardware dependencies :• DL algorithms heavily depend on high-end machines (GPU),
contrary to traditional machine learning algorithms, which can
work on low-end machines.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Traditional ML vs. Deep Learning
• Feature Engineering
• The performance of most of the ML algorithm depends on how
accurately the features are identified and extracted.
• DL algorithms try to learn high--level features from the data.
• DL reduces the task of developing new feature extractor for every
problem.
• Problem solving approach
• In ML, we break the problem down into different parts, solve
them individually and combine them to get the result.
• DL in contrast advocates to solve the problem end-to-end.
• Execution time :- ML comparatively takes much less time to train
in comparison to DL.
• Interpretability :- ML algorithms are more preferred choice where
interpretability is essential.
38
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Traditional ML vs. Deep Learning
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Feature Engineering
• Feature engineering is the process of using domain
knowledge of the data to create features that make ML
algorithms work.
• In traditional ML algorithms, we need to hand--craft the
features.
• In DL algorithms feature engineering is done automatically
by the algorithm.
• Feature engineering is difficult, time-consuming and
requires domain expertise.
• The promise of DL is more accurate compared to traditional
ML with less or no feature engineering.
40
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Feature Engineering
Traditional Machine Learning Flow
Consumes lots of time
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Feature Engineering
Deep Learning Flow
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Feature Engineering
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Deep Learning Architecture
• A deep neural network consist of hierarchy of layers,
whereby each layer transforms the input data into more
abstract representations.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Deep Learning Architecture
• Increasing the number of layers allows an increasing the
number of neurons.
• The deeper the network the better the learning of features.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Types of Deep Neural Networks (Selected)
•
•
•
•
•
•
•
•
•
Deep Feed Forward Networks
Deep Belief Networks
Recurrent Neural Networks
Convolutional Neural Networks
Autoencoders
Generative Adversarial Networks
Boltzmann Machines
Capsule Network
Liquid State Machine
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Neural Networks Zoo
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Neural Networks Zoo
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Deep Learning Frameworks
• Why Deep Learning is so good?
• It is simply great in terms of accuracy when trained with a
huge amount of data.
• Plays a significant role to fill the gap when a scenario is
challenging for the human brain.
So, quite logical this contributed to a whole slew of new
frameworks appearing.
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Machine Learning and Deep Learning Frameworks
• Machine learning (ML) and Deep Learning (DL) frameworks are
interfaces that allow data scientists and developers to build and
deploy machine learning models faster and easier.
• Different frameworks for ML and DL
• Python
• MATLAB
• R
50
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra
Thank You
Application of Machine Learning in Drug Discovery & Development
Subrajeet Mohapatra
Department of CSE, BIT Mesra