MACHINE LEARNING
TECHNIQUES
(KCS 055)
Pragya Gupta, JSS academy of technical education, Noida
Unit 1: Introduction to MLT
Syllabus:
INTRODUCTION – Learning, Types of Learning, Well defined learning
problems, Designing a Learning System, History of ML, Introduction of
Machine Learning Approaches – (Artificial Neural Network, Clustering,
Reinforcement Learning, Decision Tree Learning, Bayesian networks,
Support Vector Machine, Genetic Algorithm), Issues in Machine
Learning and Data Science Vs Machine Learning;
Pragya Gupta, JSS academy of technical education, Noida
Unit-1
In this Presentation, we will discuss:
• Issues in Machine learning
• Data Science Vs Machine Learning
• Introduction of Machine Learning Approaches
Pragya Gupta, JSS academy of technical education, Noida
Issues in Machine learning
1. Data Quality and Quantity:
• Insufficient Data: Many machine learning algorithms require large amounts of
high-quality data to generalize well. In many real-world scenarios, obtaining
sufficient data can be a significant challenge.
• Biased Data: Biases in training data can lead to biased models, reinforcing
existing social, gender, or racial biases.
2. Scalability and Efficiency:
• Training complex models can be computationally expensive and time
consuming. Scalability is a concern when dealing with large datasets or
deploying models in real-time applications with resource constraints.
Pragya Gupta, JSS academy of technical education, Noida
Issues in Machine learning
3. Ethical Considerations:
Machine learning applications raise ethical questions regarding privacy,
surveillance, and the potential for automation to displace jobs. It's essential to
address these ethical concerns when developing and deploying ML systems.
4. Data Privacy:
Handling and storing sensitive data for training models can pose privacy risks.
Techniques like differential privacy and federated learning aim to mitigate these
concerns.
5. Lack of Transparency:
The inner workings of some machine learning algorithms are complex and not
well-understood by the general public. Lack of transparency can lead to mistrust
and misconceptions about AI.
Pragya Gupta, JSS academy of technical education, Noida
Issues in Machine learning
6. Robustness and Reliability:
•Models can perform well under normal conditions but fail when faced with
unexpected inputs or adversarial scenarios. Ensuring the robustness and reliability
of models is an ongoing challenge.
7. Regulatory and Legal Issues:
•The deployment of machine learning models may be subject to legal and
regulatory constraints, especially in sectors like healthcare and finance.
8. Environmental Impact:
•Training large deep learning models requires substantial computational resources,
contributing to the carbon footprint of AI research and applications.
Pragya Gupta, JSS academy of technical education, Noida
Data Science Vs Machine Learning
Parameters
Data Science
Machine Learning
Definition and
scope
• Data science is a broader field that • Machine learning is a subset of data
encompasses various aspects of data,
science that focuses specifically on
including data collection, cleaning,
the development of algorithms that
exploration, visualization, analysis,
can learn patterns and make
and interpretation.
predictions from data.
Objective
• The primary goal of data science is to • The primary goal of machine learning
extract knowledge and insights from
is to build predictive models that can
data to support decision-making and
make
accurate
predictions
or
solve complex problems.
decisions
without
explicit
programming.
• Machine learning algorithms aim to
find patterns and relationships in data
that can be used for future
predictions.
Pragya Gupta, JSS academy of technical education, Noida
Data Science Vs Machine Learning
Parameters
Skills
Data Science
Machine Learning
• Data scientists require a diverse skill • Machine learning engineers and
set, including proficiency in statistics,
practitioners need strong programming
data wrangling, data visualization,
skills, knowledge of various machine
domain knowledge, and business
learning algorithms, and expertise in
acumen.
model evaluation and optimization.
• They may use various tools and • They often work with libraries and
programming languages like Python,
frameworks like TensorFlow, scikitSQL, and data visualization libraries
learn, and PyTorch.
• Data science is applied in a wide range • Machine learning is used for tasks like
of domains, including business
image and speech recognition, natural
analytics,
finance,
healthcare,
language processing, recommendation
marketing, and social sciences.
systems, autonomous vehicles, and
• It's used for tasks like customer
anomaly detection.
segmentation, fraud detection, demand
forecasting,
and
recommendation
Pragya Gupta, JSS academy of technical education, Noida
systems.
Applications
Data Science Vs Machine Learning
Parameters
Real life
Example
Data Science
Machine Learning
• Example: Netflix uses Data Science • Example: Facebook uses Machine
technology.
Learning technology.
Pragya Gupta, JSS academy of technical education, Noida
Data Science Vs Machine Learning
• In summary, while data science encompasses a broader range of
activities related to data, including descriptive and diagnostic
analytics, machine learning is a specific subset of data science focused
on building predictive models.
• Data science serves as the foundation for machine learning, providing
the data and insights needed to develop and train machine learning
models.
• Both fields play essential roles in extracting value from data and
driving data-driven decision-making in various industries.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
The various Machine learning approaches or techniques are as
follows:
• Artificial Neural Network,
• Clustering,
• Reinforcement Learning,
• Decision Tree Learning,
• Bayesian networks,
• Support Vector Machine,
• Genetic Algorithm
Pragya Gupta, JSS academy of technical education, Noida
Artificial Neural Network
Pragya Gupta, BBPS Brij Vihar
Introduction of Machine Learning Approaches
Artificial Neural Network:
• An artificial neural network (ANN),
often referred to simply as a neural
network, is a computational model
inspired by the structure and function
of the human brain.
• Neural networks are a subset of
machine learning models that have
gained tremendous popularity due to
their ability to learn complex patterns
and make predictions from data.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
Basic Concepts:
1. Neurons (Artificial Neurons):
1. The fundamental building blocks of neural networks are artificial neurons,
also known as perceptron's or nodes.
2. Each neuron processes input data, performs a weighted sum of inputs, applies
an activation function, and produces an output.
2. Layers:
1. Neurons are organized into layers within a neural network.
2. The input layer receives raw data, while one or more hidden layers process
information, and the output layer produces the final predictions or outputs.
3. Networks with one or more hidden layers are referred to as "deep" neural
networks.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
3. Weights and Biases:
•Each connection (synapse) between neurons has an associated weight that
determines the strength of the connection.
•Neurons also have biases that allow them to learn and adapt.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
How Neural network works:
1. Forward Propagation:
•During forward propagation, data is fed into the input layer, and the neural network
processes the information layer by layer.
•Each neuron computes a weighted sum of its inputs, adds a bias term, and passes
the result through an activation function.
•The output of the final layer represents the network's prediction.
2.Activation Functions:
•Activation functions introduce non-linearity into the model, allowing neural
networks to learn complex relationships in data.
•Common activation functions include ReLU (Rectified Linear Unit), Sigmoid
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
How Neural network works:
3. Loss Function:
•A loss function measures the difference between the network's predictions and the
true target values.
•The goal during training is to minimize this loss by adjusting the weights and
biases of the network.
4. Backpropagation:
•Backpropagation is the process of computing gradients of the loss with respect to
the model's parameters (weights and biases) and using these gradients to update the
parameters via optimization algorithms like gradient descent.
•This iterative process helps the network learn and improve its predictions.
Pragya Gupta, JSS academy of technical education, Noida
Clustering
Introduction of Machine Learning Approaches
Introduction to Clustering:
• Clustering is a technique in machine learning and data analysis that involves
grouping similar data points together based on certain features or characteristics.
• Clustering is the task of dividing the population or data points into a number of
groups such that data points in the same groups are more similar to other data
points in the same group and dissimilar to the data points in other groups.
• It is basically a type of unsupervised learning method. Generally, it is used as a
process to find meaningful structure, explanatory underlying processes,
generative features, and groupings inherent in a set of examples.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
For example The data points in the graph below clustered together can be
classified into one single group. We can distinguish the clusters, and we can
identify that there are 3 clusters in the below picture.
Data points within the same cluster should be more similar to each other than to those in different clusters.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
Applications:
Clustering has numerous real-world applications, including:
• Earthquake studies: By learning the earthquake-affected areas we can
determine the dangerous zones.
• Image Processing: Clustering can be used to group similar images together,
classify images based on content, and identify patterns in image data.
• Cybersecurity: Clustering is used to group similar patterns of network
traffic or system behavior, which can help in detecting and preventing
cyberattacks.
• Marketing: It can be used to characterize & discover customer segments for
marketing purposes.
• Social network analysis: Clustering is used to identify communities or
groups within social networks, which can help in understanding social
behavior, influence, and trends.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
Various Algorithms:
There are various clustering algorithms, each with its own approach and
characteristics.
Some popular clustering algorithms include:
•K-Means: It partitions data into a predefined number of clusters, aiming to
minimize the sum of squared distances between data points and the centroids of
their respective clusters.
•Hierarchical Clustering: This approach creates a tree-like structure of clusters,
allowing for both divisive (top-down) and agglomerative (bottom-up) clustering.
•DBSCAN: It identifies clusters as dense regions separated by sparser areas,
making it suitable for datasets with irregular shapes and varying cluster densities.
•Gaussian Mixture Models (GMM): GMM assumes that the data points are
generated from a mixture of Gaussian distributions and models clusters based on
these distributions.
Pragya Gupta, JSS academy of technical education, Noida
K-means clustering algorithm
• K-Means Clustering is an Unsupervised Learning algorithm, which groups the
unlabeled dataset into different clusters.
• Here K defines the number of pre-defined clusters that need to be created in
the process, as if K=2, there will be two clusters, and for K=3, there will be
three clusters, and so on.
• It is an iterative algorithm that divides the unlabeled dataset into k different
clusters in such a way that each dataset belongs only one group that has
similar properties.
• It is a centroid-based algorithm, where each cluster is associated with a
centroid. The main aim of this algorithm is to minimize the sum of distances
between the data point and their corresponding clusters.
Pragya Gupta, JSS academy of technical education, Noida
K-means clustering algorithm
• The
k-means
clustering
algorithm mainly performs two
tasks:
•Determines the best value for
K center points or centroids by
an iterative process.
•Assigns each data point to its
closest k-center. Those data
points which are near to the
particular k-center, create a
cluster.
Pragya Gupta, JSS academy of technical education, Noida
K-means clustering algorithm
• How does the K-Means Algorithm Work?
Step-1: Select the number K to decide the number of clusters.
Step-2: Select random K points or centroids.
Step-3: Now for each data point calculate the Euclidean distance from the
predefined cluster centroids. Thereafter, assign each data point to their closest
centroid (or cluster)
𝑬𝒅 =
𝑿𝟎 − 𝑿𝑪 𝟐 + (𝒀𝟎 − 𝒀𝑪 )𝟐
Step-4: Calculate a new centroid of each cluster.
Step-5: Repeat the third steps, which means reassign each datapoint to the new
closest centroid of each cluster.
Step-6: If any reassignment occurs, then go to step-4 else go to FINISH.
Step-7: The model is ready.
Pragya Gupta, JSS academy of technical education, Noida
K-means clustering algorithm
Example: For the given data points, use K-means clustering algorithm to form a cluster.
Data
points
Height (in cm)
Weight
1
185
72
2
170
56
3
168
60
4
179
68
5
182
72
6
188
77
7
180
71
8
180
70
9
183
84
10
180
88
11
180
67
12 Pragya Gupta, JSS 177
76Noida
academy of technical education,
Assuming K=2
• Randomly we can form
𝐾1 𝑎𝑛𝑑 𝐾 2 with centroid points
(185,72) and (170,56)
2
cluster
• Now for 3rd data point: 𝑋0 = 168, 𝑌0 =
60, calculate Euclidean distance with respect
to two centroid point K1 and K2
𝑬𝒅 =
𝑿𝟎 − 𝑿𝑪
𝟐
+ (𝒀𝟎 − 𝒀𝑪 )𝟐
• The data point will lie in that cluster which
will have minimum 𝐸𝑑
K-means clustering algorithm
Example: For the given data points, use K-means clustering algorithm to form a cluster.
Data
points
Height (in cm)
Weight
1
185
72
2
170
56
3
168
60
4
179
68
5
182
72
6
188
77
7
180
71
8
180
70
9
183
84
10
180
88
11
180
67
12 Pragya Gupta, JSS 177
76Noida
academy of technical education,
• Also calculate the new centroid point by
taking the average of previous point and new
data point.
• Repeat the same with all data points, and
finally each data point will be mapped inside
the selected two clusters.
Solution
• 𝐾1 = 1,4,5,6,7,7,9,10,11,12
𝐾2 = {2,3}
Decision Tree Learning
Pragya Gupta, BBPS Brij Vihar
Introduction of Machine Learning Approaches
• A decision tree is a supervised learning algorithm for classification and
regression tasks.
• It has a hierarchical tree structure consisting of a root node, branches, internal
nodes, and leaf nodes.
• A decision tree is a hierarchical model used in decision support that depicts
decisions and their potential outcomes, incorporating chance events, resource
expenses, and utility.
• The tree structure is comprised of a root node, branches, internal nodes, and
leaf nodes, forming a hierarchical, tree-like structure
• It starts with a root node and ends with a decision made by leaves.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
•Root Node: The initial node at the beginning of a
decision tree, where the entire population or dataset
starts dividing based on various features or conditions.
•Decision Nodes: Nodes resulting from the splitting of
root nodes are known as decision nodes. These nodes
represent intermediate decisions or conditions within the
tree.
•Leaf Nodes: Nodes where further splitting is not
possible, often indicating the final classification or
outcome. Leaf nodes are also referred to as terminal
nodes.
•Sub-Tree: Similar to a subsection of a graph being
called a sub-graph, a sub-section of a decision tree is
referred to as a sub-tree. It represents a specific portion
of the decision
tree.
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
•Let’s understand decision trees with the help of an example:
•Day wise weather, temperature, humidity, wind is given, and decision need to be taken if
Tennis match can be played or not
Pragya Gupta, JSS academy of technical education, Noida
Introduction of Machine Learning Approaches
• Decision trees are upside down which means
the root is at the top and then this root is split
into various several nodes.
• Decision trees are nothing but a bunch of if-else
statements in layman terms. It checks if the
condition is true and if it is then it goes to the
next node attached to that decision.
• In the figure, the tree will first ask what is the
weather? Is it sunny, cloudy, or rainy? If yes
then it will go to the next feature which is
humidity and wind. It will again check if there
is a strong wind or weak, if it’s a weak wind
and it’s rainy then the person may go and play.
Pragya Gupta, JSS academy of technical education, Noida
Unit 2
Syllabus:
REGRESSION: Linear Regression and Logistic Regression
BAYESIAN LEARNING - Bayes theorem, Concept learning, Bayes Optimal
Classifier, Naïve Bayes classifier, Bayesian belief networks, EM algorithm.
SUPPORT VECTOR MACHINE: Introduction, Types of support vector kernel –
(Linear kernel, polynomial kernel, and Gaussian kernel), Hyperplane – (Decision
surface), Properties of SVM, and Issues in SVM.
Pragya Gupta, JSS academy of technical education, Noida
Regression
• Regression in machine learning (ML) is a supervised learning task that deals
with predicting a continuous or numerical output variable (dependent
variable) based on one or more input variables (independent variables or
features).
• The primary goal of regression is to model the relationship between the input
variables and the output variable so that you can make accurate predictions or
infer the strength and direction of these relationships.
Pragya Gupta, JSS academy of technical education, Noida
Linear Regression and Logistic Regression
Linear Regression
Logistic Regression
Linear Regression is a supervised regression Logistic Regression is a supervised classification
model.
model.
Equation of linear regression:
𝒚 = 𝒂𝟎 + 𝒂𝟏 𝒙𝟏 + 𝒂𝟐 𝒙𝟐 + … + 𝒂𝒊 𝒙𝒊
Here,
y = response variable
xi = ith predictor variable
ai = average effect on y as xi increases by 1
Equation of logistic regression
y(x) =
𝒆 𝒂𝟎 + 𝒂𝟏𝒙𝟏 + 𝒂𝟐𝒙𝟐 + … + 𝒂𝒊𝒙𝒊
𝟏 + 𝒆 𝒂𝟎 + 𝒂𝟏𝒙𝟏 + 𝒂𝟐𝒙𝟐 + … + 𝒂𝒊𝒙𝒊
Here,
y = response variable
xi = ith predictor variable
ai = average effect on y as xi increases by 1
In Linear Regression, we predict the value by an In Logistic Regression, we predict the value by 1
integer number.
or 0.
Here we calculate Root Mean Square Here we use precision to predict the next weight
Pragya Gupta, JSS academy of technical education, Noida
Error(RMSE)
to predict the next weight value.
value.
Linear Regression and Logistic Regression
Linear Regression
Applications of linear regression:
• Financial risk assessment
• Business insights
• Market analysis
Pragya Gupta, JSS academy of technical education, Noida
Logistic Regression
Applications of logistic regression:
• Medicine
• Credit scoring
• Hotel Booking
• Gaming
• Text editing