Introduction to Neural
Networks
Dev Mittal B20MT016
What is a Neural Network?
Definition:
●
●
A neural network is an information processing system inspired
by the way biological nervous systems, such as the brain,
process information.
It consists of interconnected processing elements called
neurons.
Purpose:
●
Used for pattern recognition, classification, regression, and
decision-making tasks.
Basic Structure of a Neural Network
Layers:
○
○
○
Input Layer: Receives input data.
Hidden Layers: Intermediate layers that process
inputs.
Output Layer: Produces the final output.
Neurons:
○
Each neuron in a layer receives input, processes it,
and passes the output to the next layer.
How Neural Networks Learn
1. Initialization: Weights and biases are initialized.
2. Forward Propagation: Inputs pass through the network to
produce an output.
3. Loss Calculation: The difference between the predicted output
and actual output is calculated.
4. Backward Propagation: The network adjusts weights and biases
to minimize the loss.
5. Iteration: Steps are repeated for many epochs to improve
accuracy.
Activation Functions
Stochastic Gradient Descent (SGD) is a variant of the Gradient Descent algorithm that is used
for optimizing machine learning models.
In SGD, instead of using the entire dataset for each iteration, only a single random training example
(or a small batch) is selected to calculate the gradient and update the model parameters. This
random selection introduces randomness into the optimization process, hence the term “stochastic”
in stochastic Gradient Descent
To implement stochastic gradient descent in practice you also need an outer loop generating minibatches of training examples, and an outer loop stepping through multiple epochs of training.
Advantages and challenges
Advantages:
● Can model complex relationships.
● Capable of learning and adapting.
Challenges:
● Require large amounts of data.
● Computationally intensive.
● Risk of overfitting.
Applications
Computer Vision: Image and facial recognition.
Natural Language Processing: Machine translation, sentiment analysis.
Healthcare: Disease prediction, medical image analysis.
Finance: Fraud detection, stock market prediction.