FOUNDATIONS OF ARTIFICIAL
INTELLIGENCE
A Comprehensive Self-Study Module for Engineering Students
1. Introduction to Artificial Intelligence
Artificial Intelligence (AI) represents the simulation of human intelligence processes by machines,
especially computer systems. These processes include learning (the acquisition of information and
rules for using the information), reasoning (using rules to reach approximate or definite conclusions),
and self-correction. Historically, AI has evolved from symbolic processing and expert systems to
modern data-driven paradigms.
Core Paradigm Shift
Traditional programming relies on explicit hard-coded rules and inputs to generate outputs. In
contrast, modern Machine Learning flips this dynamic: data and known outcomes are fed into the
system to discover the underlying mathematical mappings automatically.
2. Mathematical Foundations of Machine Learning
At the core of machine learning algorithms are foundational concepts from linear algebra, calculus,
and probability theory. Understanding how optimization functions minimize error metrics is critical
to mastering model training dynamics.
Linear Regression & Cost Minimization
Consider a simple linear relationship where the goal is to predict a continuous target variable y based
on an input feature vector x. The hypothesis model is formally framed as:
hθ(x) = θ0 + θ1x
To evaluate the inaccuracies or deviations of our model assumptions across a dataset containing m
discrete training samples, we employ the Mean Squared Error (MSE) cost function, commonly
denoted as J(θ):
J(θ0, θ1) = 1/(2m) × Σi=1m (hθ(x(i)) − y(i))2
Optimization algorithms such as Gradient Descent iteratively adjust parameters θj in the direction of
steepest descent, proportional to the negative gradient step controlled by the learning rate parameter
α:
Department of Computer Science & Engineering • Self-Study Series
Page 1 of 2
θj := θj − α × ∂/∂θj J(θ0, θ1)
3. Categorization of AI Frameworks
Modern algorithmic architectures are generally categorized based on the nature of the learning
feedback loop and the structure of training instances available.
Learning Paradigm
Core Architecture
Primary Applications
Supervised
Labeled training data mapping features to
Image Classification, Fraud Detection, House
Learning
distinct target values.
Price Forecasting.
Unsupervised
Unlabeled data parsed to find hidden
Customer Segmentation, Anomaly Detection,
Learning
underlying structures or clusters.
Dimensionality Reduction.
Reinforcement
Agent learning through environment trial-
Robotics Control Systems, Autonomous
Learning
and-error interactions via rewards.
Vehicles, Complex Strategy Games.
4. Deep Learning & Neural Infrastructures
Deep Learning expands standard machine learning methodologies by utilizing layered artificial
neural networks. These deep layers automatically compute hierarchical feature abstractions,
removing the manual task of engineering handcrafted domain features.
Modern paradigms have transitioned from standard Deep Neural Networks (DNNs) toward highly
contextual models, including Convolutional Neural Networks (CNNs) optimized for spatial arrays, and
Transformer networks that utilize self-attention mechanisms to process sequential texts and complex
tokens efficiently.
SELF-STUDY CHECKPOINT & EXERCISE
Objective: Verify your intuitive grasp of optimization rates and cost gradients.
1. Explain what happens mathematically to parameter updates when the learning rate (α) is
configured excessively high versus infinitely small.
2. Differentiate between standard batch Gradient Descent and Stochastic Gradient Descent (SGD)
in terms of computational overhead and optimization path trajectory.
Department of Computer Science & Engineering • Self-Study Series
Page 2 of 2