INTRODUCTION TO ARTIFICIAL INTELLIGENCE, MACHINE
LEARNING AND CONVERGENT TECHNOLOGIES
Course Code: GET 307
List of Topics(Chapters)
1. Concepts of Human and Artificial Intelligence
2. Artificial/Computational Intelligence Paradigms
3. Search, Logic and Learning Algorithms
4. Machine Learning and Nature-Inspired Algorithms
5. Natural Language Understanding
6. Knowledge Representation and Elicitation
7. Mathematical and Logic Foundations of AI
8. Expert Systems and Automated Reasoning
9. Pattern Recognition
10.Distributed Systems
11.Data and Information Security
12.Intelligent Web Technologies
13.Convergent Technologies
14.Neural Networks and Deep Learning
15. Introduction to Python AI Libraries
1
CHAPTER 1: CONCEPTS OF HUMAN AND ARTIFICIAL INTELLIGENCE
1.2 Introduction to Intelligence
Intelligence is generally defined as the ability to acquire and apply knowledge and skills. It
encompasses a range of cognitive abilities. The Key Components/Facets of Intelligence (Common
to both Human and AI research) includes acquiring new information and skills, drawing
conclusions from facts or premises, finding a solution to a difficult or complex issue, interpreting
sensory information (visual, auditory, etc.) to understand the environment, comprehending and
generating human language (Natural Language Processing - NLP).
1.2 Human Intelligence (HI)
Human Intelligence is the inherent cognitive capacity of humans, supported by the biological
structure of the brain and nervous system. The core characteristics are
 Consciousness and Self-Awareness: The ability to be aware of oneself and one's thoughts.
 Emotional Intelligence: The ability to perceive, evaluate, and manage emotions (in oneself
and others).
 Creativity and Imagination: Generating novel ideas, solutions, or artistic works.
 Adaptability and Generalization: The ability to apply learned knowledge/skills to a wide
variety of new, unstructured, or unexpected situations.
 Abstract Thinking: The capacity to understand concepts that are not tied to physical objects
or events (e.g., morality, justice).
 Moral and Ethical Reasoning: Incorporating values, ethics, and social norms into decisionmaking.
Artificial Intelligence (AI)
John McCarthy in 1956 define AI as "The science and engineering of making intelligent machines,
especially intelligent computer programs." This definition has been expanded considering recent
researches in the field. The Modern Definition of AI is “The capability of a machine to imitate
intelligent human behavior, such as learning, reasoning, problem-solving, perception, and
decision-making”. The Goal is to develop computer systems that can store knowledge and
effectively use that knowledge to solve problems and accomplish tasks.
Brief History of Artificial Intelligence (AI)
The history of AI is a fascinating journey of alternating periods of optimism ("booms") and
disappointment ("winters").
The Conception (Pre-1956)
2
AI concept started as ancient dreams. The idea of creating intelligent artificial beings (like
automata in Greek myths) is ancient:
In 1943 (Formalizing Neurons), Warren McCulloch and Walter Pitts publish a paper proposing a
model of artificial neurons, laying the groundwork for neural networks. Formalizing Neurons
1950 (The Turing Test) Alan Turing publishes his seminal paper, "Computing Machinery and
Intelligence," posing the question, "Can machines think?" and proposing the Turing Test (or
"Imitation Game") as a measure of machine intelligence.
The Birth and The Golden Years (1956–Mid-1970s)
1956: The Dartmouth Workshop: This pivotal summer conference, organized by John McCarthy,
is considered the official birth of AI as an academic field. At the conference, John McCarthy coined
the term "Artificial Intelligence." The attendees held the optimistic belief that "every aspect of
learning or any other feature of intelligence can in principle be so precisely described that a
machine can be made to simulate it."
1957: The Perceptron: Frank Rosenblatt introduces the Perceptron, an early model of a neural
network capable of pattern recognition.
1966: ELIZA: Joseph Weizenbaum develops ELIZA, the first well-known chatbot. It simulated a
psychotherapist using simple pattern matching in Natural Language Processing (NLP).
1969: Shakey the Robot: Developed at SRI, Shakey was the first general-purpose mobile robot
capable of logical reasoning and planning its actions.
The First AI Winter (1974–1980)
The initial over-optimistic predictions failed to materialize. Fundamental limitations in early
symbolic AI (the "common-sense knowledge" problem) and a lack of computational power
(expensive, slow computers) led to reduced government funding and general disappointment.14
The Expert Systems Boom (1980–Mid-1987)
1980s: Expert Systems: AI sees a commercial resurgence with the development of Expert
Systems—programs designed to mimic the knowledge and decision-making ability of a human
expert in a narrow domain (e.g., medical diagnosis, configuring computer systems like XCON).
1986: Back propagation: The back propagation algorithm is successfully applied to train
multilayer neural networks, setting the stage for deep learning years later.
The Second AI Winter (1987–Mid-1990s)
The market for expert systems collapsed due to high maintenance costs and limited adaptability,
leading to another period of decreased funding and interest.
The Machine Learning Era (Mid-1990s–2010)
3
Research moves from rule-based symbolic systems toward statistical and probability-based
approaches (Machine Learning).
1997: Deep Blue: IBM's supercomputer Deep Blue defeats the reigning world chess champion,
Garry Kasparov, showcasing the power of brute-force computation combined with sophisticated
search algorithms.
2006: Deep Learning Re-emerges: Research by Geoffrey Hinton and others revives interest in
neural networks, coin the term "Deep Learning," and demonstrate how to train deeper networks
effectively.
The Deep Learning and Generative AI Revolution (2010–Present)
There are three key drivers for this modern AI
1. Big Data: The internet and cloud computing provide massive datasets for training.
2. GPU Power: Graphics Processing Units (GPUs) provide the parallel processing power needed for
large-scale neural network training.
3. Algorithmic Advances: Deep Learning breakthroughs (e.g., Convolutional Neural Networks,
Transformers).
1.3 Types of AI
There are three main types of AIs
1. Artificial Narrow Intelligence (ANI) / Weak AI: ANI is designed and trained for a single,
specific task (e.g., chess program, virtual assistant like Siri, image recognition). All current
practical AI applications are ANI.
2. Artificial General Intelligence (AGI) / Strong AI: A theoretical machine with the
intellectual ability to understand, learn, and apply its intelligence to solve any problem—
at a human or superhuman level. It would possess consciousness and self-awareness
(Currently only exists in theory/science fiction).
3. Artificial Super Intelligence (ASI): A hypothetical AI that is vastly smarter than the best
human brains in virtually every field, including scientific creativity, general wisdom, and
social skills.
1.4 Core Components/Subfields of Modern AI
4
a.) Machine Learning (ML): A subset of AI where systems learn from data without being explicitly
programmed.
Supervised Learning: Learning from labeled data (input-output pairs).
ii. Unsupervised Learning: Learning from unlabeled data to find hidden patterns.
iii. Reinforcement Learning (RL): Learning through trial-and-error with a system of
rewards and penalties.
i.
b.) Deep Learning (DL): A subset of ML that uses Artificial Neural Networks (ANNs) with
multiple layers (deep networks) to analyze vast, complex, and unstructured data (e.g., in computer
vision, advanced NLP).
c.) Natural Language Processing (NLP): Enables computers to understand, interpret, and generate
human language.
d.) Computer Vision (CV): Enables machines to "see" and interpret visual information from the
world (e.g., image recognition, object tracking).
e.) Robotics: The branch of engineering that involves the design, construction, operation, and
application of robots (often integrating perception, planning, and control from AI).
Comparison of Human Intelligence and Artificial Intelligence
Figure 1.1: Human Intelligence and Artificial Intelligence
5
Table 1.1: Comparison of Human Intelligence and Artificial Intelligence
Parameter
Human Intelligence (HI)
Artificial Intelligence (AI)
Foundation
Biological
(Brain,
Nervous System)
Digital (Algorithms, Data, Hardware)
Learning Style
Experiential,
intuitive,
emotional, generalizable.
Data-driven, algorithmic, specific to
training data.
Speed
Volume
Slower processing, limited
data volume.
Extremely fast processing, virtually
unlimited data volume.
Creativity
High; based on intuition,
abstract thinking, and
emotion. Can create new
concepts.
Limited (Currently); output is based on
patterns in existing data (e.g.,
Generative AI).
DecisionMaking
Subjective; influenced by
emotions, ethics, values,
and context.
Objective; purely based on data analysis
and programmed logic/algorithms.
Adaptability
High; quickly adapts to
new, unexpected, and
unstructured
environments.
Low/Moderate; requires re-training or
reprogramming for significant changes
outside its trained scope (AGI is the
goal for high adaptability).
Energy
&
Maintenance
Requires rest, prone to
fatigue and distraction.
Can operate 24/7 without fatigue;
requires
electricity
and
data
maintenance.
Ethics
Morality
Inherently possesses a
moral
code
and
conscience.
Lacks inherent moral code; follows
programmed ethical guidelines (Focus
of AI Ethics and alignment research).
Excellent; can apply a
concept learned in one
area to a completely
different area.
Poor/Narrow; struggles to apply
knowledge outside its trained domain
(ANI).
&
&
Generalization
6
CHAPTER 2: ARTIFICIAL/COMPUTATIONAL INTELLIGENCE PARADIGMS
2.I. Introduction: The Divide in Intelligence Modeling
Artificial Intelligence (AI) and Computational Intelligence (CI) are broad fields, but the methods
used to achieve "intelligent" behavior generally fall into a few distinct schools of thought, or
Paradigms. These paradigms represent fundamentally different approaches to knowledge
representation, learning, and reasoning. We can broadly categorize the paradigms into two
historical approaches—Symbolic AI and Connectionist AI—and a field derived from natural
optimization called Computational Intelligence (CI).
2.2. The Core AI Paradigms
A. Symbolic AI (The Top-Down Approach) also known as Classical AI, Rule-Based AI, Good
Old-Fashioned AI (GOFAI), Logic-Based AI.
In Symbolic AI, intelligence is achieved by manipulating symbols according to explicit, logical
rules. It attempts to model human high-level cognitive processes like conscious reasoning,
planning, and language. Explicit knowledge is hand-coded into a structured Knowledge Base
using logic statements, rules (If-Then), or semantic networks. It uses formal logic and search
algorithms (Inference Engine) to deduce new facts or determine actions. The reasoning process is
clear, logical, and traceable. It excels in domains with clear, well-defined rules (e.g., math, formal
games, regulations). The key weaknesses is that it struggles with the real-world complexity, noise,
and sheer volume of "common sense" knowledge required and does not easily adapt to new or
unexpected situations not covered by the hand-coded rules. Some areas of applications include
expert Systems (e.g., XCON), Automated Theorem Proving, Early Chess Programs.
B. Connectionist AI (The Bottom-Up Approach) also known as Sub-Symbolic AI, Neural
Networks, Deep Learning.
In connectionist AI, intelligence emerges from interconnected, simple processing units (artificial
neurons) that learn patterns from large amounts of data. It models the structure and function of the
biological brain. The knowledge is implicitly stored as numerical weights and connections across
a network structure. Knowledge is distributed, not explicitly coded. The reasoning follows pattern
recognition, feature extraction, and statistical inference through signal propagation. It main
strength is Learning and Adaptability. It excels at learning complex, non-linear relationships and
adapting to new data. It is highly effective in handling noisy, high-dimensional, and unstructured
data (images, sound, text). However, its weaknesses are the black box problem (lack of
interpretability): difficult to trace or explain why a particular decision was made and data
dependency: requires massive amounts of data and significant computational resources for
7
training. Its applications includes Image Recognition (CNNs), Natural Language Processing
(LLMs like ChatGPT), Speech Recognition, Autonomous Driving.
C. The Computational Intelligence (CI) Paradigm
Computational Intelligence (CI) is a subset of AI that focuses on nature-inspired, iterative, and
adaptive methods to solve complex real-world problems. CI methods are generally characterized
by adaptation and learning from data and experience, often without prior explicit knowledge
(rules).
Table 2.1: The major CI sub-paradigms
CI Paradigm
Biological/Natural
Inspiration
Key Characteristic
Core Algorithms/Methods
Artificial
Neural
Networks
(ANN)
The
Human
Brain/Nervous System
Learning
patterns
from data.
Perceptrons,
Multi-Layer
Perceptrons (MLP), Deep
Learning
(CNN,
RNN,
Transformer).
Evolutionary
Computation
(EC)
Natural
Selection/Biological
Evolution
Optimization
and
search
through
repeated
mutation,
crossover,
and
selection.
Genetic Algorithms (GAs),
Genetic
Programming,
Evolutionary Strategies.
Human Reasoning with
Vagueness/Uncertainty
Handling imprecise,
subjective,
or
incomplete
information
with
degrees of truth.
Fuzzy Sets, Fuzzy Logic
Controllers,
Membership
Functions.
Social
Behavior
of
Animals (e.g., ants, birds)
Distributed problemsolving
through
simple,
local
interactions between
agents.
Ant Colony Optimization
(ACO), Particle Swarm
Optimization (PSO).
Fuzzy Systems
(FS)
Swarm
Intelligence
(SI)
complex
implicitly
D. Machine Learning Paradigms (Learning Styles)
Within Connectionist/CI approaches, we classify the method based on the type of data and
feedback provided to the learning agent. Table 3.2 is explains the three fundamental ML
paradigms.
8
Table 3.2: The three fundamental ML paradigms
Learning
Paradigm
Data Type
Goal
Feedback
Mechanism
Typical Tasks
Supervised
Learning
Labeled
Data
(InputOutput
pairs)
To predict a
specific output
for a given
input.
Correct output is
given; the model
minimizes the
error (loss).
Classification
(Spam/Not
Spam),
Regression
(Price
Prediction).
Unlabeled
Data (Input
only)
To find hidden
patterns,
structures,
or
representations
within the data.
No
correct
output is given;
the
model
identifies
similarity.
Clustering
(Market
Segmentation),
Dimensionality
Reduction
(PCA).
No
fixed
data
(Learns
from
interaction)
To learn an
optimal
sequence
of
actions
to
maximize
cumulative
reward in an
environment.
Reward/Penalty
signals from the
environment
based on actions
taken.
Autonomous
Driving,
Robotics, Game
Playing
(AlphaGo).
Unsupervised
Learning
Reinforcement
Learning (RL)
V. The Modern Trend: Hybrid/Neuro-Symbolic AI
In modern AI research, there is a strong move toward Hybrid systems, often called NeuroSymbolic AI, which attempts to integrate the strengths of both Symbolic and Connectionist
paradigms. The goal is to achieve the learning and pattern recognition power of Deep Learning
with the interpretability, reasoning, and knowledge structure of Symbolic AI.
Example: Using a Neural Network (Connectionist) to perceive a scene (e.g., identify objects), and
then feeding those symbolic representations into a Logic Program (Symbolic) to answer complex
questions (e.g., "Is the red block on top of the green block?").
9
CHAPTER 3. SEARCH, LOGIC AND LEARNING ALGORITHMS
3.1 AI Search
Search is process of exploring possible states to find a solution to a problem. Many AI problems
(e.g., path finding, games, logic puzzles) can be formulated as searching for a sequence of actions
that leads from an initial state to a goal state. Below are key components of a search problem:
State Space: The set of all possible states.
Initial State: The state where the agent starts.
Actions/Successor Function: A description of what actions are possible from a given state and
what state results.
Goal Test: A way to check if a state is the goal state.
Path Cost: A function that assigns a numerical cost to a path
Types of Search
A.) Uninformed (Blind) Search – No extra information about the goal. Uninformed search
algorithms explore the problem space using only the information available in the problem
definition ie Initial state ,
Available actions and Goal state. They do not use any additional
guidance or heuristics to estimate how close a state is to the goal. That’s why they’re called
“blind”—they search without knowing which direction is better.
Examples:
Algorithm
Strategy
Strengths
Weaknesses
Breadth-First
Explores all nodes level by Guarantees shortest path
High memory usage
Search (BFS)
level
(if costs are uniform)
Depth-First Search Explores one path deeply
May get stuck in deep
Low memory usage
(DFS)
before backtracking
or infinite paths
Uniform
Cost Expands node with lowest
Can be slow with
Finds least-cost solution
Search (UCS)
path cost
widely varying costs
Bidirectional
Searches forward from start Efficient for certain Requires goal state to
Search
and backward from goal
problems
be known
B.) Informed (Heuristic) Search – Uses domain knowledge to guide the search. A heuristic is an
estimate of the cost or distance from a given state to the goal. It’s like having a compass that points
you in the direction of the solution, rather than wandering blindly.
For example, in a path finding problem, a heuristic might be the straight-line distance to the
destination. Other applications include path finding in maps, puzzle solving, robot navigation
Examples: Popular Informed Search Algorithms
10
Algorithm
Heuristic Use
Strengths
Weaknesses
Greedy
Best- Chooses node with lowest Fast
and
memoryMay miss optimal path
First Search
heuristic value
efficient
Combines path cost + Finds optimal path if Can be slow and
A* (A-Star)
heuristic: f(n) = g(n) + heuristic is admissible memory-intensive
h(n)
Beam Search
Explores top-k best nodes at
May discard optimal
Reduces memory usage
each level
paths
Example: A* in Action
Imagine navigating a map from Ozoro to Lagos:
g(n) = actual distance traveled so far
h(n) = straight-line distance to Lagos
f(n) = total estimated cost
A* will prioritize paths that minimize f(n)—balancing progress and promise.
3.2 Logic in AI
Logic is a formal system for representing knowledge and reasoning. It is the backbone of
reasoning, decision-making, and knowledge representation. It enables machines to simulate
human-like thinking by applying formal rules to facts and statements. . It allows AI systems to
represent facts about the world and infer new facts from existing ones and Make decisions based
on logical reasoning
Types of Logic in AI
Type of Logic
Propositional
Logic
First-Order
Logic
Description
Use Cases in AI
Deals with simple statements (propositions) that Rule-based systems, basic
are either true or false
inference
Extends propositional logic with quantifiers and Expert systems, semantic
predicates
analysis
Handles concepts like possibility, necessity, and Planning, reasoning under
Modal Logic
belief
uncertainty
Deals with degrees of truth rather than binary Control systems, decisionFuzzy Logic
true/false
making
Machine
learning,
Bayesian Logic Probabilistic reasoning based on Bayes' theorem
predictive modeling
Importance of Logic in AI
11
I.
II.
III.
IV.
Precision: Logic provides a structured way to define rules and relationships.
Inference: AI can deduce new knowledge from existing facts.
Explainability: Logical systems are transparent and interpretable.
Verification: Logical models can be tested for consistency and correctness.
Example: Propositional Logic
Let’s say:


P: "It is raining."
Q: "The ground is wet."
Using logic:



Implication: P → Q ("If it is raining, then the ground is wet")
Negation: ¬P ("It is not raining")
Conjunction: P ∧ Q ("It is raining and the ground is wet")
These logical operations help AI systems reason about the world.
3.3: Learning Algorithms
AI learning algorithms are sets of rules or instructions that an AI system uses to learn from data
and make decisions or predictions. They form the core of machine learning (ML), a subset of AI.
The algorithms are broadly categorized into three main types based on the nature of the training
data and the goal of the learning process.
1. Supervised Learning
The model is trained on labeled data, meaning the training data includes input variables and their
corresponding correct output labels. This is like learning with a teacher. The goal is to learn a
mapping from the input to the output so it can predict the output for new, unseen data.
Types of Problems
 Classification: Predicting a categorical output (e.g., spam or not spam, cat or dog).
 Regression: Predicting a continuous numerical value (e.g., house prices, temperature).
 Common Algorithms: Linear Regression, Logistic Regression, Decision Trees, Random
Forest, Support Vector Machines (SVM), k-Nearest Neighbors (k-NN), and Neural
Networks.
2. Unsupervised Learning
The model is trained on unlabeled data, which is data without pre-defined output labels. The
algorithm must find patterns, structures, or relationships within the data on its own. The goal is to
explore the data and discover hidden patterns or intrinsic structures.
12
Types of Problems




Clustering: Grouping similar data points together (e.g., customer segmentation).
Dimensionality Reduction: Reducing the number of features in a dataset while retaining
important information (e.g., Principal Component Analysis, or PCA).
Association: Discovering rules that describe relationships between variables (e.g., market
basket analysis like "people who buy X also buy Y").
Common Algorithms: K-Means Clustering, Hierarchical Clustering, DBSCAN, and
Principal Component Analysis (PCA).
3. Reinforcement Learning (RL)
The model, called an "agent," learns by interacting with an "environment." It learns the optimal
sequence of actions to take to achieve a goal through a system of rewards and penalties
(feedback). The goal is to learn a policy, which is a mapping from states to actions, that maximizes
the cumulative reward. This is often used for sequential decision-making.
Common Algorithms: Q-Learning, Deep Q-Networks (DQN), and Policy Gradient Methods
(e.g., REINFORCE, PPO).
Other Notable Categories:
Semi-Supervised Learning: Uses a small amount of labeled data combined with a large amount
of unlabeled data for training.
Deep Learning: A subfield of machine learning that uses multi-layered neural networks (deep
neural networks) to learn complex patterns directly from raw data, often used in conjunction with
supervised or unsupervised methods.
3.4. Model Selection and Evaluation
Model selection is the process of choosing the most appropriate model from a set of candidates
based on performance, complexity, and suitability for the task.
Criteria for Selection





Accuracy: How well the model predicts or simulates outcomes.
Complexity: Simpler models are preferred if performance is comparable.
Interpretability: Ease of understanding and explaining the model.
Scalability: Ability to handle large data or complex systems.
Computational Efficiency: Time and resources required to train or simulate.
Common Techniques
• Cross-validation: Splitting data into training and testing sets to assess generalization.
• Grid Search / Random Search: Systematic exploration of hyperparameters.
13
• Information Criteria: AIC (Akaike Information Criterion), BIC (Bayesian Information Criterion)
for statistical models.
• Domain Knowledge: Leveraging expert insight to guide model choice.
Model Evaluation
Model evaluation assesses how well a selected model performs using quantitative metrics and
qualitative analysis.
B. Evaluation Metrics
Validation Techniques
• Holdout Method: Train/test split.
• K-Fold Cross-Validation: Repeated training/testing across folds.
• Leave-One-Out: Extreme form of cross-validation.
• Bootstrapping: Resampling technique for estimating accuracy.
Cross-Validation:



k-fold cross-validation
Leave-one-out cross-validation
Stratified cross-validation
Performance Metrics:



Classification: Accuracy, precision, recall, F1-score, ROC-AUC
Regression: MSE, MAE, R-squared
Clustering: Silhouette score, adjusted rand index
Bias-Variance Tradeoff:
 Bias: Error due to overly simplistic assumptions
 Variance: Error due to sensitivity to small fluctuations
 Optimal Complexity: Balance between bias and variance
14
CHAPTER 4: MACHINE LEARNING AND NATURE-INSPIRED ALGORITHMS
4.I. Nature-Inspired Algorithms (NIAs)
NIAs is a class of metaheuristic algorithms that mimic natural processes, phenomena, and
behaviors to solve complex optimization problems. This is to find a near-optimal or optimal
solution efficiently, especially for problems where traditional mathematical methods are too slow,
computationally expensive, or struggle with non-linear, high-dimensional search spaces. The
Inspiration Sources include:
Biological Evolution: Natural selection, genetics (e.g., Genetic Algorithms).
Swarm Intelligence: Collective behavior of social insects or animals (e.g., Particle Swarm
Optimization, Ant Colony Optimization).
Physics/Chemistry: Laws of nature (e.g., Simulated Annealing, Gravitational Search Algorithm).
B. Key Advantages over Traditional Optimization
15
C. Generic Steps of a Nature-Inspired Optimization Algorithm
1. Initialization: Create a population of potential solutions (individuals/particles/agents) often
randomly in the search space.
2. Evaluation: Calculate the fitness of each solution using a predefined objective function (also
called a fitness function).
Update Loop (Iteration): Apply nature-inspired operators (e.g., mutation, velocity update,
pheromone deposition) to generate new solutions; Update the positions/states of the agents based
on their own best experience and the best experience of the collective (swarm/population) and Reevaluate the fitness of the new solutions.
3. Selection/Elitism: Keep the best-performing individuals for the next iteration.
4. Termination: Stop when a maximum number of iterations is reached, a satisfactory solution is
found (precision threshold met), or no significant improvement is observed.
NIAs in Machine Learning (ML)
NIAs are primarily used in ML as powerful optimization tools to solve problems that are otherwise
computationally expensive or non-convex. The optimizations are detailed below.
A. Optimization of Model Parameters (Training)
The fundamental goal of most ML models (e.g., Neural Networks, SVMs) is to minimize a Loss
Function (the objective function). While gradient-based methods (like back propagation) are
standard, NIAs can offer an alternative, especially for complex or non-differentiable loss functions.
Example: Using PSO or GA to find the optimal weights and biases of a Neural Network,
effectively replacing or supplementing the traditional backpropagation algorithm.
B. Hyperparameter Optimization
Hyperparameters (e.g., learning rate, number of layers, regularization strength) are parameters set
before training a model. The search space for optimal hyperparameters is often vast, non-linear,
and discrete, making NIAs highly suitable. The process is listed below.
1. An individual in the NIA population represents a set of hyperparameters.
2. The fitness function is the cross-validation performance (e.g., accuracy,
-score) of the ML
model trained with those hyperparameters.
3. The NIA evolves the hyperparameter sets over generations/iterations to maximize the fitness
(model performance).
16
Table 4.2 Summary of Key Nature-Inspired Algorithms
Algorithm
Genetic
Algorithm (GA)
Inspiration
Category
Source
Natural
Evolutionary
Selection/Genetics
Particle Swarm Bird
Optimization
Flocking/Fish
(PSO)
Schooling
Swarm
Intelligence
Ant
Colony Foraging behavior Swarm
Optimization
of Ants
Intelligence
(ACO)
Artificial Bee Foraging behavior Swarm
Colony (ABC)
of Honey Bees
Intelligence
Cuckoo Search Brood Parasitism Bio-inspired
(CS)
of Cuckoos
Algorithm
Inspiration Source
Category
Genetic
Algorithm (GA)
Natural
Evolutionary
Selection/Genetics
Core
Mechanism
Selection,
Crossover,
Mutation
to
evolve
a
population.
Particles follow
their own best
position () and
the global best
position ( ).
Artificial "ants"
use
virtual
"pheromones" to
find the shortest
path on a graph.
Employed bees,
onlooker bees,
and scout bees
explore
and
exploit
food
sources
(solutions).
Cuckoos
lay
eggs (solutions)
in host nests,
with a small
probability of
eggs
being
discovered
(Lévy flights for
global search).
Core
Mechanism
Selection,
Crossover,
Mutation
to
evolve
a
population.
ML Application
Example
Feature
Selection,
Hyperparameter
Optimization
Training
weights,
Clustering
NN
Feature
Selection,
Network
Routing
Clustering,
Continuous
Function
Optimization
Optimization of
Model
Parameters
ML Application
Example
Feature
Selection,
Hyperparameter
Optimization
17
4.3 Engineering Applications
Here are some applications in various fields of engineering
4.3.1 Structural Optimization
Problem: Design structures with optimal performance characteristics
Objectives:



Minimize weight while maintaining strength
Maximize stiffness with material constraints
Optimize shape for specific loading conditions
Nature-Inspired Solutions:
 GA for topology optimization
 PSO for size and shape optimization
 ACO for discrete design variables
4.3.2 Control System Design
Problem: Design controllers for dynamic systems
Applications:
 PID controller tuning using GA
 Fuzzy controller optimization with PSO
 Robust controller design using evolution strategies
4.3.3 Signal Processing
Applications:



Filter design using genetic algorithms
Feature selection with particle swarm optimization
Signal denoising using evolutionary algorithms
4.3.4 Manufacturing and Production
Applications:



Production scheduling using genetic algorithms
Supply chain optimization with ant colony optimization
Quality control using machine learning
18
CHAPTER 5: LECTURE NOTES ON NATURAL LANGUAGE PROCESSING (NLP)
BASICS
5.1 Introduction to Natural Language Processing (NLP)
NLP is a multidisciplinary field within AI and Linguistics that empowers computers to understand,
interpret, and generate human language (text or speech). This help to bridge the communication
gap between humans and machines.
Main Components of NLP
1. Natural Language Understanding (NLU): This focuses on comprehending human
language — grammar, meaning, and intent. For example: Understanding that “Book a
flight” is a request, not a statement.
2. Natural Language Generation (NLG): It involves producing human-like language from
data or structured information. For Instance, generating a weather report like “It will rain
tomorrow.”
3. Speech Recognition and Synthesis: This converts spoken words to text and vice versa.
For Example: Virtual assistants like Alexa or Siri.
The Challenge of Language Ambiguity
Human language is difficult for computers due to these challenges:
 Ambiguity: Words or sentences having multiple meanings (e.g., polysemy, homonymy).
 Context Dependency: Meaning relies heavily on surrounding text and common-sense
knowledge.
 Variations: Handling sarcasm, slang, dialects, and grammatical errors.
 Multilingual and code-mixed texts
5.2. Traditional NLP Pipeline: Levels of Analysis
The process of NLU traditionally follows a layered analysis of language structure:
1. Lexical Analysis: Tokenization (splitting text into words/units) and analyzing word structure.
2. Syntactic Analysis (Parsing): Analyzing grammatical structure and the relationships between
words (e.g., identifying subject, object).
3. Semantic Analysis: Determining the literal meaning of words and phrases.
4. Discourse Integration: Understanding how a sentence's meaning is influenced by previous
sentences (e.g., anaphora resolution for pronouns like "it," "he").
5. Pragmatic Analysis: Interpreting the sentence based on the real-world context and the speaker's
intended meaning.
Figure 5.1 is the NLP pipeline diagram with custom labels, vibrant colors, and additional stages
like Speech Recognition and Multilingual Processing
19
Figure 5.1” NPL Pipeline
1.
Input (Speech/Text)
2.
Speech Processing
3.
Text Processing (Tokenization, Lemmatization, etc.)
4.
Multilingual Processing (Language Detection, Translation)
5.
Feature Extraction (TF-IDF, Embeddings)
6.
Feature Selection
7.
Model Training
8.
Output (Sentiment, Named Entities, Translated Text)
Table 5.1: Core Tasks in NLP
Task
Description
Example
Tokenization
Splitting text into words or
“I love AI.” → [I, love, AI]
sentences
Part-of-Speech
Tagging
(POS) Identifying word roles (noun, “AI runs fast.” → AI/Noun,
verb, etc.)
runs/Verb
“Google is in California.” →
Named Entity Recognition
Identifying names, places, dates
Google/Org
(NER)
Sentiment Analysis
Detecting emotion or opinion
“The movie was great!” →
Positive
Machine Translation
Translating languages
English → French
Text Summarization
Creating shorter summaries
“News article → short summary”
20
Task
Description
Example
Question Answering
Finding precise answers from “Who is
data
Nigeria?”
the
president
of
5.3. Modeling the Language: From Counting to Deep Learning
5.3.1. Statistical Models (The Counting Era)
N-Gram Language Models: Simple probabilistic models that predict the next word based on the
N – 1 preceding words (Markov assumption). Its main Limitation is data sparsity; many long
sequences do not appear in the corpus, leading to zero-probability estimates.
5.3.2. Distributional Semantics (Vector Embeddings)
The meaning of a word is derived from the context in which it appears ("A word is characterized
by the company it keeps."). For instance Word2Vec (and GloVe) is an algorithms that create dense
vector representations (embeddings) for words, capturing semantic and syntactic relationships.
The Key Featureis that it allows for vector arithmetic that models human relationships (e.g.,
vector(’King’) − vector(’Man’) + vector(’Woman’) ≈ vector(’Queen’)).
5.3.3. Recurrent Neural Networks (RNNs)
Neural networks designed for sequences, using a hidden state (memory) to process text one token
at a time. LSTM / GRUis an advanced RNN architectures that use gates (input, forget, output) to
manage the information flow, thereby solving the vanishing gradient problem and capturing
long-term dependencies.
5.4. The Deep Learning Revolution: The Transformer
5.4.1 The Deep Learning Revolution
Deep learning aAllows the model to dynamically look at the entire input sequence at once and
assign different weights (importance scores) to different words when processing a single word.
It is a mechanism where all elements of a single input sequence attend to each other to compute a
representation for the entire sequence. The advantage is that it eliminates the sequential processing
constraint of RNNs, enabling massive parallelization of computations and speeding up training
exponentially.
5.4.2. Large Language Models (LLMs)
21
The Transformer architecture enabled the scaling of models into LLMs. These are categorized
based on their Transformer stack design:
Model
Type
Architecture
Primary Task/Focus
Example Models
Uses the Encoder
stack only.
NLU Tasks:
Classifying, extracting
information,
understanding context.
BERT (Bidirectional
Encoder
Representations from
Transformers)
DecoderOnly
Uses the Decoder
stack only.
NLG Tasks:
Generating sequential,
coherent text (Causal
Language Modeling).
GPT (Generative Pretrained Transformer)
series
EncoderDecoder
Uses both Encoder
(for input context)
and Decoder (for
output generation).
Seq2Seq Tasks:
Translation,
summarization,
question answering.
T5 (Text-to-Text
Transfer Transformer)
EncoderOnly
5.5 Key Applications of NLP
NLP is the driving force behind numerous real-world applications:
1. Sentiment Analysis (Opinion Mining): Determining the emotional tone or attitude
(positive, negative, neutral) expressed in a piece of text (e.g., customer reviews, social
media).
2. Machine Translation: Automatically translating text or speech from one natural language
to another (e.g., Google Translate).
3. Chatbots and Virtual Assistants: Enabling conversational interfaces to understand user
input and provide appropriate responses (e.g., Siri, Alexa, customer service chatbots).
4. Named Entity Recognition (NER): Identifying and classifying named entities in text into
pre-defined categories such as person names, organizations, locations, dates, and
quantities.
5. Text Summarization: Creating a concise summary of a longer document.
o Extractive: Pulling key sentences directly from the source text.
o Abstractive: Generating new sentences that paraphrase the content.
6. Speech Recognition (Speech-to-Text): Converting spoken language into written text.
7. Text Classification: Categorizing text into predefined labels (e.g., spam detection, topic
categorization).
22
CHAPTER 6: KNOWLEDGE REPRESENTATION AND ELICITATION
Knowledge Representation and Elicitation
6.1. Introduction
Knowledge Representation (KR) and Knowledge Elicitation (KE) are two fundamental concepts
in Artificial Intelligence (AI) and Expert Systems. They deal with how knowledge is acquired,
structured, stored, and used for intelligent reasoning and problem-solving.
6.2. Knowledge Representation (KR)
Knowledge Representation is the process of encoding knowledge about the world into a form
that a computer system can understand and utilize to solve complex problems. It bridges
human understanding and machine reasoning.
Objectives




To represent real-world facts in a way that is computable.
To enable inference and reasoning.
To support learning and updating of knowledge.
To facilitate communication between humans and AI systems.
6.3. Types of Knowledge
a.) Declarative Knowledge (Know-What)
Knowledge about facts and things - descriptive knowledge.
Examples:



"Paris is the capital of France"
"Water boils at 100°C at sea level"
"A square has four equal sides"
Characteristics:



Factual information
Can be stated explicitly
Static descriptions
b.) Procedural Knowledge (Know-How)
Knowledge about how to perform tasks or procedures.
Examples:
23



How to ride a bicycle
Steps to diagnose a disease
Algorithm for sorting numbers
Characteristics:



Process-oriented
Action sequences
Often difficult to articulate
c.) Heuristic Knowledge
Rules of thumb, best practices, and expert intuitions.
Examples:


"If a patient has fever and cough, check for respiratory infection first"
"In chess, control the center of the board early"
Characteristics:



Based on experience
Not always 100% accurate
Domain-specific expertise
d.) Meta-Knowledge
Knowledge about knowledge - understanding the scope and limitations of what is known.
Examples:



Knowing which expert to consult
Understanding when a rule applies
Confidence levels in beliefs
e.) Structural Knowledge
Knowledge about relationships and organization of concepts.
Examples:



Taxonomies (hierarchical classifications)
Ontologies (formal specifications of concepts)
Conceptual frameworks
6.4. Methods of Knowledge Representation
24
Representation Method Description
Example
Graph structures with nodes (concepts)
“Bird → can fly”
Semantic Networks
and edges (relationships)
Data structures with slots and fillers Frame for “Car” with slots:
Frames
(attributes and values)
color, engine type
IF temperature > 38°C
“If–Then” rules representing logic
Production Rules
THEN fever = true
Logic (Propositional &
Mathematical logic to represent facts ∀x (Human(x) → Mortal(x))
Predicate)
Hierarchical conceptual structures OWL or RDF ontologies in
Ontologies
shared across domains
AI systems
Restaurant script: enter →
Represent typical sequences of events
Scripts
order → eat → pay
6.5. Characteristics of a Good Knowledge Representation




Representational adequacy: Can express all needed knowledge.
Inferential adequacy: Supports reasoning and inference.
Inferential efficiency: Allows efficient reasoning.
Acquisitional efficiency: Easy to update and acquire new knowledge.
6.6. Knowledge Elicitation (KE)
Knowledge Elicitation is the process of extracting knowledge from human experts, documents, or
data sources to build a knowledge base for AI or expert systems. The Purpose is to gather domain
expertise from human specialists in a form that computers can use.
7. Methods of Knowledge Elicitation
i.
4.1 Interview-Based Methods: This could be structured or unstructured
Structured Interviews




Predetermined questions
Consistent format
Easy to analyze
May miss important details
Unstructured Interviews




Open-ended discussion
Flexible exploration
Rich information
Harder to analyze
25
Best Practices:




ii.
Prepare questions in advance
Listen actively
Avoid leading questions
Record and document thoroughly
Protocol Analysis
Think-Aloud Protocol: The expert verbalizes thoughts while performing tasks. It captures
reasoning process and Reveals tacit knowledge
Example: Ask a chess expert to think aloud while playing:



"I see the opponent's queen is exposed..."
"I'm considering knight to F3 to control the center..."
"This creates a fork opportunity..."
Advantages:



Captures procedural knowledge
Shows reasoning steps
Reveals decision-making
Challenges:



iii.
Experts may struggle to articulate
Talking may affect performance
Time-consuming to analyze
Observation
Technique: Watch experts perform tasks, Note actions, decisions, patterns and Document
exceptional cases
Advantages:



Captures actual behavior
Reveals unconscious knowledge
Identifies context
Disadvantages:



Observer effect (people act differently when watched)
Cannot capture internal reasoning
Time-intensive
26

Questionnaires and Surveys: This will Gather structured information, Reach multiple
experts and Quantify knowledge. This could be Rating scales, Multiple choice, Openended questions and Ranking exercises
Advantages:



Efficient for multiple experts
Easy to quantify
Standardized responses
Disadvantages:



Limited depth
May miss nuances
Response bias
V. Concept Mapping: The Expert creates visual diagram of concepts and relationships, shows
hierarchies and connections and collaborative process. The process involve:
1.
2.
3.
4.
5.
Identify key concepts
Arrange spatially
Draw relationships
Label connections
Refine and iterate
Advantages:




Visual and intuitive
Reveals structure
Collaborative tool
Good for validation
vi. Repertory Grid: This is a Systematic method to identify constructs experts use and compare
and contrast elements. The process involve :
1.
2.
3.
4.
5.
Identify elements (e.g., different diseases)
Present triads (three at a time)
Ask: "Which two are similar, and how do they differ from the third?"
Extract constructs (bipolar dimensions)
Rate all elements on each construct
Example: Elements: Disease A, B, C "A and B are both viral, while C is bacterial" Construct:
Viral ↔ Bacterial
Advantages
27



Systematic
Reveals expert's classification scheme
Quantifiable results
Disadvantages:
 Time-consuming
 May be tedious
 Requires statistical analysis
Hierarchical Task Analysis (HTA): Break down complex tasks into subtasks and operations.
Example: Diagnose Patient
1. Collect patient history
1.1 Ask about symptoms
1.2 Review medical records
1.3 Note relevant family history
2. Perform physical examination
2.1 Check vital signs
2.2 Examine affected areas
3. Order tests
3.1 Determine necessary tests
3.2 Interpret results
4. Formulate diagnosis
4.1 Compare symptoms to known conditions
4.2 Rule out alternatives
4.3 Confirm with evidence
Advantages:



Structured approach
Identifies task structure
Shows decision points
viii. Case-Based Methods: This method Use real or hypothetical cases where the expert explains
reasoning and captures contextual knowledge. The Approaches are:



Critical incident technique: Focus on exceptional cases
Case studies: Deep analysis of specific situations
Scenario-based elicitation: Present hypothetical scenarios
Advantages:
 Contextual
 Reveals decision criteria
 Shows handling of complexity
28
ix. Laddering: This involve asking Series of "why" and "how" questions, Moves up/down
abstraction levels
Example:
 "How do you diagnose this condition?" (procedural)
 "Why do you check that symptom first?" (strategic)
 "Why is that important?" (higher-level goal)
Advantages:
 Reveals reasoning hierarchy
 Uncovers goals and motivations
 Simple to conduct
x. Automated Knowledge Elicitation: This is the Machine Learning Approaches which involve
 Extract patterns from data
 Learn from examples
 Build models automatically
Text Mining:
 Extract knowledge from documents
 Identify key concepts
 Discover relationships
Crowdsourcing:
 Gather knowledge from many people
 Aggregate responses
 Validate through consensus
6.7. Steps in Knowledge Elicitation
1. Define the domain and scope
2. Select experts and data sources
3. Choose elicitation methods
4. Conduct sessions (interviews, observations, etc.)
5. Analyze and validate the acquired knowledge
6. Convert to formal representation (rules, ontology, etc.)
6.8. Challenges in Knowledge Elicitation
 Tacit knowledge is hard to express.
 Experts may be biased or inconsistent.
 Communication barriers between experts and engineers.
 Time constraints and information overload.
Relationship Between KR and KE
29
Human Experts provide domain-specific insights. The Knowledge Engineer elicits this
information and translates it into formal structures. These structures are stored in the Knowledge
Base for use in intelligent systems.
Aspect Knowledge Elicitation (KE)
Focus Extracting knowledge from experts
Output Raw or semi-structured knowledge
Tools Interviews, observation, protocol analysis
Goal Capture expertise
6.9 The Knowledge Engineering Process
Phase 1: Knowledge Identification




Knowledge Representation (KR)
Structuring and storing the knowledge
Formalized, computable knowledge
Logic, rules, semantic networks
Enable reasoning and decision-making
Identify problem domain
Define scope and boundaries
Identify key concepts
Locate knowledge sources
Phase 2: Knowledge Acquisition
 Select elicitation techniques
 Gather knowledge from experts
 Collect relevant documents
 Document findings
Phase 3: Knowledge Analysis
 Organize knowledge
 Identify patterns
 Resolve inconsistencies
 Structure information
Phase 4: Knowledge Representation
 Choose representation scheme
 Formalize knowledge
 Build knowledge base
 Implement in system
30
Phase 5: Knowledge Validation
 Test with experts
 Verify accuracy
 Check completeness
 Identify gaps
Phase 6: Knowledge Maintenance
 Update regularly
 Incorporate feedback
 Refine and improve
 Document changes
6.10. Applications
a) Expert systems (medical diagnosis, troubleshooting)
b) Decision support systems
c) Intelligent tutoring systems
d) Robotics and automation
e) Natural language processing
f) Knowledge management systems
Summary
Knowledge representation provides the foundation for intelligent systems to reason and solve
problems. Multiple representation schemes exist, each with strengths and weaknesses. Knowledge
elicitation is challenging but critical for building effective AI systems.Combination approaches
often work best, using multiple techniques. Modern AI increasingly combines symbolic and subsymbolic (neural) approaches. Successful knowledge engineering requires both technical skills
and domain understanding. The field continues to evolve with knowledge graphs and neuralsymbolic integration at the forefront. Understanding both how to represent knowledge formally
and how to extract it from experts remains fundamental to building intelligent systems that can
reason, explain, and be trusted in critical applications.
31
CHAPTER 7. MATHEMATICAL AND LOGIC FOUNDATIONS OF AI
7.1. Introduction
Artificial Intelligence (AI) aims to create machines that can reason, learn, and make decisions like
humans. To achieve this, AI relies heavily on mathematics for numerical modeling and logic for
reasoning.
Mathematics enables quantitative reasoning (e.g., calculating probabilities, optimizing models),
while Logic provides qualitative reasoning (e.g., making inferences, understanding relationships).
Example:

A self-driving car uses mathematics (neural networks and optimization) to recognize
objects, and logic (if–then rules) to decide when to stop or turn.
7.2. Mathematical Foundations of AI
These include:
1.
2.
3.
4.
5.
Linear Algebra
Calculus and Optimization
Probability and Statistics
Set Theory and Graph Theory
Information Theory
7.2.1 Linear Algebra
Linear algebra is the backbone of modern AI, especially machine learning and deep learning.
Key Concepts:




Scalars: single numbers (e.g., temperature = 35°C)
Vectors: ordered lists of numbers (e.g., sensor readings [2.5, 3.0, 1.5])
Matrices: 2D arrays of numbers representing data or transformations
Tensors: multidimensional generalizations of matrices
Example 1: Image as a Matrix
An image (100×100 pixels) is represented as a matrix of numbers — each value corresponds to
pixel intensity. If color, it’s a tensor with 3 layers (Red, Green, Blue).
Example 2: Neural Network Computation
y = Wx + b
Where
32
W = weight matrix
x = input vector
b = bias
y = output
This operation (matrix multiplication) is repeated through layers during training.
2.2 Calculus and Optimization
Calculus deals with change and optimization, crucial for learning models. AI models (like neural
networks) learn by adjusting their parameters to minimize error.
Gradient Descent Algorithm:
Example:
If a model predicts house prices, and the true price is ₦10 million while prediction = ₦8 million,
the algorithm calculates the error (cost) and adjusts weights (using derivatives) to reduce that
difference next time.
7.2.3 Probability and Statistics
AI often deals with uncertainty — the world isn’t deterministic. Probability quantifies how likely
an event is to occur.
33
7.2.4 Set Theory
Set theory defines collections of objects and their relationships — essential for data structures and
logic. Basic Operations include:
 Union (A ∪ B): elements in A or B
 Intersection (A ∩ B): elements common to A and B
 Complement (A′): elements not in A
 Cartesian Product (A × B): all possible ordered pairs
Example:
Let
A = {AI, ML, DL}
B = {ML, CV, NLP}
Then
A ∩ B = {ML} (common topic)
Set theory helps represent classes, features, and relations — e.g., objects detected in an image
form a “set of entities.”
34
7.2.5 Graph Theory
Graphs represent relationships between entities (nodes) through connections (edges).
Applications in AI:
 Search Algorithms: BFS, DFS, A* for path finding
 Knowledge Graphs: linking related concepts
 Neural Networks: modeled as directed graphs (neurons = nodes, weights = edges)
Example:
A graph of cities connected by roads can be used to find the shortest route using A* algorithm —
just like Google Maps.
7.2.6 Information Theory
Quantifies uncertainty and information content.
Meaning: 1 bit of information per toss.
AI Application:
It is used in Decision Trees to choose attributes that provide the most information gain (reduce
uncertainty).
7.3. Logic Foundations of AI
AI systems also reason symbolically using logic — the formal language of thought and reasoning.
7.3.1 Propositional Logic represents simple statements that are either true or false.
Syntax Example:
P : “It is raining.”
Q : “The ground is wet.”
Rule: ( P
Q ) (If it rains, then the ground is wet)
Inference Example (Modus Ponens):
If ( P
Q ) and ( P ) are true, then ( Q ) must be true.
So, if “It rains,” we conclude “The ground is wet.”
35
3.2 Predicate Logic (First-Order Logic) This extends propositional logic by including
quantifiers and variables.
Example:
“All humans are mortal.”
Predicate logic enables AI to represent facts about objects and reason about them — essential for
knowledge bases and ontologies.
3.3 Fuzzy Logic is Used when reasoning involves imprecise or vague information. Instead of
True/False, fuzzy logic uses degrees of truth (0 to 1).
Example:
“The water is hot”
o 80°C → 0.9 (mostly true)
o 50°C → 0.5 (partly true)
Applications:
 Air conditioners
 Washing machines
 Human-like decision systems
3.4 Modal and Temporal Logic


Modal Logic: Deals with beliefs, intentions, or possibilities
Example: “The agent believes it is raining.”
Temporal Logic: Deals with time-dependent reasoning
Example: “If it rains today, the road will be wet tomorrow.”
Used in autonomous agents, robotics, and planning systems.
4. Integration: Math + Logic in AI
Mathematical Concept Logical Counterpart AI Application
Linear Algebra
Structured reasoning Neural networks
Probability
Bayesian logic
Uncertainty reasoning
Calculus
Optimization
Learning models
36
Mathematical Concept Logical Counterpart AI Application
Set Theory
Predicate logic
Ontologies
Information Theory
Inference
Decision trees
Example:
A medical expert system may use:


Logic: “If symptom A and symptom B → Disease X”
Math:
“Probability
that
symptom
A
occurs
=
Together → System computes likelihood of disease and explains reasoning.
0.7”
Summary
Mathematics provides tools for quantitative modeling and learning while Logic provides structure
for symbolic reasoning and decision-making. Both together form the core of AI intelligence —
enabling reasoning, prediction, and explanation.
Class Quiz:
Section A – Multiple Choice Questions (MCQs)
1. Which mathematical concept is primarily used for representing and manipulating data in
neural networks?
A. Set Theory
B. Calculus
C. Linear Algebra
D. Probability
✅ Answer: C.
2. The gradient descent algorithm in AI is mainly used for:
A. Data storage
B. Optimization and learning
C. Logical reasoning
D. Pattern visualization
✅ Answer: B.
3. In predicate logic, the statement “All cats are animals” can be represented as:
A. ( Cat(x) \rightarrow Animal(x) )
B. ( \forall x (Cat(x) \rightarrow Animal(x)) )
C. ( \exists x (Cat(x) \rightarrow Animal(x)) )
D. ( Cat \land Animal )
✅ Answer: B.
37
4. Entropy in Information Theory measures:
A. Energy consumption of an algorithm
B. Uncertainty in a random variable
C. Accuracy of an AI model
D. Speed of data transfer
✅ Answer: B.
5. Which logic type allows reasoning with degrees of truth rather than strict true/false values?
A. Predicate Logic
B. Modal Logic
C. Fuzzy Logic
D. Temporal Logic
✅ Answer: C.
Section B – Short Answer Questions
6. Explain how Linear Algebra is applied in Artificial Neural Networks.
7. Define Bayes’ Theorem and explain its role in AI..
8. Differentiate between Propositional Logic and Predicate Logic with examples.
✅ Sample Answer:
9. Describe how Information Theory is used in building Decision Trees.
10. Give two real-world examples showing the integration of Mathematics and Logic in AI
systems.
38
CHAPTER 8 : EXPERT SYSTEMS AND AUTOMATED REASONING
8.1 Introduction
An Expert System (ES) is a computer program that emulates the decision-making ability of a
human expert in a specific domain. It uses knowledge and inference procedures to solve problems
that normally require human expertise.
Key Characteristics
 Domain-specific expertise: Focused on narrow, well-defined areas
 Reasoning capability: Can draw conclusions from available information
 Explanation facility: Can explain its reasoning process
 Knowledge-intensive: Relies on accumulated expert knowledge rather than conventional
algorithms
┌─────────────────────────────────────────────────────┐
│
USER INTERFACE
│
└─────────────┬───────────────────────┬────────────────┘
│
│
▼
▼
┌──────────────────┐
┌──────────────────┐
│ EXPLANATION
│
│ KNOWLEDGE
│
│ FACILITY
│
│ ACQUISITION
│
│
│
│ FACILITY
│
└────────┬─────────┘
└────────┬─────────┘
│
│
│
┌────────────┐
│
└───►│ INFERENCE │◄────┘
│ ENGINE
│
└──────┬─────┘
│
┌──────▼─────┐
│ KNOWLEDGE │
│
BASE
│
└────────────┘
│
┌──────▼─────┐
│ WORKING
│
│ MEMORY
│
└────────────┘
Figure 8.1: Architecture of an Expert System
8.3. Core Components
8.3.1 Knowledge Base
39
The Knowledge Base is the repository of domain-specific knowledge containing facts, rules,
heuristics, and relationships that represent expert knowledge.
Types of Knowledge Stored
Facts
Statements about the domain that are known to be true.
Examples:
- Water boils at 100°C at sea level
- Aspirin is used to reduce fever
- Python is a programming language
- The heart pumps blood throughout the body
Rules
IF-THEN statements that encode expert reasoning.
Example 1 - Medical Diagnosis:
RULE-001: Flu Diagnosis
IF patient has fever > 101°F
AND patient has body aches
AND patient has fatigue
AND patient has cough
THEN diagnosis = "Influenza" with confidence 0.85
Example 2 - Car Troubleshooting:
RULE-045: Battery Problem
IF engine does not start
AND headlights are dim or off
AND battery age > 3 years
THEN problem = "Dead battery"
recommendation = "Replace or recharge battery"
Example 3 - Loan Approval:
RULE-120: Loan Decision
IF credit_score > 750
AND debt_to_income_ratio < 0.35
AND employment_duration > 2 years
THEN loan_approval = "Approved"
interest_rate = "Prime rate"
Heuristics
Rules of thumb based on expert experience.
Examples:
40
- "In network troubleshooting, always check physical connections
first"
- "If a patient's symptoms worsen at night, consider respiratory
issues"
- "In stock trading, don't invest more than 10% in a single
stock"
Knowledge Representation Formats
Production Rules
IF <conditions>
THEN <actions or conclusions>
Frames
FRAME: Car
Make: Toyota
Model: Camry
Year: 2020
Problem: [Unknown]
Symptoms: [Engine won't start, Dim lights]
Diagnosis: [To be determined]
Semantic Networks
Disease → causes → Symptoms
Disease → treated_by → Medication
Patient → has → Symptoms
8.3.2 Inference Engine
The Inference Engine is the "brain" of the expert system that applies logical rules to the knowledge
base to deduce new information and make decisions.
Main Functions
1.
2.
3.
4.
Pattern matching: Match current facts against rule conditions
Rule selection: Choose which rule to apply next
Inference: Draw conclusions from available information
Control strategy: Manage the reasoning process
41
Inference Strategies
Figure 8.2 Reasoning flow for Forward and Backward Chaining
Forward Chaining (Data-Driven): It reason from facts to conclusions; starts with available data
and moves forward.
Process:
1. Start with known facts
2. Find rules whose conditions match facts
3. Execute rules (fire rules)
4. Add conclusions to fact base
5. Repeat until goal reached or no more rules apply
Example - Medical Diagnosis:
Known Facts:
- Patient has fever (102°F)
- Patient has sore throat
- Patient has swollen lymph nodes
Step 1: Match facts to rules
RULE-1: IF fever AND sore throat
THEN possible_infection = "throat infection"
[RULE FIRES] → Add: possible_infection = "throat infection"
Step 2: New fact enables another rule
RULE-2: IF possible_infection = "throat infection"
AND swollen_lymph_nodes
THEN diagnosis = "Strep throat"
42
[RULE FIRES] → Conclusion: "Strep throat"
Step 3: Prescription rule
RULE-3: IF diagnosis = "Strep throat"
THEN prescribe = "Antibiotics"
[RULE FIRES] → Recommend: "Antibiotics"
Use Cases:
 Diagnosis systems
 Monitoring and control systems
 Planning systems
 Configuration systems
Backward Chaining (Goal-Driven): It reason from goals to facts; starts with a hypothesis and
works backward to verify it.
Process:
1.
2.
3.
4.
5.
Start with a goal/hypothesis
Find rules that conclude this goal
Check if conditions are satisfied
If not, set conditions as sub-goals
Repeat recursively until proven or disproven
Example - Animal Identification:
Goal: Identify the animal
Hypothesis: Animal is a "Cheetah"
RULE: IF animal is a mammal
AND animal is a carnivore
AND animal has spots
AND animal runs very fast
THEN animal is a cheetah
Check Sub-goals:
├─ Is it a mammal?
│ └─ RULE: IF has hair AND nurses young THEN mammal
│
├─ Has hair? [ASK USER] → Yes
│
└─ Nurses young? [ASK USER] → Yes
│
[CONFIRMED: Mammal]
│
├─ Is it a carnivore?
│ └─ RULE: IF eats meat THEN carnivore
│
└─ Eats meat? [ASK USER] → Yes
│
[CONFIRMED: Carnivore]
43
│
├─ Has spots? [ASK USER] → Yes
└─ Runs very fast? [ASK USER] → Yes
[CONCLUSION: Animal is a Cheetah]
Use Cases:



Diagnostic systems
Prescription systems
Question-answering systems
Mixed/Hybrid Chaining: Combines both approaches for optimal reasoning.
Conflict Resolution Strategies
When multiple rules match, the system needs to decide which to fire first:
1.
2.
3.
4.
Refraction: Don't fire the same rule on same data twice
Recency: Prefer rules using most recently added facts
Specificity: Prefer more specific rules over general ones
Priority: Use explicit rule priorities set by experts
Example:
Working Memory: [fever=103°F, cough=yes, age=5, rash=yes]
Matching Rules:
- RULE-A: IF fever > 100 THEN check_infection (Priority: 3)
- RULE-B: IF fever > 102 AND age < 12 THEN urgent_care (Priority:
1)
- RULE-C: IF rash AND fever THEN check_measles (Priority: 2)
Selection Order:
1. RULE-B (highest priority, most specific)
2. RULE-C (medium priority)
3. RULE-A (lowest priority, most general)
8.3.3 Working Memory (Blackboard)
The Working Memory (also called blackboard or fact base) stores temporary facts, data, and
intermediate conclusions during the reasoning process. The contents are :





Initial problem data
Intermediate results
Conclusions
User inputs
Current state of reasoning
44
Example - Car Diagnosis Session
INITIAL STATE:
├─ car_starts = "no"
├─ battery_age = "4 years"
├─ headlights = "dim"
└─ clicking_sound = "yes"
AFTER INFERENCE:
├─ car_starts = "no"
├─ battery_age = "4 years"
├─ headlights = "dim"
├─ clicking_sound = "yes"
├─ battery_voltage = "low" [INFERRED]
├─ problem_identified = "dead battery" [INFERRED]
└─ recommended_action = "replace battery" [INFERRED]
Operations




Assert: Add new facts
Retract: Remove facts
Modify: Update existing facts
Query: Check for facts
8.3.4 User Interface
The User Interface manages communication between the user and the expert system, making it
accessible to non-technical users.
Functions
1.
2.
3.
4.
Input collection: Gather information from users
Output presentation: Display results and recommendations
Query handling: Ask clarifying questions
Visualization: Present information clearly
Types of Interfaces
Question-Answer Interface
Sequential questions to gather information.
Example - Medical System:
System: What is the patient's body temperature?
User: 102°F
System: Does the patient have a cough?
User: Yes
45
System: Is the cough dry or productive?
User: Productive
System: Does the patient have difficulty breathing?
User: No
Menu-Based Interface
AUTOMOTIVE EXPERT SYSTEM
─────────────────────────
Select Problem Category:
1. Engine Problems
2. Electrical Problems
3. Brake Problems
4. Transmission Problems
5. Exit
Your choice: 1
ENGINE PROBLEMS
─────────────────
What symptom are you experiencing?
1. Engine won't start
2. Engine stalls
3. Engine overheats
4. Strange noises
5. Back to main menu
Your choice: 1
Form-Based Interface
┌──────────────────────────────────────┐
│
LOAN APPROVAL EXPERT SYSTEM
│
├──────────────────────────────────────┤
│ Applicant Name: ________________
│
│ Annual Income: $________________
│
│ Credit Score: ___
│
│ Employment Years: ___
│
│ Requested Loan: $________________
│
│ Debt-to-Income Ratio: ___%
│
│
│
│ [Submit] [Clear] [Help]
│
└──────────────────────────────────────┘
Natural Language Interface
46
User: My car won't start and the lights are very dim.
System: I understand you're having trouble starting your
car and the lights appear dim. This typically indicates
a battery problem. Let me ask a few questions:
1. How old is your car battery?
2. Do you hear any clicking sound when you turn the key?
3. Have you had any electrical issues recently?
Graphical Interface




Decision trees visualization
Diagnostic flowcharts
Interactive diagrams
Progress indicators
8.3.5 Explanation Facility
The Explanation Facility enables the system to explain its reasoning, justifying conclusions and
building user trust.
Types of Explanations
"How" Explanations
Explains how a conclusion was reached.
Example:
User: How did you diagnose strep throat?
System: I diagnosed strep throat based on the following
reasoning chain:
1. You reported fever (102°F) and sore throat
→ Applied RULE-15: Possible throat infection
2. You confirmed swollen lymph nodes and white patches
on tonsils
→ Applied RULE-22: Strong indicator of bacterial infection
3. Rapid strep test came back positive
→ Applied RULE-35: Confirms streptococcal infection
Therefore, diagnosis: Strep Throat (Confidence: 95%)
47
"Why" Explanations
Explains why a particular question was asked or why specific information is needed.
Example:
System: What is your credit score?
User: Why do you need my credit score?
System: Your credit score is needed because:
1. It's a primary factor in determining loan eligibility
2. RULE-101 states: Credit score > 650 is required
for standard loans
3. It affects the interest rate calculation
4. Lenders use it to assess credit risk
Would you like to provide your credit score now?
"Why Not" Explanations
Explains why something was NOT concluded.
Example:
User: Why didn't you recommend aspirin?
System: Aspirin was not recommended because:
1. Patient age is 8 years old
2. RULE-240: Children under 12 should not take aspirin
due to Reye's syndrome risk
3. Alternative: Acetaminophen is safer for children
4. Medical guideline reference: AAP-2023-156
Recommendation: Use children's acetaminophen instead.
Trace Explanations
Shows the complete reasoning path.
Example:
REASONING TRACE
═══════════════════════════════════════
Goal: Determine loan approval
Step 1: Check credit score
48
└─ Input: Credit score = 780
└─ Evaluation: Passes threshold (> 650) ✓
Step 2: Check debt-to-income ratio
└─ Input: Monthly debt = $1,500
└─ Input: Monthly income = $5,000
└─ Calculation: 1500/5000 = 0.30 (30%)
└─ Evaluation: Below limit (< 43%) ✓
Step 3: Check employment history
└─ Input: Employment duration = 5 years
└─ Evaluation: Exceeds minimum (> 2 years) ✓
Step 4: Apply decision rule (RULE-105)
└─ All conditions satisfied
└─ CONCLUSION: Loan APPROVED
└─ Interest rate: Prime + 0.5%
└─ Confidence: 100%
═══════════════════════════════════════
Benefits of Explanation





Transparency: Users understand system decisions
Trust: Builds confidence in recommendations
Learning: Users learn from system reasoning
Debugging: Helps identify errors in knowledge base
Validation: Experts can verify correctness
8.3.6 Knowledge Acquisition Facility
The Knowledge Acquisition Facility helps domain experts and knowledge engineers add, modify,
and maintain knowledge in the system without programming.
Functions
1.
2.
3.
4.
5.
Knowledge entry: Add new rules and facts
Knowledge editing: Modify existing knowledge
Consistency checking: Detect contradictions
Validation: Verify knowledge correctness
Organization: Structure and categorize knowledge
Methods
Direct Knowledge Entry
49
┌──────────────────────────────────────┐
│
ADD NEW RULE WIZARD
│
├──────────────────────────────────────┤
│ Rule ID: RULE-250
│
│
│
│ IF Conditions:
│
│ [+] fever > 100
│
│ [+] headache = severe
│
│ [+] stiff_neck = yes
│
│
│
│ THEN Conclusion:
│
│ diagnosis = "possible meningitis"
│
│ urgency = "immediate"
│
│ action = "Emergency room now"
│
│
│
│ Confidence Factor: [0.90]
│
│
│
│ [Save Rule] [Test Rule] [Cancel]
│
└──────────────────────────────────────┘
Automated Learning from Examples
Example Case 1:
Input: [fever=103, cough=yes, congestion=yes, age=25]
Output: "Common cold"
Example Case 2:
Input: [fever=104, cough=yes, breathing_difficulty=yes, age=65]
Output: "Pneumonia"
Example Case 3:
Input: [fever=102, cough=yes, body_aches=yes, age=30]
Output: "Influenza"
System: Based on these examples, I suggest this rule:
RULE (LEARNED):
IF fever > 103 AND cough = yes AND breathing_difficulty = yes
THEN suspect = "Pneumonia"
priority = "High"
Do you approve this rule? [Yes/No/Modify]
Knowledge Refinement
INCONSISTENCY DETECTED
══════════════════════════════════════
50
RULE-100: IF temperature > 100°F THEN fever = "yes"
RULE-230: IF temperature > 99°F THEN fever = "yes"
Issue: Conflicting thresholds for fever determination
Suggestion:
1. Keep RULE-100 (WHO standard: 100°F)
2. Modify RULE-230 to 100°F
3. Create separate rule for "low-grade fever" at 99°F
Select action: [1] [2] [3] [Custom]
Interactive Knowledge Acquisition
System: I need to learn how to diagnose engine problems.
System: When the engine doesn't start, what do you check first?
Expert: I check if the battery has power.
System: How do you know if the battery has power?
Expert: I check if the headlights are bright.
System: What if the headlights are dim?
Expert: That indicates a weak or dead battery.
System: I'll create this rule:
RULE: IF engine_wont_start AND headlights_are_dim
THEN battery_problem = "weak or dead"
Is this correct? [Yes/No/Refine]
Expert: Yes
4. Example: MYCIN Medical Expert System
MYCIN is one of the most famous expert systems, developed at Stanford in the 1970s for
diagnosing bacterial infections and recommending antibiotics.
Components in MYCIN
Knowledge Base
Total Rules: ~600 rules
Sample Rules:
RULE-052:
51
IF: 1) The infection is primary-bacteremia
2) The site of the culture is one of the sterile sites
3) The suspected portal of entry is the gastrointestinal tract
THEN: There is suggestive evidence (0.7) that the identity
of the organism is bacteroides
RULE-145:
IF: 1) The patient is allergic to penicillin
2) The organism requires penicillin treatment
THEN: Use erythromycin as alternative antibiotic
Inference Engine



Used backward chaining
Goal: Identify organism and recommend treatment
Certainty factors (0-1 scale) for probabilistic reasoning
Working Memory
Patient Session #1247
════════════════════════════════════
Patient: John Doe
Age: 45
Symptoms:
- Fever: 103°F
- Chills: Yes
- Recent surgery: Yes (5 days ago)
Lab Results:
- White blood cell count: Elevated
- Blood culture: Pending
Interim Conclusions:
- Infection type: Post-surgical
- Likely organism: Staphylococcus (0.75 confidence)
- Allergies: None reported
Recommended Tests:
- Blood culture
- Gram stain
User Interface
MYCIN CONSULTATION SESSION
══════════════════════════════════════
Q1: Is the patient's name?
52
A1: John Doe
Q2: What is the patient's age?
A2: 45
Q3: Have you obtained positive cultures?
A3: Yes
Q4: What is the infection type?
A4: Primary-bacteremia
Q5: Has the patient had recent surgery?
A5: Yes - 5 days ago
...
MYCIN DIAGNOSIS
══════════════════════════════════════
Most likely organisms:
1. Staphylococcus aureus (0.85)
2. E. coli (0.45)
3. Streptococcus (0.30)
RECOMMENDED TREATMENT:
Primary: Vancomycin 1g IV every 12 hours
Alternative: If allergic, use daptomycin
Duration: 14 days
Monitoring: Check kidney function weekly
Explanation Facility
USER: Why do you suspect Staphylococcus?
MYCIN: I concluded Staphylococcus aureus (0.85) because:
1. RULE-052 (0.7): Patient has primary bacteremia after
recent surgery, which is strongly associated with
Staph infections
2. RULE-089 (0.8): Patient has no history of UTIs or
GI procedures, making E. coli less likely
3. RULE-134 (0.6): Elevated WBC with fever post-surgery
is typical of Staph infection
Combined certainty: 0.85
53
5. Another Example: DENDRAL Chemical Analysis Expert System
Identify molecular structure from mass spectrometry data.
Components
Knowledge Base



Rules about chemical structures
Mass spectrometry patterns
Chemical bonding principles
RULE-23:
IF: Mass spectrum shows peak at M-15
AND: Compound contains carbon chain
THEN: Likely loss of CH₃ group (methyl group)
Confidence: 0.9
RULE-87:
IF: Strong peak at m/z = 43
AND: Molecular formula contains oxygen
THEN: Possible acetyl group (CH₃CO)
Confidence: 0.75
Inference Engine



Generate possible structures
Test against spectral data
Rank by likelihood
Output Example
DENDRAL ANALYSIS RESULTS
═══════════════════════════════════════
Input Data:
- Molecular formula: C₈H₁₀O
- Mass spectrum peaks: M+ at 122, strong at 107, 79
Possible Structures (ranked):
1. Acetophenone (94% match)
Reasoning:
- Loss of CH₃ (15) gives peak at 107 ✓
- Phenyl cation at 77 ✓
- Acetyl group confirmed ✓
54
2. Methyl benzyl ether (67% match)
Reasoning:
- Possible but less consistent
- Some peaks unexplained
3. Para-cresol methyl ether (45% match)
...
RECOMMENDATION: Structure #1 (Acetophenone)
6. Example: XCON/R1 Computer Configuration System
Configure VAX computer systems for Digital Equipment Corporation (DEC).
Components
Knowledge Base
10,000+ rules about:
- Component compatibility
- System requirements
- Physical layout constraints
- Cable requirements
RULE-1542:
IF: System includes disk drive model RA81
AND: Cabinet selected is BA11-K
THEN: Error - RA81 requires larger cabinet (BA623)
Action: Suggest BA623 cabinet
RULE-2891:
IF: System has 3+ tape drives
THEN: Require additional controller
Add: Second TU81 controller
Cable: 15-foot BC23M cable
Configuration Process
USER INPUT:
- Purpose: Database server
- Max users: 50
- Storage: 500GB
- Backup: Tape system required
XCON PROCESSING:
Step 1: Select base system (VAX 8650) ✓
55
Step 2: Calculate memory (256MB) ✓
Step 3: Select disk array (RA81 × 4) ✓
Step 4: Add tape backup system ✓
Step 5: Configure controllers ✓
Step 6: Verify compatibility ✓
Step 7: Generate floor layout ✓
OUTPUT:
Complete system configuration with:
- Bill of materials (127 components)
- Physical layout diagram
- Cable requirements (23 cables)
- Estimated cost: $245,000
- Installation time: 3 days
7. Advantages and Limitations
Advantages
1.
2.
3.
4.
5.
6.
Preserves expertise: Captures expert knowledge
Consistent decisions: No fatigue or emotional factors
Available 24/7: Always accessible
Explainable: Can justify decisions
Training tool: Educates novices
Handles uncertainty: Uses probabilistic reasoning
Limitations
1.
2.
3.
4.
5.
6.
Limited to narrow domains: Not general intelligence
Knowledge acquisition bottleneck: Time-consuming to build
Lacks common sense: Cannot reason beyond its rules
Maintenance challenge: Keeping knowledge current
Cannot learn from experience: (traditional systems)
Brittleness: Poor performance outside domain
8. Modern Evolution
Integration with Machine Learning
HYBRID SYSTEM
═══════════════════════════════════════
Rule-Based Component:
56
- Medical guidelines
- Legal regulations
- Safety protocols
ML Component:
- Pattern recognition from data
- Learning from cases
- Probability estimation
Integration Layer:
- ML suggests, rules validate
- Rules handle edge cases
- Combined confidence scores
Cloud-Based Expert Systems




Distributed knowledge bases
Real-time updates
Collaborative development
API access
Mobile Expert Systems




On-site diagnosis
Field service support
Agricultural advisors
Emergency response
Quiz Questions
1. Define an Expert System and list its main components.
2. Explain the role of the Inference Engine with an example.
3. Differentiate between Forward and Backward Chaining.
4. What is the purpose of the Explanation Facility in Expert Systems?
5. Describe Deductive, Inductive, and Abductive reasoning with examples.
6. What are the differences between Expert Systems and traditional programs?
7. Describe how MYCIN works and what it was used for.
8. Explain how Fuzzy Logic handles uncertainty in Expert Systems.
9. Mention any two software tools used for Automated Reasoning.
10. Outline the steps involved in developing an Expert System.
57
CHAPTER 9: PATTERN RECOGNITION
9.1. Introduction to Pattern Recognition
Pattern Recognition (PR) is the scientific discipline that teaches machines to recognize patterns
and regularities in data — much like humans do. It involves analyzing input data (signals, images,
speech, or sensor readings) and categorizing it into classes or groups.
Figure 9.1: Pattern Recognition System Architecture
For example:


When you unlock your phone using face recognition, the system detects your facial pattern
and matches it against stored templates.
A spam filter in Gmail recognizes patterns in emails that indicate “spam” or “not spam.”
The Goal is to automatically classify or identify objects, signals, or phenomena using their
measurable features.
Everyday Examples
Domain
Finance
Healthcare
Agriculture
Telecommunications
Security
Pattern Recognition Example
Detecting fraudulent transactions in banks
Recognizing tumors from MRI scans
Identifying diseased plants from leaf images
Classifying network signal dropouts
Fingerprint and facial identification systems
9.2. Basic Concepts and Terminology
A pattern is a collection of features that uniquely represents an object or event.
Example: A handwritten digit “5” — pattern = shape, curves, stroke thickness.
Feature
58
A measurable property of the pattern. For the digit “5”, features could be number of edges, pixel
density, and stroke direction.
Feature Vector
A numerical representation of all features:
x = [x_1, x_2, x_3, ..., x_n]
Example: A face image might have a vector of 128 features from an AI model.
Classification
Assigning an unknown sample to a predefined class.
Example: Classifying an animal as cat or dog.
Clustering
Grouping data based on similarity without predefined labels.
Example: Grouping customers into segments based on shopping behavior.
9.3. Pattern Recognition System Architecture
A typical system follows this flow:
Raw Data → Preprocessing → Feature Extraction → Classification → Decision Output
Let’s break it down:
a. Data Acquisition : This is to collect data using sensors, cameras, or microphones.
👉 Example: A camera capturing facial images.
b. Preprocessing: Clean the data (remove noise, adjust lighting, normalize size).
👉 Example: Converting all images to grayscale and resizing to 100×100 pixels.
c. Feature Extraction:Extract important characteristics or patterns.
👉 Example: Extracting edge lines, texture, or color histograms from an image.
d. Classification / Decision Making :Use algorithms to decide which category the pattern
belongs to.
👉 Example: Neural network classifies the image as “Human Face”.
9.4. Major Types of Pattern Recognition Techniques
9.4.1 Statistical Pattern Recognition This Relies on probability theory to make classification
decisions.
59
Example:
A bank fraud detection system may classify a transaction as:


Class 1: Normal Transaction
Class 2: Fraudulent Transaction
It calculates probabilities using Bayes’ theorem:
and assigns the data to the class with the highest posterior probability.
9.4.2 Structural (Syntactic) Pattern Recognition It describes patterns using symbolic structures
(like strings or graphs) and grammar rules.
Example:
A computer vision system identifying road signs:


"STOP" → (Hexagonal shape + Red color + White text)
Grammar rules define how features combine to form a valid sign.
9.4.3 Neural Pattern Recognition It uses Artificial Neural Networks (ANNs) or Deep Learning
to learn from data automatically.
Example:


A CNN (Convolutional Neural Network) used for facial recognition in smartphones.
A Recurrent Neural Network (RNN) recognizing speech commands (“play music”,
“stop”).
9.4.4 Template Matching It compares the input with a stored template pixel by pixel.
Example:
Optical Character Recognition (OCR) systems use template matching to recognize printed
characters.
Limitation: Does not handle rotated, scaled, or noisy patterns well.
9.4.5 Machine Learning-Based Recognition
This uses algorithms that learn from labeled data (supervised) or find patterns in unlabeled data
(unsupervised).
60
Algorithm
Description
Example
digit
k-Nearest Neighbors (k- Classifies based on the majority of nearby Handwritten
samples
recognition
NN)
Builds rules based on feature values
Credit approval systems
Decision Tree
Support Vector Machine
Finds optimal boundary between classes Face detection
(SVM)
Probabilistic classifier
Email spam detection
Naïve Bayes
9.5. Supervised vs. Unsupervised Pattern Recognition
Aspect
Supervised
Unsupervised
Labeled
Unlabeled
Input Data
Predict known categories
Find natural groupings
Goal
Common Algorithms SVM, ANN, Decision Tree K-means, DBSCAN
Classify emails as spam or not Cluster customers by purchase patterns
Example
9.6. Feature Extraction and Selection
Feature Extraction It transforms raw data into a compact and informative representation.
Examples:



From audio → Extract frequency spectrum
From images → Extract texture, color histograms, or edges
From text → Extract word embeddings or TF-IDF vectors
Feature Selection It chooses only the most relevant features to reduce computational cost.
Techniques:



Filter methods: Select features using statistical tests (e.g., correlation).
Wrapper methods: Evaluate subsets of features (e.g., forward selection).
Embedded methods: Select features during model training (e.g., LASSO regression).
9.7. Classifier Design and Decision Making
A classifier uses decision boundaries to assign unknown samples to classes.
Example 1: Minimum Distance Classifier
61
9.8. Performance Evaluation Metrics
9.9. Real-World Applications
Field
Healthcare
Security
Agriculture
Finance
Application
Cancer detection, ECG pattern analysis
Face, fingerprint, iris recognition
Crop disease recognition using drone images
Credit scoring and fraud detection
62
Field
Application
Traffic sign and object recognition
Transportation
Environmental Monitoring Detecting air pollution patterns from sensors
9. 10. Challenges in Pattern Recognition
I.
II.
III.
IV.
V.
High-dimensional data: Too many features cause overfitting.
Noisy data: Errors in sensors or input devices.
Incomplete datasets: Missing values or labels.
Computation cost: Deep models require high resources.
Explainability: Many AI models are “black boxes.”
9.11. Applications in Engineering
9.4.1 Computer Vision
Image Classification: Object recognition in images, Scene understanding, Medical image analysis
and Quality control inspection
Object Detection: Localization and classification, Real-time detection systems , Autonomous
vehicle perception and Security and surveillance
Image Segmentation: Pixel-level classification, Medical image segmentation. Semantic
segmentation and Instance segmentation
9.4.2 Signal Processing
Speech Recognition: Feature extraction (MFCC, LPC). Hidden Markov Models. Deep neural
networks and Real-time processing
Biomedical Signals: ECG analysis and arrhythmia detection. EEG pattern recognition, EMG
signal classification and Medical monitoring systems
Robotics: Object recognition and grasping. Navigation and mapping, Human-robot interaction
.Quiz / Self-Assessment
A. Multiple Choice Questions (MCQs)
1. Pattern Recognition primarily deals with:
a) Data Compression
b) Identifying patterns in data
c) Hardware design
d) Software compilation
Answer: b
63
2. Which of the following is unsupervised?
a) Decision Tree
b) SVM
c) K-means Clustering
d) Naïve Bayes
Answer: c
3. In pattern recognition, features are:
a) Random noise
b) Properties that describe the data
c) Hidden layers of neural networks
d) Output labels
Answer: b
4. Template matching is not suitable when:
a) The image is noisy
b) The pattern is identical
c) Lighting is constant
d) Rotation is fixed
Answer: a
5. The main goal of feature selection is:
a) Increase dimensionality
b) Reduce irrelevant features
c) Add noise
d) None of the above
Answer: b
B. Short Answer Questions
1.
2.
3.
4.
5.
Differentiate between supervised and unsupervised pattern recognition.
Describe three common feature extraction techniques.
Explain how neural networks perform pattern recognition.
What are the main challenges in designing a pattern recognition system?
List three real-world applications of pattern recognition in Nigeria.
C. Assignment Idea
Design a simple pattern recognition system using Python and scikit-learn to classify
handwritten digits (MNIST dataset).
Include data preprocessing, feature extraction, model training, and performance evaluation.
64
CHAPTER 10: DISTRIBUTED SYSTEMS
10.1 Fundamentals of Distributed Systems
Distributed systems consist of multiple independent computers (nodes) connected through a
network that cooperate to appear as a single system to the user.
Key Characteristics
1. Distribution Transparency – hides complexity from the user.
o Access Transparency: Access local or remote resources using the same operations.
Example: Accessing a file via NFS as if it’s on your local drive.
o Location Transparency: The user doesn’t need to know where a resource is located.
Example: Cloud databases like Google Firestore.
o Migration Transparency: Resources can move between nodes without affecting
access.
Example: Virtual machines migrating between data centers.
o Replication Transparency: Multiple copies of data appear as a single entity.
Example: CDN servers delivering identical web content globally.
2. Scalability
o Size Scalability: The system can grow by adding more nodes.
Example: Expanding a Hadoop cluster by adding more nodes.
o Geographical Scalability: System spans wide areas (e.g., cloud datacenters across
continents).
o Administrative Scalability: Supports multiple organizations managing different
parts.
Example: Federated cloud computing.
3. Fault Tolerance
o Availability: Keeps running despite failures (e.g., redundant servers).
o Reliability: Performs correctly for a specified duration.
o Safety: Avoids catastrophic results during faults.
o Maintainability: Easy repair and upgrades.
Challenges
1. Network Partitions
o Failures may isolate nodes, causing data inconsistency or “split-brain.”
o The CAP theorem states:
A distributed system can only guarantee two of the following three —
Consistency, Availability, and Partition Tolerance.
2. Consistency Models
o Strong
Consistency:
All
nodes
see
updates
Example: Banking systems.*
o Eventual
Consistency:
All
nodes
converge
Example: DNS, Amazon DynamoDB.*
immediately.
over
time.
65
o
o
Weak Consistency: No timing guarantees.
Causal
Consistency:
Related
events
are
seen
in
order.
Example: Chat applications maintaining message order.*
3. Coordination and Synchronization
o Clock Synchronization: Ensures time consistency (e.g., NTP).
o Distributed Mutual Exclusion: Prevents concurrent access to shared resources.
o Leader Election: Selects a node for coordination.
o Distributed Snapshots: Capture system state for debugging or checkpointing.
10.2 Distributed AI Architectures
10.2.1 Multi-Agent Systems(MAS)
A Multi-Agent System (MAS) consists of autonomous entities (agents) that interact to achieve
goals.
Agent Characteristics
 Autonomy: Acts without direct control.
 Reactivity: Responds to environment changes.
 Proactivity: Goal-oriented behavior.
 Social Ability: Communicates with others.
Agent Architectures:



Reactive Agents: Simple, rule-based (e.g., thermostat).
Deliberative Agents: Plan before acting (e.g., chess programs).
Hybrid Agents: Combine reactive + deliberative models (e.g., self-driving cars).
66

BDI Agents: Use Beliefs, Desires, Intentions for reasoning.
Communication & Coordination:



Message Passing (e.g., FIPA ACL).
Ontologies: Common understanding (e.g., OWL).
Negotiation: Agents resolve conflicts collaboratively.
10.2.2 Federated Learning
A decentralized ML approach where data stays on devices, and only model updates are shared.
Workflow:
1. Each client trains a local model.
2. Model parameters (not raw data) are sent to a central server.
3. The server aggregates updates → global model.
Algorithms:




FedAvg: Average parameters.
FedProx: Handle data heterogeneity.
FedNova: Normalize updates for fairness.
Personalized FL: Client-specific models.
Challenges:




Statistical Heterogeneity: Clients have diverse data.
System Heterogeneity: Devices vary in performance.
Communication Efficiency: Limited bandwidth.
Privacy: Ensuring data confidentiality.
Example: Google’s Gboard uses federated learning to improve text prediction without uploading
your data.
10.2.3 Distributed Machine Learning
Parallelization Types:
1. Data Parallelism: Split data among nodes.
Example: Each GPU trains on a different batch.*
2. Model Parallelism: Split large models across devices.
Example: Training GPT models across clusters.*
3. Hybrid Approaches: Combine both; includes pipeline and tensor parallelism.
10.3 Consensus and Coordination
67
10.3.1 Consensus Algorithms
Goal: Agree on a single data value among distributed nodes.


Byzantine Fault Tolerance (BFT):
o Tolerates malicious or faulty nodes.
o PBFT (Practical BFT): Requires at least 3f + 1 nodes to tolerate f failures.
o Used in blockchain systems.
Crash Fault Tolerance:
o Paxos: Classic algorithm ensuring agreement.
o Raft: Easier-to-understand consensus for replicated logs.
o Applications: Google Spanner, etcd, Zookeeper.
10.3.2 Distributed Coordination
 Leader Election:
o Bully Algorithm: Highest ID node wins.
o Ring Algorithm: Token passing for fairness.
 Mutual Exclusion:
o Ricart-Agrawala: Permission-based.
o Token Ring: Token gives access to critical section.
 Distributed Snapshots:
o Chandy-Lamport Algorithm captures consistent global states for recovery.
10.4 Applications in AI
10.4.1 Distributed Search



Parallel A* – Search space divided among nodes.
Distributed Constraint Satisfaction – Multi-agent problem solving.
MapReduce Search – For large-scale problems (e.g., web indexing).
Load Balancing: Work stealing, hash partitioning, adaptive partitioning to ensure even task
distribution.
10.4.2 Distributed Reasoning


Knowledge Base Distribution: Each node holds part of the knowledge.
Collaborative Problem Solving:
o Contract Net Protocol – Tasks offered, agents bid.
o Auction Mechanisms – Optimal resource allocation.
10.4.3 Swarm Intelligence
 Ant Colony Optimization (ACO): Agents (ants) deposit pheromones to find optimal
paths.
 Particle Swarm Optimization (PSO): Particles move through solution space influenced
by best solutions.
68

Applications: Routing, scheduling, robotics, energy optimization.
Distributed systems enable large-scale, resilient, and collaborative computation — essential for
modern AI, IoT, and cloud computing. AI techniques like federated learning, multi-agent
systems, and swarm intelligence leverage distribution to enhance scalability and robustness.
Quiz Questions
1. What is location transparency in distributed systems?
2. Explain the CAP theorem and its implications.
3. Differentiate between strong and eventual consistency.
4. Describe how federated learning enhances data privacy.
5. What are the roles of agents in a multi-agent system?
6. Name and explain any two consensus algorithms.
7. What is the purpose of the Chandy-Lamport algorithm?
8. Give one real-world example of distributed machine learning.
9. How does Ant Colony Optimization apply to distributed AI?
69
CHAPTER 11: DATA AND INFORMATION SECURITY
11.1 Security Fundamentals
Security ensures that systems, data, and services remain confidential, accurate, and available
while resisting unauthorized access or disruption.
11.1.1 Security Objectives (CIA Triad + Additional Properties)
 Confidentiality
Ensures that information is accessible only to authorized entities.



Mechanisms: Encryption (AES, RSA), access controls, user authentication.
Privacy Preservation: Protection of personal data under GDPR or NDPR.
Data Classification: Public, confidential, restricted categories.
Example: HTTPS encryption prevents eavesdroppers from reading transmitted data.
 Integrity
Ensures that information remains accurate, complete, and unaltered.


Mechanisms: Checksums (SHA-256), digital signatures, version control, audit trails.
Tamper Detection: Hash comparisons or blockchain immutability.
Example: Git version control ensures code integrity via hashes.
 Availability
Ensures that authorized users can access data when needed.



Mechanisms: Redundancy, fault-tolerant design, load balancing.
Disaster Recovery: Backup sites, recovery plans.
Performance Optimization: Ensures service uptime.
Example: Cloud providers replicate data across multiple regions for availability.
Additional Security Properties



Authenticity: Verifies user or message identity (e.g., digital certificates).
Non-repudiation: Prevents denial of actions (e.g., signed transactions).
Accountability: Logs and audits to track user actions.
70
Figure 11.1: The CIA Triad with Extended Properties
(Triangle labeled Confidentiality, Integrity, Availability; surrounding ring showing Authenticity,
Accountability, Non-repudiation.)
11.1.2 Threat Models
A threat model defines potential attackers, their goals, and attack methods.
Adversary Capabilities include:
 Passive Attackers: Eavesdrop but don’t alter data (e.g., Wi-Fi sniffing).
 Active Attackers: Modify or inject data (e.g., Man-in-the-Middle).
 Insider Threats: Authorized users who act maliciously.
 Advanced Persistent Threats (APT): Long-term, stealthy, state-sponsored attacks.
Attack Vectors
 Network-based: Man-in-the-middle, sniffing, DDoS.
 Host-based: Malware, ransomware, privilege escalation.
 Application-based: SQL injection, buffer overflow, XSS.
 Social Engineering: Phishing, pretexting, baiting.
Example: The 2017 WannaCry ransomware combined network and host-based attacks.
11.2 Cryptographic Foundations
Cryptography secures data through mathematical transformations that ensure confidentiality,
integrity, and authenticity.
11.2.1 Symmetric Cryptography: This uses a single shared secret key for encryption and
decryption.
Block Ciphers


AES (Advanced Encryption Standard): Standard symmetric cipher (128/192/256-bit keys).
DES/3DES: Legacy standards (now obsolete).
71


Modes of Operation:
o ECB: Simple but insecure.
o CBC: Uses chaining for security.
o CTR, GCM: Enable parallel encryption and authentication.
Key Management: Secure generation and rotation are critical.
Example: AES-256-CBC is widely used in VPNs.
Stream Ciphers It encrypts data bit-by-bit or byte-by-byte.



RC4: Deprecated due to bias.
ChaCha20: Modern and secure (used in TLS 1.3).
Advantages: Fast, low memory usage — ideal for real-time data.
Hash Functions Convert data into fixed-length “fingerprints.”



SHA-2/SHA-3: Secure modern hashes.
MD5, SHA-1: Broken, avoid using.
Applications: Password hashing, digital signatures.
Example: Blockchain uses SHA-256 to link blocks immutably.
Symmetric Encryption Process
(Shows plaintext → encryption (key) → ciphertext → decryption (same key) → original plaintext.)
11.2.2 Asymmetric Cryptography
Uses two keys — a public key (for encryption) and a private key (for decryption).
RSA (Rivest–Shamir–Adleman)
 Mechanism: Based on factoring large primes.
 Applications: Secure email (PGP), digital signatures.
 Recommended Key Size: ≥2048 bits.
Elliptic Curve Cryptography (ECC)



Smaller keys, faster operations.
Algorithms:
o ECDSA: Digital signatures.
o ECDH: Secure key exchange.
Use Case: Mobile/IoT devices with limited power.
Key Exchange Protocols

Diffie–Hellman: Creates shared secret without transmitting it.
72


ECDH: Elliptic curve variant for efficiency.
Perfect Forward Secrecy (PFS): Protects past sessions even if keys are compromised.
Example: TLS 1.3 uses ECDHE for secure key exchange.
11.3 Security in AI Systems
As AI models handle sensitive data, machine learning pipelines themselves become attack
surfaces.
11.3.1 Privacy-Preserving Machine Learning
Differential Privacy



Adds statistical noise to protect individuals’ data.
ε (epsilon): Controls privacy-utility trade-off.
Example: Apple’s iOS analytics and Google’s Chrome telemetry.
Homomorphic Encryption



Enables computation directly on encrypted data.
Fully Homomorphic Encryption (FHE): Allows arbitrary computation, but is slow.
Example: Secure cloud-based medical data processing.*
Secure Multi-party Computation (SMC)


Multiple parties compute joint results without sharing raw data.
Example: Banks jointly detect fraud patterns without revealing transactions.*
Federated Learning Privacy



Local Differential Privacy: Clients add noise before sending updates.
Secure Aggregation: Server aggregates encrypted gradients.
Membership Inference Defense: Prevents attackers from identifying training records.
Figure 11.2: Privacy-Preserving ML Workflow
73
11.3.2 Adversarial Machine Learning
Adversarial Examples




Slightly perturbed inputs that fool AI models.
Attack Types:
o White-box: Attacker knows model parameters.
o Black-box: Attacker uses queries only.
Generation: FGSM, PGD, C&W.
Impact: Misclassification in image or speech systems.
Example: Altering a stop sign to mislead an autonomous car.
Defense Mechanisms



Adversarial Training: Train on adversarial inputs.
Defensive Distillation: Smooths model outputs.
Certified Defenses: Provide provable robustness.
Model Extraction Attacks
Attackers clone a deployed ML model via repeated queries.

Defenses: Query limits, noise injection, watermarking.
11.3.3 AI System Vulnerabilities
Attack Type
Data Poisoning
Model Inversion
Membership
Inference
Description
Example Defense
Inject malicious samples during Data
validation,
training
detection
Reconstruct sensitive training data
Differential privacy
Detect if a data point was in training
outlier
Regularization, noise addition
Example: Extracting faces from facial recognition model parameters.
11.4 Secure AI Applications
11.4.1 Biometric Security
Modalities:
Fingerprint,
face,
iris,
Challenges: Irrevocable once stolen (unlike passwords).
voice,
behavioral.
74
Template Protection:



Cancelable Biometrics: Revocable transformation of templates.
Biometric Cryptosystems: Derive encryption keys from biometrics.
Template Encryption: Stored securely with privacy guarantees.
Example: FaceID uses local storage and secure enclaves for biometric data.
11.4.2 Intrusion Detection Systems (IDS)
Detect malicious or abnormal network/system activity.



Anomaly Detection: Learns normal behavior → flags deviations.
Signature-based: Matches known attack patterns.
Hybrid: Combines both for improved accuracy.
Example: Snort (signature-based) and DeepLog (AI-based anomaly detection).
Figure 11.3: Intrusion Detection Architecture
(Data sources → feature extraction → ML detection module → alert system.)
11.4.3 Blockchain and AI
Blockchain enhances AI trust and transparency.




Smart Contracts: Self-executing logic; AI can trigger or verify conditions.
Decentralized AI: Training and inference across blockchain nodes.
Data Markets: Tokenized exchange for AI datasets.
Consensus Mechanisms: AI models can assist in optimizing energy-efficient consensus.
Example: Ocean Protocol uses blockchain to trade AI datasets securely.
75
Figure 11.4: Blockchain–AI Integration (Smart contracts linking data sources, AI models, and
blockchain ledger.)
Summary
Data and Information Security is a multi-layered discipline encompassing:




Traditional cryptographic protection (AES, RSA, ECC),
AI-specific privacy mechanisms (differential privacy, SMC),
Resilience to AI-targeted threats (adversarial attacks, data poisoning), and
Secure, transparent AI applications (biometrics, IDS, blockchain-AI).
Quiz Questions
1. What are the three main objectives of data security (CIA triad)?
2. Differentiate between symmetric and asymmetric cryptography.
3. Explain how differential privacy protects sensitive data.
4. What is the main weakness of MD5 and SHA-1?
5. Describe how homomorphic encryption benefits cloud computing.
6. What is an adversarial example in ML, and why is it dangerous?
7. Compare insider threats vs. APTs.
8. How does blockchain enhance trust in AI systems?
9. What are cancelable biometrics?
10. Explain the concept of Perfect Forward Secrecy.

76
CHAPTER 12: INTELLIGENT WEB TECHNOLOGIES
12.1 Web Intelligence Foundations
Web Intelligence (WI) is a multidisciplinary field that focuses on the theories, methodologies, and
technologies for designing, developing, and evaluating knowledge-intensive, intelligent systems
and services in the context of the World Wide Web.
12.1.1 Web Structure and Content
The foundational element of the Web is the massive graph of interconnected documents.
Component
Description
Relevance
Web Graph
Nodes
(Web
pages/documents)
connected by Edges (Hyperlinks).
Exhibit Scale-free (a few
nodes have many links, most
have few) and Small-world
(short paths between any two
nodes) properties.
Content
Types
Structured Data (e.g., HTML tables,
XML, relational databases), Semistructured Data (e.g., JSON, RDF,
microformats), and Unstructured Data
(e.g., text, images, video).
Intelligent systems must
handle and transform all three
types for processing.
Metadata
Data about data (e.g., Dublin Core for
documents, Schema.org for entities).
Provides explicit semantic
meaning, enabling smarter
systems.
Web
Standards
HTML5 (markup), CSS3 (styling),
JavaScript (client-side behavior), and
HTTP/HTTPS (communication).
Define the syntax and
protocols for data creation
and transfer.
12.1.2 Semantic Web
The Semantic Web, envisioned by Tim Berners-Lee, is an extension of the current web in which
information is given well-defined meaning, enabling computers and people to work in cooperation.

Resource Description Framework (RDF):
Core Concept: Models information as directed, labeled graphs, using Triples (Subject-PredicateObject).
Example: (<BarackObama>, <is-the-father-of>, <MaliaObama>).

URIs (Uniform Resource Identifiers): Used to uniquely identify subjects, predicates, and
objects, enabling global, unambiguous referencing.
77
Serializations: RDF data can be written in formats like RDF/XML, Turtle, N3, or JSON-LD (a
modern, Web-friendly format).
Web Ontology Language (OWL):
A vocabulary used to formally represent knowledge about a domain (an Ontology). It defines
Classes (concepts) and Properties (relationships between concepts). OWL allows for automated
Inference (deducing new knowledge) and Consistency Checking (identifying contradictions) using
logical axioms and constraints.
Example: If a person is a Father, and all Fathers are Parents, an OWL Reasoner can infer that the
person is a Parent.
SPARQL Query Language:
A SQL-like language specifically designed for querying RDF data (triples and knowledge graphs).
It uses Graph Pattern Matching to find subgraphs that match a specified template. It can execute
queries across multiple distributed RDF data sources, key for large-scale Semantic Web
applications.
12.2 Web Mining
Web Mining is the use of data mining techniques to automatically discover and extract information
from Web documents and services. It is often categorized based on what is being mined: content,
structure, or usage.
12.2.1 Web Content Mining
It focuses on the actual content of Web pages—text, images, video, etc.
 Text Mining: Applies Natural Language Processing (NLP) techniques:
o Information Extraction: Identifying and classifying key data, such as Named
Entities (people, places, organizations) and Relationships (e.g., CEO of).
o Sentiment Analysis: Determining the emotional tone (positive, negative, neutral)
of user-generated content (e.g., product reviews, social media posts).
 Multimedia Mining: Processing non-textual data:
o Image Analysis: Using Deep Learning (CNNs) for object recognition and scene
understanding.
o Cross-modal Analysis: Combining textual context (e.g., caption) with the image
content to improve accuracy.
12.2.2 Web Structure Mining
Focuses on the link structure (the Web Graph) to determine the authority and relevance of pages.

o
Link Analysis:
PageRank Algorithm: Developed by Google, it recursively measures a page's importance by
counting the number and quality of links pointing to it. A link from an authoritative page is worth
more.
78
o

o
o
HITS Algorithm (Hypertext Induced Topic Search): Identifies two types of pages: Authorities
(pages with relevant content) and Hubs (pages that link to many authorities).
Web Graph Analysis:
Centrality Measures: Mathematical tools to quantify the importance of a node in a graph (e.g.,
Betweenness - how often a node is on the shortest path between others; Eigenvector - the influence
of a node).
Community Detection: Grouping closely linked nodes to identify specific web communities or
topical clusters (e.g., blogs about specific technologies).
12.2.3 Web Usage Mining
Focuses on discovering user access patterns from Web server logs, browser logs, and application
events.
 Log File Analysis: Analyzing access logs and click-streams to reconstruct user sessions.
o Session Identification: Crucial step of grouping page views into a single visit or
user session.
o Path Analysis: Discovering common navigation sequences, helping optimize site
structure.
 User Behavior Modeling:
o Web Analytics: Using metrics (traffic, bounce rate, time on page) to understand
overall site performance.
o Conversion Analysis: Tracking user paths leading to a desired action (e.g., a
purchase, a sign-up).
o Personalization: Using identified patterns to customize the user experience (e.g.,
targeted advertising, content display).
12.3 Search Engines and Information Retrieval
Search engines are the most visible application of Intelligent Web Technologies, relying on
efficient data management and sophisticated ranking algorithms.
12.3.1 Search Engine Architecture
1. Crawling and Indexing:
o Web Crawlers: Programs that systematically browse the Web, discovering new pages. Strategies
include Breadth-first (visiting all pages at a depth before moving deeper) or Focused Crawling
(prioritizing pages based on relevance to a topic).
o Index Construction: Creating an Inverted Index (mapping terms to the documents that contain
them), which is the primary structure for fast search.
2. Ranking Algorithms: Determine the order of search results.
o TF-IDF (Term Frequency-Inverse Document Frequency): Weights a term's importance based on
how often it appears in a document (TF) versus how rare it is across the entire corpus (IDF).
o BM25 (Best Matching 25): A probabilistic ranking model that has been a standard in IR for
decades, improving on TF-IDF.
79
o
Machine Learning (Learning to Rank): Modern search engines use complex ML models trained
on hundreds of features (link quality, query match, user clicks) to optimize result ranking.
3. Query Processing:
o Query Understanding: Recognizing the user's intent and expanding the query (e.g., mapping
synonyms).
o Retrieval Models: Algorithms that match the processed query against the index (e.g., Boolean,
Vector Space).
12.3.2 Semantic Search
A search paradigm that goes beyond simple keyword matching to understand the meaning and
context of the search query and the content.
 Knowledge Graphs: Large networks of entities (people, places, concepts) and the semantic
relationships between them.
o Mechanism: Algorithms perform Entity Recognition and Relationship Extraction
to build and query this structured knowledge base.
 Natural Language Queries (NLQ): Allows users to ask questions naturally.
o Question Answering (QA): The system provides a direct, concise answer rather
than a list of documents.
o Conversational Search: The system maintains context across multiple turns,
enabling complex, extended interactions.
12.4 Recommender Systems
Systems that predict user preferences and suggest items (products, movies, news) of potential
interest.
12.4.1 Recommendation Approaches
1. Collaborative Filtering (CF): Based on the idea that users who agreed in the past will agree in
the future.
o User-based CF: Recommends items liked by similar users.
o Item-based CF: Recommends items similar to those the user already liked.
o Matrix Factorization (MF): A powerful technique that discovers Latent Factors (hidden
features) to explain user-item interactions (e.g., gender, genre taste).
o Deep Learning (Neural CF): Uses neural networks to model non-linear interactions between
users and items.
2. Content-based Filtering (CBF): Recommends items that are similar to items the user has liked
in the past, based purely on item features.
o User Profiles: Built from the features of items the user interacted with.
o Hybrid Approaches: The most effective systems combine CF (good for unexpected novelty) and
CBF (good for explaining recommendations).
o
12.4.2 Evaluation and Challenges

o
o

Evaluation Metrics:
Accuracy: (RMSE, MAE) for predicting numerical ratings.
Ranking: (,Precision@k, Recall@k,, NDCG) for evaluating the quality of the top k recommendations.
System Challenges:
80
o
o
o
Cold Start: The system cannot recommend items to new users or recommend new items, as there
is insufficient interaction data.
Scalability & Sparsity: Handling huge datasets while managing the problem of very few user-item
interactions.
Privacy: Protecting user interaction data is critical.
12.5 Web-based AI Applications
12.5.1 Chatbots and Virtual Assistants
 Dialog Systems: The core intelligence behind conversational agents.
o Rule-based: Simple, predefined responses; limited flexibility.
o Retrieval-based: Selects the best response from a corpus of pre-written answers.
o Generative: Uses large language models (LLMs) to generate novel, human-like
responses.
 Natural Language Understanding (NLU):
o Intent Classification: Identifying the user's goal (e.g., "book a flight").
o Entity Extraction: Pulling out parameters (e.g., "to Paris," "on Friday").
o Context Management: Tracking the state of the conversation to handle Multi-turn
Dialogue seamlessly.
12.5.2 Web Analytics and Intelligence
Applying AI and data mining to large-scale web and social media data for business insights.
 Business Intelligence (BI): Uses descriptive and predictive analytics on web usage data.
o KPI Monitoring: Tracking Key Performance Indicators (e.g., conversion rate).
o Predictive Analytics: Forecasting future user trends, traffic, or sales.
 Social Media Analytics:
o Sentiment Analysis: Monitoring public opinion about a brand or topic.
o Influence Analysis: Identifying Key Opinion Leaders (KOLs) and tracking the
spread of content.
o Brand Monitoring: Automated reputation management by detecting emerging
positive or negative trends.
Sample Quiz
1. What are the three types of web mining and what does each focus on?
2. Explain how PageRank determines web page importance.
3. Differentiate between structured and unstructured web data.
4. How does the Semantic Web improve traditional web search?
5. Describe two challenges faced by recommender systems.
6. Outline the core components of a search engine architecture.
7. What role does RDF play in Semantic Web representation?
8. Explain the difference between collaborative and content-based filtering.
9. How does semantic search interpret user intent?
10. Give an example of a web-based AI application and its key technologies.
81
CHAPTER 13. CONVERGENT TECHNOLOGIES
13.1 Definition and Significance
13.1.1 Concept of Convergent Technologies
Convergent technologies refer to the synergistic combination of multiple technological domains
that merge to create new capabilities, applications, and paradigms. This convergence typically
involves the integration of nanotechnology, biotechnology, information technology, and cognitive
science (NBIC convergence).
Key Characteristics:




Interdisciplinary Integration: Combining knowledge from multiple fields
Emergent Properties: New capabilities arising from integration
Accelerated Innovation: Faster technological development
Paradigm Shifts: Fundamental changes in approaches and possibilities
Convergence Drivers:




Miniaturization: Decreasing size of components
Digitization: Converting analog to digital formats
Networking: Increased connectivity and communication
Intelligence: Embedding AI capabilities
13.1.2 NBIC Convergence
Nanotechnology (N):




Scale: Manipulation at molecular and atomic levels
Materials: Novel properties at nanoscale
Applications: Electronics, medicine, energy
AI Integration: Intelligent nanomaterials and systems
Biotechnology (B):




Living Systems: Engineering biological processes
Applications: Medicine, agriculture, environmental
AI Integration: Bioinformatics, computational biology
Synthetic Biology: Design and construction of biological parts
Information Technology (I):




Computing: Processing and storage capabilities
Communication: Networking and connectivity
AI Integration: Machine learning, data analytics
Quantum Computing: Next-generation processing
82
Cognitive Science (C):




Brain Understanding: Neural mechanisms and processes
AI Development: Brain-inspired computing
Human Enhancement: Augmenting cognitive abilities
Brain-Computer Interfaces: Direct neural interaction
13.2 AI-Enabled Convergent Technologies
13.2.1 Cyber-Physical Systems (CPS)
Architecture:




Physical Components: Sensors, actuators, mechanical systems
Cyber Components: Computation, communication, control
Integration: Real-time interaction between cyber and physical
AI Integration: Intelligent decision making and adaptation
Key Features:




Real-time Processing: Time-critical system responses
Networked Systems: Distributed and connected components
Adaptability: Learning and evolution capabilities
Reliability: Fault tolerance and safety assurance
Applications:




Smart Manufacturing: Industry 4.0, intelligent factories
Autonomous Vehicles: Self-driving cars and drones
Smart Grids: Intelligent power distribution
Healthcare Systems: Patient monitoring and treatment
13.2.2 Internet of Things (IoT) and AI
IoT Architecture:




Device Layer: Sensors, actuators, embedded systems
Connectivity Layer: Communication protocols and networks
Data Processing Layer: Edge and cloud computing
Application Layer: User interfaces and services
AI Integration in IoT:



Edge AI: Local intelligence at device level
Predictive Analytics: Forecasting and anomaly detection
Automated Decision Making: Autonomous system responses
83

Natural Language Interfaces: Voice and conversational AI
AIoT Applications:


Smart Cities: Traffic management, energy optimization
Industrial IoT: Predictive maintenance, quality
84
CHAPTER 14: NEURAL NETWORKS AND DEEP LEARNING
Deep Learning is a specialized subfield of Machine Learning that uses Artificial Neural Networks
(ANNs) with multiple processing layers to learn representations of data with multiple levels of
abstraction.
14.1 Foundations of Neural Networks
14.1.1 The Artificial Neuron (Perceptron)
Figure 14.1: Artificial Neuron
The basic building block of any ANN is the artificial neuron, often called a Perceptron (for singlelayer networks). It is a mathematical model inspired by the structure and function of a biological
neuron.
Inputs (x): Data features or outputs from previous neurons.
Weights (w): Numerical values associated with each input, representing the strength of the
connection.
Bias (b): A term added to the weighted sum, acting as a threshold that makes the neuron fire more
easily.
Summation Function (Net Input, z): Calculates the weighted sum of inputs and the bias:
85
The non-linearity is crucial for the network to learn complex, non-linear relationships.
14.1.2 Activation Functions
Activation functions introduce non-linearity, allowing ANNs to approximate any non-linear
function.
14.2 Architecture of Neural Networks
14.2.1 Layers and Topologies
Neural networks are organized into layers of interconnected neurons (nodes).
 Input Layer: Receives the raw data features (x).
 Hidden Layer(s): Layers between the input and output where feature transformations
occur. Their values are not directly observed (hence, "hidden").
 Output Layer: Produces the network's final result (prediction, y^).
 Shallow Neural Network: Contains one hidden layer.
 Deep Neural Network (DNN): Contains multiple hidden layers (typically 3 or more).
Deep networks learn increasingly abstract, hierarchical features.
Network Topologies:

Feed-Forward Networks (FFN): Connections only flow in one direction (input → hidden
output). No loops or cycles. This is the most common and simplest form (e.g., Multi-Layer
Perceptrons).
86

Recurrent Networks (RNN): Contain feedback loops, allowing information to persist across time
steps. Ideal for sequential data (e.g., text, time series).
14.2.2 Forward Propagation
The process of calculating the network's output for a given input.
1. Input x is fed into the input layer.
2. In each subsequent layer l::
3. The process repeats until the output layer is reached, yielding the prediction
.
87
the direction of steepest increase in the cost.
1. Parameter Update: Move the parameters in the opposite direction of the gradient to minimize the
cost.
o
o
o
Learning Rate (η): A hyperparameter that controls the size of the step taken in the direction of the
minimum.
Stochastic Gradient Descent (SGD): Uses one training example at a time.
Mini-Batch Gradient Descent: Uses a small batch of examples (the most common and effective
method).
14.3.3 Back propagation
The efficient algorithm used to calculate the gradient (∇ J) include.
1. Backward Pass: The error signal (the difference between y^ and y) is propagated backward from
the output layer through the network.
2. Chain Rule: Backpropagation applies the chain rule of calculus to compute the derivative of the
cost function with respect to each weight in the network.
3. The calculated derivatives are then used by Gradient Descent to update the weights.
Deep Learning and Advanced Architectures
Deep Learning leverages multiple layers to automatically extract hierarchical features from data,
eliminating the need for manual feature engineering.

Why Deep? Earlier layers learn simple features (e.g., edges, textures), while deeper layers
combine these to learn complex, abstract concepts (e.g., eyes, noses, or full object classes).
88
Types of Deep Neural Networks
(a) Feedforward Neural Networks (FNNs).
(b) Convolutional Neural Networks (CNNs)
(c) Recurrent Neural Networks (RNNs)
Figure shows Side-by-side comparison of FNN, CNN, and RNN architectures.
(d) Autoencoders
 Learn efficient data representations (encodings).
 Encoder compresses input → latent vector → decoder reconstructs data.
 Used for noise reduction, dimensionality reduction.
(e) Generative Adversarial Networks (GANs)



Two models:
o Generator: Produces fake data.
o Discriminator: Distinguishes real from fake.
Trains adversarially until generator fools discriminator.
Used for image synthesis, data augmentation, art generation.
Figure 14.8: GAN architecture showing generator–discriminator competition.
14.4 Optimization Techniques
14.4.1 Convolutional Neural Networks (CNNs) Its primary use for Image processing, computer
vision. The Key Components include:
Convolutional Layers: Use filters (kernels) to learn local patterns in the input data (e.g., edge
detection).
Pooling Layers: Down sample the feature maps, reducing dimensionality and computation while
making features more robust to location shifts.
Fully Connected Layers: Perform final classification based on the learned features.
14.4.2 Recurrent Neural Networks (RNNs) Its primary use include Sequential data (Natural
Language Processing, time series, speech).
Mechanism: Have internal memory loops that allow the output of the current step to influence the
89
input of the next step.
Challenges: Standard RNNs suffer from vanishing/exploding gradients over long sequences.

o
o
Advanced RNNs:
Long Short-Term Memory (LSTM): Uses internal "gates" (input, forget, output) to regulate the
flow of information, effectively managing long-term dependencies.
Gated Recurrent Unit (GRU): A simpler, two-gate variant of the LSTM.
14.4.3 Transformers


Primary Use: State-of-the-art for NLP (e.g., BERT, GPT).
Mechanism: Rely entirely on an Attention Mechanism to weigh the importance of different parts
of the input sequence, replacing the sequential processing of RNNs. This allows for massive
parallelization and handling of very long-range dependencies, powering Large Language Models
(LLMs).
Applications of Deep Learning
Domain
Computer Vision
Applications
Object detection, face recognition, autonomous vehicles
Natural Language Processing Machine translation, chatbots, sentiment analysis
Speech Recognition
Voice assistants, transcription
Healthcare
Disease detection, medical imaging
Finance
Fraud detection, stock forecasting
Agriculture
Crop monitoring, pest detection
Sample Quiz
1. 1 What is the main difference between ANN and CNN?
2. Explain the purpose of an activation function.
3. Describe the steps of backpropagation.
4. What role does the learning rate play in training?
5. Differentiate between LSTM and GRU.
6. What are GANs used for?
7. Name two common challenges in deep learning systems.
8. Explain the concept of overfitting and how dropout helps prevent it.
9. List three real-world applications of deep learning.
10. What is the difference between supervised and unsupervised deep learning?
.
90
CHAPTER 15: INTRODUCTION TO PYTHON AI LIBRARIES
15.1 Overview
Python is the most widely used language in Artificial Intelligence (AI) because it is:




Easy to learn and read
Rich in AI-focused libraries
Community-driven with strong documentation
Supports integration with C/C++ and GPU computing
AI applications such as image recognition, speech processing, and predictive analytics rely heavily
on Python libraries.
15.2 Categories of AI Libraries
Category
Purpose
Examples
Machine Learning
Model training, testing, and evaluation Scikit-learn, XGBoost
Deep Learning
Neural networks and GPU computation TensorFlow, PyTorch, Keras
Data Processing
Data manipulation and cleaning
NumPy, Pandas
Visualization
Data visualization and plotting
Matplotlib, Seaborn
NLP
Text and language processing
NLTK, spaCy, Transformers
Reinforcement Learning Training agents via rewards
OpenAI Gym
Computer Vision
Image processing and analysis
OpenCV, Pillow
15.3 Core Libraries
15.3.1 NumPy – Numerical Computing
NumPy (Numerical Python) provides high-performance multi-dimensional arrays and
mathematical operations.
Example 1 – Array Creation
import numpy as np
arr = np.array([1, 2, 3, 4, 5])
print("Array:", arr)
Example 2 – Matrix Operations
A = np.array([[1, 2], [3, 4]])
B = np.array([[5, 6], [7, 8]])
print("Matrix Product:\n", np.dot(A, B))
91
Figure 15.1 NumPy arrays as a matrix grid used in AI computations.
15.3.2 Pandas – Data Handling
Pandas is used for data cleaning, transformation, and analysis using DataFrame objects.
Example – Reading and summarizing data
import pandas as pd
data = pd.DataFrame({
"Age": [21, 25, 30],
"Score": [88, 92, 95]
})
print(data.describe())
Example – Handling Missing Data
data.loc[1, 'Score'] = None
data.fillna(data['Score'].mean(), inplace=True)
print(data)
15.3.3 Scikit-learn – Machine Learning Made Simple
Scikit-learn provides a wide range of machine learning algorithms and evaluation tools.
Example – Linear Regression
from sklearn.linear_model import LinearRegression
import numpy as np
92
X = np.array([[1], [2], [3], [4]])
y = np.array([3, 5, 7, 9]) # Simple linear relation y = 2x + 1
model = LinearRegression().fit(X, y)
print("Prediction for x=5:", model.predict([[5]]))
Example – Model Evaluation
from sklearn.metrics import mean_squared_error
y_pred = model.predict(X)
print("MSE:", mean_squared_error(y, y_pred))
15.3.4 TensorFlow – Deep Learning Framework
TensorFlow enables large-scale neural networks using GPU acceleration.
Example – Basic Neural Network
import tensorflow as tf
model = tf.keras.Sequential([
tf.keras.layers.Dense(8, activation='relu', input_shape=(4,)),
tf.keras.layers.Dense(1)
])
model.compile(optimizer='adam', loss='mse')
model.summary()
Example – Training
import numpy as np
X = np.random.rand(100, 4)
y = X.sum(axis=1)
model.fit(X, y, epochs=5)
15.3.5 PyTorch – Dynamic Computation Graphs
PyTorch provides flexibility for research-based deep learning with dynamic graph execution.
Example – Linear Layer
import torch
import torch.nn as nn
x = torch.randn(1, 10)
layer = nn.Linear(10, 3)
93
output = layer(x)
print("Output:", output)
Example – Activation Function
relu = nn.ReLU()
print("After ReLU:", relu(output))
Figure 15.2: Forward propagation flow in PyTorch (input → layer → activation → output).
15.3.6 Keras – Simplified Deep Learning
Keras is a user-friendly API for building neural networks (now integrated with TensorFlow).
Example – Classification Network
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
model = Sequential([
Dense(8, activation='relu', input_shape=(4,)),
Dense(3, activation='softmax')
])
model.summary()
15.3.7 OpenCV – Computer Vision
OpenCV (Open Computer Vision) supports image and video processing.
Example – Reading and Converting Images
import cv2
img = cv2.imread('catfish.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
cv2.imshow('Gray Image', gray)
cv2.waitKey(0)
94
Example – Edge Detection
edges = cv2.Canny(gray, 100, 200)
cv2.imshow('Edges', edges)
cv2.waitKey(0)
15.4 Visualization Libraries
Matplotlib
Used for 2D plotting (line graphs, scatter plots, histograms).
Example – Simple Plot
import matplotlib.pyplot as plt
x = [1, 2, 3, 4]
y = [10, 20, 25, 30]
plt.plot(x, y)
plt.title("Simple Line Plot")
plt.show()
Seaborn
Provides statistical and aesthetic plots.
Example – Histogram
import seaborn as sns
import numpy as np
sns.histplot(np.random.randn(1000))
15.5 Emerging Libraries
Field
Library
Description
NLP
Hugging Face Transformers Pre-trained models (BERT, GPT, etc.)
Reinforcement Learning OpenAI Gym
Simulated training environments
Explainability
SHAP, LIME
Model interpretability tools
AutoML
Auto-sklearn, TPOT
Automated ML pipelines
Example – Using Transformers
from transformers import pipeline
classifier = pipeline("sentiment-analysis")
95
print(classifier("Python AI libraries are amazing!"))
15.6 AI Workflow Integration Example
Workflow Steps:
1.
2.
3.
4.
5.
Load data using Pandas
Preprocess with NumPy
Train model using Scikit-learn
Evaluate performance
Visualize results using Matplotlib
Example:
import pandas as pd
from sklearn.linear_model import LinearRegression
import matplotlib.pyplot as plt
data = pd.DataFrame({"x": [1,2,3,4,5], "y": [3,4,2,5,6]})
X, y = data[['x']], data['y']
model = LinearRegression().fit(X, y)
plt.scatter(X, y)
plt.plot(X, model.predict(X), color='red')
plt.show()
Python
provides
a
comprehensive
ecosystem
for
AI
development.
From data manipulation (NumPy, Pandas) to model training (Scikit-learn, TensorFlow,
PyTorch) and visualization (Matplotlib, Seaborn), it enables rapid prototyping and deployment
of AI systems.
Class Quiz
1. Briefly explain what makes Python suitable for Artificial Intelligence development.
2. What is the main difference between NumPy and Pandas?
3. State two features of the Scikit-learn library.
4. What is the role of TensorFlow in deep learning applications?
5. Mention two advantages of using PyTorch for AI research.
6. What type of problems can OpenCV solve in Artificial Intelligence?
7. Differentiate between Matplotlib and Seaborn in terms of visualization.
8. What is Keras, and how is it related to TensorFlow?
9. Name any two Python libraries used for Natural Language Processing (NLP).
10. Describe the general workflow of an AI project using Python libraries.
96
97