History of AI
Prof. Laxmidhar Behera
Director, IIT Mandi
Birth of artificial intelligence (1930-40) –
Connecting Biology and Computation
• 1930-1940 - The Human Brain as an Electrical Network
Research in neurology - Brain was an electrical network of neurons that fired in all-or-nothing
pulses. Neurons will either transmit an impulse over the synapse to the next neuron
completely or not at all.
•
1936 - The Birth of Digital Computation
Alan Turing - Computation could be performed using a simple machine
that operated using only two states: 0 and 1
Turing's "machine" could theoretically perform any calculation that a
computer can perform.
•
1940s : Information Theory and Digital Signals
Claude Shannon - Developed information theory, how information could be represented and
transmitted using digital signals, which, again, are based on 'on' and 'off' states (0s and 1s).
• The close relationship between these ideas suggested that it might be possible to construct an "electronic brain".
Birth of artificial intelligence (1941-56) McCulloch and Pitts model
• 1943 - Warren McCulloch and Walter Pitts - A simplified mathematical
model of how the neurons in our brains operate
A neuron model that sums binary inputs and
outputs a 1 if the sum exceeds a certain
threshold value and otherwise outputs a 0.
𝑦𝑜𝑢𝑡 = ቊ
where
1,
0,
𝑦𝑠𝑢𝑚 ≥ 𝜃
𝑦𝑠𝑢𝑚 < 𝜃
is the threshold
Birth of artificial intelligence (1941-56) - Early
Autonomous Robots – Pre-Digital Era
Late 1940s:
Grey Walter's Turtles
• W. Grey Walter, a neurophysiologist, built experimental robots
• To understand how simple biological systems, like the nervous
systems of simple animals, could produce complex behaviors
• Instead of using digital computers (which use 0s and 1s), Walter's
turtles were controlled by analog circuitry
• They could navigate towards light sources, avoid obstacles
The Hopkins Beast
•Used a combination of sensors (sonar and photocells) and
analog circuitry to navigate and recharge itself.
Birth of artificial intelligence (1941-56) –
Hebbian learning
• 1949 - Donald Hebb – Hebbain theory -- “Neurons that fire together, wire
together’’ - if two neurons are active simultaneously, the connection between
them should be strengthened.
Presynaptic
neuron
Postsynaptic
neuron
is the change in the synaptic weight between the i-th presynaptic neuron and
the j-th postsynaptic neuron.
is the activation of the i-th presynaptic neuron.
is the activation of the j-th postsynaptic neuron.
Birth of artificial intelligence (1941-56) –
Hebbian learning – An example
Scenario: We have two "neurons" (or feature detectors):
• Neuron A: Detects vertical lines.
• Neuron B: Detects horizontal lines.
• We present the system with several handwritten images of digits.
Example 1: The Digit "1"
• 𝑥𝐴 = 1, 𝑥𝐵 = 0.1 → ∆𝑤𝐴𝐵 = η * 1 * 0.1 = 0.1η
Neuron A:
Vertical lines
Neuron B:
Horizontal lines
• The weight is increased slightly.
Example 2: The Digit “7"
• 𝑥𝐴 = 0.7, 𝑥𝐵 = 0.8 → ∆𝑤𝐴𝐵 = η * 0.7 * 0.8 = 0.56η
• The weight is increased slightly more than in the case of the "1“.
Example 3: The Digit “4"
• 𝑥𝐴 = 0.9, 𝑥𝐵 = 0.9 → ∆𝑤𝐴𝐵 = η * 0.9* 0.9 = 0.81η
• The weight is increased even more.
• Hebbian learning : Neurons in the visual cortex adapt to prefer specific orientations of lines or edges based on the patterns of
visual input they receive during development. This process strengthens connections between neurons that are repeatedly
activated together, helping the brain interpret visual stimuli.
Birth of artificial intelligence (1941-56) - Turing Test
(Can machines Think)
1950: Alan Turing published “Computing Machinery
and Intelligence”, proposing the Turing Test to define
"thinking machines.
The Turing Test involves a human interrogator
engaging in a natural language conversation with one
human and one machine—both hidden from view.
If the interrogator cannot reliably distinguish the
machine from the human, the machine is considered
to have passed the test—human-like intelligence.
Birth of artificial intelligence (1941-56) First AI Game Programs
1951 – Inspired by the vision of Turing, Strachey and Prinz
created the first AI programs for checkers and chess,
simulate human-like decision-making using brute-force
search on all the legal moves and basic decisionmaking.
1952 - Arthur Samuel developed the first computer
learning program for checkers, which improved by
analyzing winning moves, marking an early milestone in
machine learning.
Birth of artificial intelligence (1941-56) - The
Dartmouth Conference
Finally in 1956, the term “artificial intelligence’’ was coined by John
McCharthy at a Dartmouth Conference, and AI became a formal field of
study.
Attendees: John McCharthy, Marvin Minsky, Claude Shannon, Nathaniel
Rochester, Arthur Samuel, Trenchard More, Ray Solomonoff, Oliver
Selfridge, Allen Newell, Herbert Simon.
They agreed to bring together experts interested in neural networks, the
theory of computation and automata theory.
To explore if machines could simulate various aspects of human
intelligence.
Early Successes in AI (1956–1974) –
Perceptron model
• 1958 - Frank Rosenblatt introduced the perceptron, a single-layer neural network that extends the
McCulloch-Pitts model by incorporating a learning rule for adjusting weights
• Rosenblatt drew inspiration from Hebb's work because the idea of adjusting synaptic strengths based on
neuronal activity was a key biological inspiration for early neural networks
• He predicted its future ability to learn and translate languages.
• For each training input–output pair (𝑥, 𝑦), where 𝑥 = 𝑥1 , 𝑥2 , … , 𝑥𝑛
Error = 𝑦 − 𝑦𝑜𝑢𝑡
Compute the activation:
𝑦𝑠𝑢𝑚 = ∑𝑤𝑖 𝑥𝑖 + 𝑏,
Apply the decision rule based on threshold:
𝑦𝑜𝑢𝑡 = ቊ
1,
0,
𝑦𝑠𝑢𝑚 ≥ 𝜃
𝑦𝑠𝑢𝑚 < 𝜃
Update weights:
𝑤𝑖 ← 𝑤𝑖 + 𝜂(𝑦 −𝑦𝑜𝑢𝑡 )𝑥𝑖
Early Successes in AI (1956–1974) –
Perceptron model
A single perceptron can learn any
function, as long as the dataset is
linearly separable, like AND, OR,
NAND, and NOR!
BUT not for nonlinearly separable
such as XOR!!!
Early Successes in AI (1956–1974) - Early AI
Funding and Paradigms
In late 1950s–Early 1960s
• Increased Funding: Large-scale funding began for AI research.
• DARPA's Role: The US Defense Advanced Research Projects Agency (DARPA), created in response to Sputnik, became a
major funding source. This funding supported research at universities like MIT, Carnegie Mellon, and Stanford.
Two Dominant Paradigms Emerged
Symbolic AI
(Logic-Driven)
Connectionist AI
(Neural Networks)
Represents knowledge with
symbols and logical rules.
Represents knowledge as connections
between nodes (neurons) in a network
Replicates intelligence by
manipulating these symbols using
predefined rules.
Replicates intelligence by learning patterns
and relationships in data through adjusting
the weights of these connections
Examples include theorem proving, chess
playing, and grammar correction.
Examples include image recognition, natural
language processing, and speech recognition
Early Successes in AI (1956–1974): Logic
Theorist – Symbolic AI
The Logic Theorist (Simon & Newell, 1956) was an early symbolic AI
program designed for automated theorem proving
It employed heuristics (rules of thumb), specifically "means-ends
analysis" (comparing the current state to the goal and applying
operators to reduce the difference), to efficiently navigate the
search space, avoiding brute-force search
The program successfully proved 38 of the first 52 theorems from
Russell and Whitehead's Principia Mathematica.
Early Successes in AI (1956–1974): Logic
Theorist – An example
Tower of Hanoi Problem:
Move all the disks from the source tower to the
destination tower while maintaining their order
•Move one disk at a time.
•A larger disk cannot be placed on top of a smaller disk.
Goal state
A brute-force approach would explore every possible move at each step
The number of possibilities grows exponentially as the number of disks grow.
A simple heuristic: "Focus on moving the largest unsolved disk by repositioning smaller disks to support it” solves in less number of steps
Heuristics is a key concept in the history of AI and a central contribution of the Logic Theorist
Early Successes in AI (1956–1974): LISP
LISP (List Processing) programming language:
One of the earliest high-level programming languages.
It introduced concepts like symbolic computation, recursion,
and automatic memory management, which were revolutionary
at that time.
Task: Add 2 and 3 and store the result
Feature
Low-Level
High-Level
Abstraction
Very low; close to the hardware
High; abstracts away hardware
details
Complexity
Complex; requires understanding of
CPU architecture
Simple; focuses on the logic of the
problem
Readability
Difficult for humans to read and
understand
Easy for humans to read and
understand
Portability
Not portable; machine-specific
Portable; can run on different
machines
Memory Management
Manual; programmer is responsible
for memory allocation
Automatic (garbage collection in Lisp)
Speed (Potentially)
Can be very efficient
Might be slightly less efficient in some
cases (but the difference is often
negligible)
Machine Code
0001: LOAD (load a value into a register - a small storage location in the CPU )
0010: ADD (add two registers)
0011: STORE (store a register's value to memory)
2: 0010
3: 0011
0001 0000 0000 0010 ; LOAD 2 into Register 0 (0000 is the register number)
0001 0001 0000 0011 ; LOAD 3 into Register 1 (0001 is the register number)
0010 0010 0000 0001 ; ADD Register 0 and Register 1, store in Register 2 (0010 is the destination register)
0011 0010 0000 1000 ; STORE Register 2 into memory address 8 (1000 is the memory address in binary)
Assembly Language
LOAD 2, Register1 (Load the value 2 into a register)
LOAD 3, Register2
ADD Register1, Register2, Register3
STORE Register3, 1000 (Store the contents of Register3 into a memory location (let's say
address 1000)
LISP Language
(+ 2 3) ; This directly evaluates to 5
or to store it in a variable: (setf result (+ 2 3))
Assembly language was the initial programming method for early
microcontrollers (e.g., Intel 8051, Motorola chips), followed by
Lisp for specialized, often AI-related, microcontroller applications.
Early Successes in AI (1956–1974):
ELIZA - Early Attempts at NLP
- First program capable of attempting the Turing test.
- Developed by Joseph Weizenbaum at MIT.
- Simulated a Rogerian psychotherapist.
- Used simple pattern matching and substitution to generate responses.
- Examples
- “I am *” → “Why are you *”
- “You are *” → “What makes you think I am *”
- “Everyone *” → “Who is an example?”
- ELIZA lacked any real understanding of language, context, or the world.
Towards MICROWORLDS:
• Since the real world included too many hidden variables for early AI to feasibly interpret, the next best thing was to
create AI that could operate within restricted, digital worlds
• In the late 60’s, Marvin Minsky et al. suggested that the artificial intelligence community shift their research to working
in microworlds.
Early Successes in AI (1956–1974): SHRDLU: A
Major Advance in Natural Language Understanding
- Developed by Terry Winograd at MIT.
- Operated in the blocks world micro-world.
- Showed that computers could understand and act based on natural language
within a limited domain.
- Examples
- User: Find a block which is taller than the one you are holding and put it
into the box.
- SHRDLU: BY "IT" I ASSUME YOU MEAN THE BLOCK WHICH IS TALLER
THAN THE ONE I AM HOLDING.
- User: Is there a block which is supported by a pyramid?
- SHRDLU: YES, THE BLUE BLOCK.
- User: Did you pick up a pyramid in the past?
- SHRDLU: YES, I DID.
- Limitations: Its understanding was limited to the blocks world. It lacked
common sense and the ability to generalize to other domains.
Early Successes in AI (1956–1974): Learning
Algorithms
1969
• Minsky and Papert published the book Perceptrons, highlighting limitations of
single-layer neural networks, halting connectionist research for a decade,
contributing to the first AI winter.
• Example: Unable to represent the XOR (exclusive-or) operation.
• Multi-layer networks were not studied due to lack of learning algorithms.
1974
• Paul Werbos's 1974 Ph.D. thesis introduced backpropagation, ordered
derivatives, and laid groundwork for feedforward/feedback networks and
adaptive dynamic programming, but these pioneering neural network concepts
went largely unrecognized until the 1980s. His work, though initially
overlooked due to computational limitations and prevailing research trends,
became foundational to modern deep learning and AI.
• Backpropagation provided a method to train multi-layer networks by
computing error gradients through layers.
Early Successes in AI (1956–1974) - Backpropagation
• 1974 - Backpropagation, invented by Paul Werbos, gained attention only after the 1986 paper by David Rumelhart, Geoffrey
Hinton and Ronald Williams.
Input layer
hidden layer
output layer
Forward propagation is the process of pushing inputs through the net
The sigmoid (logistic function) is one of the most common non-linearities
Early Successes in AI (1956–1974) - Backpropagation
Cost(or Loss) function: It is a function of the difference between estimated (y) and true values (t) for the data
Central Idea - Compute the gradient of the loss function. Decrease the loss by iteratively
adjusting the weights and biases based on the obtained gradients, and the network gradually
learns to make better predictions
Input
layer
hidden
layer
output
layer
Update rule for a single weight
for the hidden to output layer
Errors for hidden to output layer
Update rule for a single weight
for the input to hidden layer
Errors for input to hidden layer
Errors are propagated backward
through the network by calculating
the gradient of the loss function
with respect to each weight using
the chain rule of calculus.
First AI Winter (1974–1980) - Challenges
• Early 1970s: Challenges and Critiques
• AI programs of the early 1970s struggled to handle complex, real-world problems. They were mostly confined
to solving puzzles, playing simple games, or operating in highly constrained environments
• Insufficient memory and processing power limited scalability (NLP used only 20 words)
• Computational resources required by symbolic AI algorithms increased exponentially with the size of the
problem
• AI required vast amounts of real-world data, unattainable with 1970s technology
• 1973: Funding Cuts
• The British government commissioned a report by Sir James Lighthill to evaluate the progress of AI research:
Report criticized AI’s lack of progress, leading to funding reductions.
• DARPA Cuts: Disappointed by slow progress, ended major grants like the Speech Understanding Research
program.
• 1980 - John Searle presented the Chinese Room argument, questioning AI’s ability to "understand" symbols.
First AI Winter (1974–1980) – Chinese Room
Argument
• 1980 - John Searle presented the Chinese Room argument, a powerful
critique against the idea that computers can truly "understand" in the same
way humans do
• The Setup: Imagine a person who doesn't understand Chinese is locked
in a room. This room contains:
• A large collection of Chinese symbols.
• A detailed rule book (in English) that explains how to manipulate these
symbols. The rules specify which symbols to give as output in response to
certain input symbols, without requiring any understanding of their meaning.
• Slots to receive input symbols and provide output symbols to the outside
world.
•
•
•
Syntax vs. Semantics: Computers excel at syntax but lack the semantic understanding required for true cognition.
Strong AI vs. Weak AI: Challenges "strong AI" (computers possessing true understanding). Supports "weak AI"
(computers as tools simulating cognitive abilities).
Mind-Body Problem: Argues understanding may require more than computation—possibly a biological basis like
human consciousness.
Few concepts to learn
Machine Learning Paradigms
Supervised
Unsupervised
Reinforcement learning
Learning using labeled data, where
each input has a corresponding
output. Goal is to learn their
relationship and generalize to
unseen data
Learning using unlabeled data. The
goal is to find hidden patterns,
structures and groupings in the
data
Doesn’t learn from labeled or
unlabeled data. An agent learns to
make decisions by interacting with
environment to maximize a
cumulative reward.
Examples: Classifying the images,
Stock market prediction, Medical
Diagnosis, Sentiment Analysis
Examples: Grouping customers
based on purchasing behaviors,
Reducing high-dimensional data
for visualization
Examples: Alpha Go, Chess and
Atari Games. Self-driving cars
optimizing driving decisions.
Supervised Learning Workflow
This example is for the problem of
binary classification, a supervised
learning problem
Feature extraction converts raw inputs to a
numeric representation that the ML algo
can understand and work with.
Unsupervised Learning Workflow
This example is for
the problem of data
clustering, an
unsupervised
learning problem
Reinforcement Learning Workflow
EXAMPLE
Reinforcement Learning Workflow
Adaptive Critic Design (ACD) : An example of Reinforcement Learning
1. Actor (Controller) 𝝅(𝒔) : Selects actions 𝑎𝑡
2. Plant & Model: The environment transitions to new state 𝑠𝑡+1 ​ and gives reward 𝑟𝑡
3. Critic 𝑽(𝒔): Evaluates 𝑎𝑡 ​ and improves the actor 𝜋(𝑠)
• Policy Update (Actor):
𝜋 𝑠 ← 𝜋(𝑠) + 𝛼 ∇𝜋 𝐽(𝜋)
where 𝐽 𝜋 is the expected cumulative reward.
• Value Function (Critic):
γ is the discount factor in reinforcement learning. It determines how
much future rewards contribute to the present value.
Geometric Perspectives of AI - Regression
Mean temperature in Delhi over years
Examples:
•Weather Forecasting: Predicting tomorrow's temperature based on historical
weather data (temperature, humidity, wind speed, etc.).
•Power Load Demand Forecasting: Estimating future electricity consumption
based on past usage patterns, time of day, and weather conditions.
•Control Problems: Determining the optimal control input (u) for a system
given its current state (e.g., adjusting the thermostat in a room to maintain a
desired temperature).
Optimal Control
Power load forecasting
Geometric Perspectives of AI
Examples:
•Spam Detection: Classifying emails as spam or not spam.
•Medical Diagnosis: Determining if a patient has a certain disease
based on their symptoms and test results.
•Fault Detection (Time Series): Classifying the operational state of
a machine (e.g., normal vs. faulty) based on sensor readings over
time with models like ARMA and NARMA.
1980s: The Rise of Artificial Neural Networks and
Second AI Winter - Neocognitron
In 1980, Kunihiko Fukushima introduced the
Neocognitron, a groundbreaking and first hierarchical
multilayered neural network designed for visual pattern
recognition.
Significantly influenced the development of
Convolutional Neural Networks (CNNs)
Neurons in a given layer are connected only to a small,
local region of the previous layer – this region is the
receptive field
This is analogous to how neurons in the visual cortex
respond to stimuli in specific regions of the visual field.
1
0
1
0
1
0
1
0
1
Kernel weights
1980s: The Rise of Artificial Neural Networks and
Second AI Winter – Expert Systems
Expert systems are computer programs designed to emulate the knowledge
and reasoning of human experts within a particular domain
They typically consist of:
•Knowledge Base: A repository of domain-specific knowledge, often
represented as a set of rules (e.g., "IF condition THEN action"), facts, and
heuristics (rules of thumb).
•Inference Engine: A mechanism that applies the knowledge base to reason about new situations and draw
conclusions. This often involves techniques like forward chaining (starting with known facts and applying rules to
derive new facts) or backward chaining (starting with a goal and working backward to find supporting evidence).
•User Interface: A way for users to interact with the system, providing input and receiving advice or explanations.
1980s: The Rise of Artificial Neural Networks and
Second AI Winter – Expert Systems
Example - XCON (eXpert CONfigurer), developed at Digital Equipment Corporation
(DEC), was a highly successful expert system
DEC's VAX minicomputers were highly configurable, offering customers a wide
range of options for CPUs, memory, storage devices, peripherals.
Human technicians, even experienced ones, made errors in the configuration
process, leading to costly rework, delays in delivery, and dissatisfied customers
VAX minicomputer
Knowledge Base: Contained thousands of rules derived from human experts, covering component compatibility
and other configuration constraints.
Inference Engine: Used these rules to analyze orders and generate accurate configurations.
Output: Provided detailed assembly instructions.
Impact: Cut configuration errors, saving millions. Proved AI's value in solving business problems
1980s: The Rise of Artificial Neural Networks and
Second AI Winter – Expert Systems
UK nuclear power plants were adopters of expert systems
(1980s-90s) to manage plant complexity and enhance
safety through applications like
•Fault Diagnosis: To diagnose faults in plant equipment
and provide operators with guidance on how to resolve
them.
•Alarm Processing: Nuclear plants generate a large
number of alarms, which can overwhelm operators.
Expert systems helped to filter and prioritize alarms,
ensuring that operators focused on the most critical
issues.
•Process Control: To monitor and control various plant
processes, such as reactor power, coolant flow, and
waste management.
1980s: The Rise of Artificial Neural Networks and
Second AI Winter – Hopfield Network
1982 - Hopfield Network: John Hopfield demonstrated a different way of thinking about
neural networks, focusing on associative memory and pattern retrieval rather than
supervised learning for classification.
They consist of a single layer of interconnected neurons where each neuron is connected to
every other neuron. The network depend on a single input and their previous output
using recurrent weights.
The energy function associated with the
state of the network
Bit maps stored in a
Hopfield network.
Reconstruction of a pattern
from a random input.
1980s: The Rise of Artificial Neural Networks and
Second AI Winter – Boltzmann Machines
• In 1984, Geoffrey Hinton and Terry Sejnowski introduced Boltzmann Machines,
a type of recurrent neural network that incorporated probabilistic elements,
drawing inspiration from statistical physics.
• Unlike deterministic networks where neurons have definite on/off states,
Boltzmann Machines use probabilistic neurons.
• This stochastic (random) behavior allows the network to explore different states
and find better solutions.
• Here, each state
Boltzmann distribution
of the network is assigned a probability given by the
In this Boltzman machine,
there are 3 hidden units
and 4 visible units
The Boltzmann machine is a generative model.
Unlike the Hopfield model, it focuses on statistical distributions of patterns rather than individual patterns
1980s: The Rise of Artificial Neural Networks
and Second AI Winter – LeNet 5
• After backpropagation got popularized (Rumelhart & Hinton in 1986), Hornik et al.
proved multilayer feedforward networks are universal approximators, capable of
implementing any function (e.g., XOR).
• In 1989, Yann LeCun successfully demonstrated its practical use by implementing a
Convolutional Neural Network (CNN) using backpropagation and named it LeNet.
• Neurons use a small set of shared weights (e.g., 5x5=25) applied across the image
• Subsequent layers process local features (e.g., edges) to form higher-order features
(e.g., shapes).
• Each layer ‘sees’ progressively larger portions of the image.
• Fully connected layers classify the digit using features from convolutional layers.
• Recognized handwritten digits on U.S. postal service zip codes with this LeNet-5
architecture.
1980s: The Rise of Artificial Neural Networks
and Second AI Winter – Fuzzy logic
• Lotfi Zadeh (1965) laid the foundation for Fuzzy logic
• In 1988, fuzzy logic started to gain significant attention and practical application in the broader AI
community
• Concept: Unlike traditional Boolean logic, where values are strictly true or false (1 or 0), fuzzy logic allows
for degrees of truth. It introduces the concept of "fuzzy sets" where elements have a membership value
between 0 and 1, indicating their degree of belonging to a set
• For example, linguistic variables like "temperature" can be represented with fuzzy sets like "hot," "warm,"
and "cold," each with overlapping boundaries
1980s: The Rise of Artificial Neural Networks and
Second AI Winter – Self organizing maps (SOM)
Self organizing maps were discovered by Teuvo Kohonen
A Kohonen Self-Organizing Map (SOM) is a type of artificial neural network used
for dimensionality reduction, clustering, and visualization of high-dimensional
data.
Unsupervised Learning: SOMs learn patterns in data without needing labeled
examples.
How it Works
• Initialization: Each neuron in the SOM is assigned a random weight vector
with the same dimensionality as the input data.
• Competition: An input data point is presented to the SOM. The neuron with
the most similar weight vector (the "winner") is identified.
• Cooperation: The winning neuron and its neighboring neurons adjust their
weight vectors to become more similar to the input data point. The size of the
neighborhood decreases over time.
winning neuron (dark blue)
neighborhood (light blue & red)
nodes outside this neighborhood (green)
Second AI Winter (1987 – 1993)
1987: Market Collapse
Specialized AI hardware (e.g., Lisp machines) collapsed due to competition from faster
and cheaper desktop computers.
Late 1980s: Expert Systems' Decline
Expert systems (e.g., XCON) became impractical, requiring high maintenance and
lacking the ability to adapt or learn.
1980s–1990s: Reduced Funding
DARPA and other funding agencies cut AI investments, focusing instead on projects
with immediate results.
1991: Japan's Fifth Generation Project Fails
The project failed to meet ambitious goals, emphasizing the gap between AI
expectations and technological reality.
1993: Commercial Decline
•Over 300 AI companies shut down, marking the end of the first commercial wave
of AI.
1990s: Machine Learning Advances –
Reinforcement Learning
• 1992: Q-Learning by Christopher Watkins - groundbreaking off-policy
algorithm in reinforcement learning
• The goal is to learn the Q-value (action-value), which represents the expected
reward for taking a specific action by an agent in a specific state over time.
Q-learning works without needing a model of the environment, handles uncertainty and delayed outcomes,
and forms the foundation for modern AI applications like game-playing agents (e.g., AlphaGo) and robotics.
1990s: Machine Learning Advances – LSTM
• 1997 – Long Short Term Memory (LSTM): Sepp Hochreiter and Jürgen Schmidhuber introduced LSTMs,
enabling Recurrent Neural Networks to learn long-term dependencies by passing memory sequentially.
Recurrent Neural Networks (RNNs)
eg: John lives in ___( Delhi / book)
•RNNs make use of Sequential Information.
•The output of RNNs depends on the previous computations and hence have a “memory”.
Weight update at nth time step:
• Since the activation functions are tanh or sigmoid, their derivatives lie
b/w 0 and 1. Hence, the gradient values are shrinking exponentially
fast, eventually vanish after few time steps due to chain rule.
•Gradient contributions from “far away” steps don’t contribute.
•Thus you may not learn long range dependencies.
1990s: Machine Learning Advances – LSTM
•To solve the vanishing gradient problem, LSTMs introduced memory cells and gates that control the flow of
information:
Input gate: Determines how much new information to add to the memory.
Forget gate: Controls which information to discard from memory.
Output gate: Decides what information to pass to the next layer or output.
Impact:
Made RNNs capable of handling long-term context,
such as Natural language processing (NLP),
Speech recognition and Forecasting Time-series
1990s: Machine Learning Advances :
Applications of LSTM
•Speech Recognition (Google Voice, Siri, Alexa, 2014–2016)
LSTMs improved end-to-end speech recognition, replacing traditional Hidden
Markov Models (HMMs).
•Machine Translation (Google Translate, 2016)
Google switched from phrase-based translation to LSTM-powered neural
machine translation (NMT), leading to more fluent and contextually accurate
translations.
•Handwriting and OCR (2013–2018)
LSTMs enabled better handwriting recognition, especially in online and offline
handwriting datasets.
•Stock Market and Financial Forecasting (2016–present)
AI-driven trading systems integrated LSTMs to model financial time-series data,
improving market trend predictions.
Big data, Deep Learning, AGI (2005–2017)
• The early 2000s saw a renewed focus on achieving Artificial General Intelligence (AGI)—AI with human-level
cognitive abilities
• 2002: Ben Goertzel and others raise concerns about the lack of focus on
Artificial General Intelligence (AGI).
• 2004: Marvin Minsky organizes a symposium on “human-level AI”.
• 2006: Geoffrey Hinton coins the term “deep learning” to describe advanced
algorithms that use many layers of interconnected artificial neurons, enabling
computers to “see” and recognize objects and text in images and videos.
• 2007: Release of “Labeled Faces in the Wild”, an annotated dataset for
face recognition - 13,000 images of faces.
• 2008: Ben Goertzel found the AGI journal and conference series.
“Labeled Faces in the Wild” Dataset
Big data, Deep Learning, AGI (2005–2017)
• 2009: Fei-Fei Li releases ImageNet, a pivotal dataset for image processing research.
• A massive dataset of over 14 million images
ImageNet
• Very large-scale, with thousands of object categories and a large number of
images per category
• Revolutionized image recognition, enabling the training of deep learning models
that achieve human-level performance on many tasks
• 2010: Founding of DeepMind by Demis Hassabis, Shane Legg, and Mustafa
Suleyman.To teach AI how to play primitive games from the seventies and eighties,
including Breakout, Pong, and Space Invaders.
• 2010: The Microsoft Kinect can track 20 human features at a rate of 30 times per
second, allowing computer vision techniques to track human movements, demonstrating
the practical applications of AI.
DeepMind AI playing Atari games
Big data, Deep Learning, AGI (2005–2017) –
IBM’s Watson
• 2011: IBM Watson's victory over human Jeopardy! champions showcased the power of AI in NLP and question
answering.
•
What is Jeopardy!?
Jeopardy! is a popular American quiz show with a unique format:
"Answers" are given, and contestants must provide the "questions”.
For example, if the "answer" is “This American president gave the Gettysburg Address,”
the correct “question” is “Who is Abraham Lincoln?”
•
Watson was loaded with a vast amount of information, including encyclopedias, dictionaries, books, and news articles.
• 2011: Around the same time, Google developed Google Brain, a large-scale deep learning project. One of its early
achievements was training a deep neural network that could learn to recognize cats in YouTube videos without
explicit programming, demonstrating the power of unsupervised learning.
Big data, deep learning, AGI (2005–2017) AlexNet
• 2012: AlexNet and the ImageNet Revolution: Alex Krizhevsky, a student of
Geoffrey Hinton, developed AlexNet, a deep convolutional neural network (CNN),
which achieved a stunning victory in the ImageNet Large Scale Visual
Recognition Challenge (ILSVRC). This was a landmark moment for deep learning.
AlexNet
• AlexNet used multiple layers of convolutional filters, combined with techniques like
ReLU activation functions and dropout for regularization
• Relu helps mitigate the vanishing gradient problem by keeping gradients large
for positive inputs.
ReLU function
• Dropout Regularization prevents overfitting by randomly "dropping out" neurons
during training. Forces the network to not rely on specific neurons, leading to better
generalization.
Before and After Dropout
Big data, deep learning, AGI (2005–2017) – word2vec
2013: Tomos Mikolov at Google released word2vec, a technique for learning word
embeddings, which represent words as numerical vectors in a high-dimensional space,
capturing semantic relationships between words
Word2vec is trained using a neural network that learns the relationships between words in large
databases of texts
It laid the groundwork for later advancements like GloVe, FastText, and transformer-based models
(e.g., BERT, GPT).
Big data, deep learning, AGI (2005–2017) –
word2vec
• Embedding – how to represent words as vectors that carry meaning of that word
• Bag of words: Converting text into a vector based on the frequency of words in the text
• TF-IDF: importance of a term is high when it occurs a lot in a given document and rarely in
others
Big data, deep learning, AGI (2005–2017) –
word2vec
• Continuous Bag-of-Words (CBOW): The CBOW model is a neural network-based approach that
captures the context of words. It learns to predict the target word based on the words that appear before
and after it in a given context window.
Word embeddings from projection layer are plotted,
similar meaning words are closer in space, indicating
their semantic similarity.
Big data, deep learning, AGI (2005–2017) –
word2vec
• Skip-gram model: A neural network-based approach that captures word relationships by learning
to predict the surrounding context words given a target word.
Word embeddings from projection layer are plotted, similar meaning
words are closer in space, indicating their semantic similarity.
Big data, deep learning, AGI (2005–2017) – word2vec
•In 2014, Ian Goodfellow and his team introduced Generative Adversarial Networks (GANs), a novel
approach to training generative models.
•GANs consist of two neural networks, a generator and a
discriminator, that are trained simultaneously.
•The generator wants to fool the discriminator into believing that its
generated data is real.The discriminator wants to correctly classify
real and fake data.
GANs are powerful generative models that learn to create
realistic data, impacting fields from image synthesis and
data augmentation to healthcare and creative arts.
Big data, deep learning, AGI (2005–2017)
Elon Musk founds OpenAI to focus on
safe and beneficial AI development.
Concerns about superintelligence and
AI existential risks gain traction, with
works by Nick Bostrom and others.
Google acquires DeepMind with ethical
constraints on its operations
2014
2014
2015
Deep Face
2015
Facebook develops DeepFace, a
software algorithm that is able to
recognize or verify individuals on photos
to the same level as humans can.
Amazon launches its own machine
learning platform.
Big data, deep learning, AGI (2005–2017)
Google’s artificial intelligence algorithm beats a
professional player at the Chinese board game Go,
which is considered the world’s most complex board
game and is many times harder than chess. The AlphaGo
algorithm developed by Google DeepMind managed to
win five games out of five in the Go competition.
Microsoft creates the Distributed Machine Learning
Toolkit, which enables the efficient distribution of
machine learning problems across multiple computers.
2015
2015
2016
2016
Over 3,000 AI and Robotics researchers, endorsed by
Stephen Hawking, Elon Musk and Steve Wozniak
(among many others), sign an open letter warning of the
danger of autonomous weapons which select and engage
targets without human intervention.
Elon Musk
Investment in AI surpasses $8 billion; AI research
reaches a "frenzy." The "value alignment problem"
becomes a major focus in AI safety research.
Big data, deep learning, AGI (2005–2017) Transformer
• 2017 - Transformer: In 2017, a groundbreaking
paper titled "Attention is All You Need" by Vaswani et
al. at Google introduced the Transformer
architecture.
• RNNs and LSTMs process input sequentially
• Transformers, in contrast, process all tokens at
once, making them much faster.
• Transformers solve the vanishing gradient problem
by using the attention mechanism, which directly
connects every word to every other word in the
sentence, regardless of distance.
Big data, deep learning, AGI (2005–2017) Transformer
The attention mechanism is basically
1. assigning weights to and
2. extracting relevant context from other
tokens of a sequence based on their
relevance or importance to a target token
(i.e. attending to them)
Big data, deep learning, AGI (2005–2017) Transformer
1.What you want to look for is the
Query.
2.What you are searching among is
Key-Value pairs.
3.The query is compared to all the
Keys to measure the
relevance/importance/similarity.
4.The Values are utilized based on the
assigned similarity measure
Big data, deep learning, AGI (2005–2017) Transformer
So far in NLP, we only have one vector for each token which is its embedding vector.
So, how should we obtain the Query, Key, and Value vectors?
Query (Q): What is this word looking for?
Key (K): How relevant is this word to others?
Value (V): What information should this word pass along?
A token's embedding represents various
contextual, structural, and syntactical
aspects.
Learnable linear layers extract specific
information from the embedding for the
Q, K, and V vectors.
Big data, deep learning, AGI (2005–2017) Transformer
Dot Product (Q · Kᵀ): Measures similarity/relevance between Query (Q) and each Key (K).
Softmax: Converts scaled attention scores into a probability distribution (values between 0 and 1, sum to 1).
Attention Output: Multiplies attention weights by Values (V) to produce a weighted representation focused
on the most relevant information.
Transformers utilize multiple attention mechanisms in parallel to process
sequential data in parallel, capturing complex dependencies and rich
contextual relationships with remarkable efficiency. This innovation
underpins advanced LLMs like BERT, GPT, and Gemini, revolutionizing
natural language processing.
Large Language Models and AI Boom (2020–Present)
2020
2022
2023
OpenAI released GPT-3, marking
a milestone in natural language
processing.
Public adoption soared with
models like ChatGPT, reaching
over 100 million users rapidly.
Generative AI dominated 30% of
AI investments; OpenAI valued at
$86 billion, while NVIDIA’s
market cap exceeded $3.3 trillion
by mid-2024.
AI investments surged, growing
from $18 billion in 2014 to $119
billion.
GPT-4 demonstrated near-human
general intelligence, fueling
debates on AGI.
2021
2023
Large Language Models and AI Boom (2020–
Present)
Ethical Concerns: Expansion of AI sparked debates on risks like job
automation and misinformation.
2023 Open Letter: AI experts and tech leaders called for a six-month pause
on training models beyond GPT-4 to address risks.
Industry Response: ChatGPT's success drove rapid competition from Google
(Gemini) and Microsoft (Bing Chat).
DeepSeek, a Chinese AI firm, disrupts the industry with its low cost, open
source R1 model, challenging U.S. tech giants and shaking up the AI market.
Timeline of AI
Image source: https://images.app.goo.gl/4V2wdCTtcnd9y69h8
Thank you