Università degli Studi di Genova
Dipartimento di Ingegneria Biofisica ed Elettronica
Sistemi di Radiocomunicazione
From Software Defined Radio
to Cognitive Radio
Prof. C.S. Regazzoni
DIBE
Outline
•
Introduction



•
From Software Defined Radio to Cognitive Radio
Software Defined Radio vs Cognitive Radio
Cognitive Radio
Cognitive Cycle




MItola’s CC, Haykin’s CC, simplified CC
Knowledge representation
Embodied Cognition
Bio-inspired Model
2
Outline
• Application examples in CR
 E2R
 Infomobility framework
Cognitive Cycle in practice
Analysis Phase
Decision Phase
Results
Remarks
Introduction
From SDR to CR – SDR Technologies
• Historically, radios have been designed to perform a
given task
• As upgrades were desidered to increase capability,
reduce life cycle costs, and so forth, software was
added to the system design for increased flexibility
• In 2000 a SDR has been defined by FCC as:
A communication device whose attributes and
capabilities are developed and/or implemented in
software
From SDR to CR – SDR Technologies
• The required additional flexibility and addiotional capabilities
have been provided step by step
• The radio system capabilities can evolve to accomodate a much
broader range of awareness, adaptivity and learning
Software Capable Radio
• A software capable radio has the following
characteristics




Fixed modulation capabilities
Manage a small range of frequencies
Limited data rate
Ability to handle data under software control
Software Programmable Radio
• A software programmable radio has been designed
upon a software capable radio and it has the
following additional characteristics:
 Ability to add new functionalities through software
 Advanced networking capabilities
Software Defined Radio
• Software Defined Radio (SDR) systems main
characteristic is the complete adjustability through
software of all radio operating parameters.
• Required reconfigurability is provided thanks to
software management of the considered system
• It is a practical reality today, thanks to the
convergence of two key technologies: digital radio,
and computer software.
Aware, Adaptive and Cognitive Radios
• Radio that sense all or part of their surrounding
environment are considered aware systems
• A radio must additionally autonomously modify its
operating parameters to be considered adaptive
If a radio is reconfigurable, aware, adaptive and learns
COGNITIVE RADIO
Aware Radio
• It is equipped with sensors able to gather
environmental information
• In general many kind of sensors can be considered in
Aware Systems, e.g. antenna, microphone, camera,
probes, etc.
• The key characteristic that raises a radio to the level
of aware is the consolidation of environmental
information not required to perform simple comms
Adaptive Radio
• Frequency, istantaneous bandwidth, modulation
scheme, error correction coding, channel mitigation
strategies, data rate, transmit power, etc, are
operating parameters that may be adapted
• Example:
 A FHSS radio is not considered adaptive because once
programmed for a hop sequence it is not changed.
 A FHSS radio that changes hop pattern to avoid/reduce
collisions may be considered adaptive.
Cognitive Radio
• A CR has the following characteristics:




Sensors creating awarness of the environment
Actuators enabling interaction with the environment
Memory and model of the environment
Learning capability that helps to select a specific action or
adaption to reach a specific goal
 Autonomy in action (unsupervised system)
 An engine able to take constrained decisions
Comparison: from SDR to CR
Cognitive Radio
Cognitive Radio
•
Cognitive radio designed upon SDR, has been
proposed as the means to promote the efficient use
of the spectrum by exploiting the existence of
opportunities
Cognitive Radio
• According to the Encyclopedia of Computer Science
cognition encompasses the following steps:
 Mental states and processes intervene between input stimuli
and output responses
 The mental states and processes are described by
algorithms
 The mental states and processes lend themselves to
scientific investigations.
Cognitive Radio
• Moreover, the interdisciplinary study of cognition is
concerned with exploring general principles of
intelligence through a synthetic methodology termed
learning by understanding.
• Putting these ideas together and bearing in mind that
cognitive radio is aimed at improved utilization of the
radio spectrum, Haykin offers the following definition
for cognitive radio
Definition
• CR is an intelligent wireless communication system
that is aware of its surrounding environment, and
uses the methodology of understanding-by-building
to learn from the environment and adapt its internal
states to variations in the perceived RF stimuli by
making corresponding changes in operating
parameters in real-time, with two primary objectives
in mind:
 highly reliable communications whenever and
wherever needed;
 efficient utilization of the radio spectrum.
Definition
Common keyword in this context are:
• Spectrum awareness
 the understanding of what is happening in the
electromagnetic spectrum
• Self Adaptation
 the capability to adapt the system parameter according to an
evolving external scenario
• Intelligence or cognition
 the capability to learn from the interaction with the
environment
• Efficiency
 The capability to efficiently exploit the available radio
spectrum
Cognitive Radio - models
• In general the Cognitive Systems can be
characterized by different cooperative phases, which,
together with continous learning, are really powerful
tools for all kind of applications.
• CR behavior can be modeled as a cognitive cycle
• In the literature different cognitive cycles have been
provided in order to describe CR behavior
Mitola’s Cognitive Cycle
According to Mitola it is possible to model the CR
behavior as follow:
• Stimuli enter the CR as sensory interrupts,
dispatched to the cognitive cycle for a response
• CR sequentially observes (senses and perceives) the
environment, orients itself, creates plans, decides,
and then acts.
Mitola’s Cognitive Cycle
• Observe (Sense and Perceive)
 The CR observes its environment by parsing incoming RF
stimuli.
• Orient
 determines the significance of an observation by binding the
observation to a previously known set of stimuli
• Plan
 reasoning over time
Mitola’s Cognitive Cycle
• Decide
 selects among the candidate plans
• Act
 initiates the selected plans using actuators which access the
external world or the CR’s internal states.
• Learning
 Learning information and experiences, together with
decision, is the most important capability for a cognitive
system
Mitola’s Cognitive Cycle
Haykin’s Cognitive Cycle
Haykin focuses on three on-line cognitive tasks:
1. Radio-scene analysis:
 estimation of interference temperature of the
radio environment;
 detection of opportunities
2. Channel identification:
 estimation of channel-state information (CSI);
 prediction of channel capacity
3. Transmit-power control and dynamic spectrum
management.
Haykin’s Cognitive Cycle
• The cognitive process starts with the passive sensing
of RF stimuli and culminates with action.
• Tasks 1) and 2) are carried out in the receiver, and
task 3) is carried out in the transmitter.
• the cognitive module in the transmitter must work in a
harmonious manner with the cognitive modules in the
receiver
Haykin’s Cognitive Cycle
1
3
2
Cognitive Cycle: a comparison
• Haykin defines the behavior of CR system through a
Cognitive Cycle, similar to Mitola's one, but much
more clustered in macro-phases
• Mitola is much more interested on the impact of the
cognitive capabilities onto the communications
market, Haykin faces the problem from a more
general point of view.
• Conversely, both researches agree on the fact that
the SDR systems are the natural platform for the
implementation of CR devices.
Cognitive Cycle: a comparison
• While in the Mitola’s vision the CR is suited to realize
the user’s preferences, in the Haykin’s one it is well
explained a cognitive communication between a
transmitter and a receiver.
• In both of the previous visions it is clear the effort to
model the CR s an entity able to


reason about and analyze the external world
modify its internal configuration to reach the best
solution
30
Cognitive Cycle for CR apps: A
simplified vision
In general, the behavior of a Cognitive Systems can be
characterized by four sequential cooperative phases, which,
together with continuous learning, are really powerful tools for
all kind of applications.
These phases constitute the four main capabilities of the
Cognitive Cycle:
DECISION
Physical
World
ACTION
Learning
ANALYSIS
SENSING
31
Cognitive Cycle: A simplified vision
Sensing
is a passive interaction component: the system has to
continuously acquire knowledge about the interacting
objects and its own internal status
E.g.
Sensing process can be view
as the scan of the
electromagnetic environment
by an antenna or the
acquisition of an image
sequence by a camera
32
Cognitive Cycle: A simplified vision
Analysis
perceived raw data need an analysis phase to represent them
and extract interesting filtered information
E.g. Analysis process extracts information of
interest like users’ positions in an angle-frequency
map or features used for classification or tracking
33
Cognitive Cycle: A simplified vision
Decision
the intelligence of a CS is expressed by the ability to
decide for the proper action, given a basic knowledge,
experience and sensed data
It is one of the most critical and complex phase of
the cycle
34
Decision phase
There are many different approaches to decision phase:
♦ Rule Based algorithm  They are based on the paradigm IF ….
THEN …
♦ Semantic Networks  It’s a graph designed by an ensemble of
nodes linked each other by arches: nodes represent objects,
situation or events, while arches mean their relations
♦ Decision Trees  It’s a framework designed on a tree where every
internal node represents an adjective and every leave represents a
label of class
♦ Memory Based Reasoning  With this technique it is possible to
classify basing on previous experiences
We will focus on bio-inspired algorithms for
knowledge representation and decision phase
EMBODIED COGNITION
APPROACH
35
Cognitive Cycle: A simplified vision
Action
expresses the active interaction the CS can take in
relation to its decision. The system tries to influence
its interacting entities to maximize the functional of its
objective
E.g.
This phase represent how the CS
interacts with the environment with
actuators and communication
systems likes antennas
36
Learning
Learning information and experiences, together with
decision phase, is the most important capacity for a
cognitive system.
There are many different approaches to this task that
can be generally divided in:
♦ Supervised Algorithms (Artificial Neural Network, Support Vector
Machine, Bayesian Learning)
♦ Unsupervised Algorithms (Self Organizing Maps, Radial Basis
Function Network, Reinforcement Learning)
We will focus on a bio-inspired learning algorithm
AUTOBIOGRAPHICAL
MEMORY
37
Knowledge representation and
organization
The cognitive cycle represents a general framework  It
is necessary to specify how the knowledge is managed
and processed within each stage of the cycle
A knowledge representation and organization is
necessary
38
Knowledge representation and
organization
In general the knowledge managed by the cognitive cycle
can be:
♦ an a-priori identification, at a symbolic level, of all the
knowledge necessary to perform the different phases of the
cycle;
♦ acquired through experiences.
It can be organized according to two principal models:
• The former model tries to describe the knowledge in
a symbolic and semantic way  i.e. the classical rulebased approach for AI (Expert Systems);
• Embodied cognition.
39
Rule-based approaches
A Rule-based expert system is a representation of the
human beings natural reasoning and problem-solving
paradigm. It models the human’s production system
using the following modules:
40
Rule-based approaches
Knowledge base - models a human’s long term memory
as a set of rules.
Working memory - models a human’s short term memory
and contains problem facts both entered and inferred by
the firing of the rules.
Inference engine - models human reasoning by
combining problem facts contained in the working
memory with rules contains in the knowledge base to
infer new information.
41
Embodied Cognition
• Embodied Cognition approach takes inspiration
from Robotics works of Rodney Brooks and looks at
intelligence as to an emergent behavior of a set of
agents.
• This approach is based on a model of representation
of the knowledge which describe in a priority manner
the physical capabilities of action of the entity where
happen decision and action.
42
Embodied Cognition
 Knowledge can be viewed as organized following spatial maps
centered on the entity, where information are represented as
created by processes of analysis and decision at different semantic
levels.
 To mantain this separation between decision and action it is
necessary a mechanism related to perception of events derived by
actions (endo-sensors)
Endo-sensors
Action
Decision
Entity
Spatial
internal
map
43
Evolution following Embodied
Cognition approach
Stage1: Decision and Action use only internal information
(endo-sensors)
Decision
Action
Evolution
Command
Command
Decision
Map of
commands
Internal
sensor
Feedback
Action
Feedback
Map of
feedbacks
44
Evolution following Embodied
Cognition approach
Stage2: Evolution leads to distinguish between internal
and external state  Decision and Action use internal
information (endo-sensors) and external (eso-sensors)
Commands
Internal
Analysis
Decision
Virtual
internal
sensor
Int.
analysis
Ext.
analysis
Map of
commands
Int. sensor
Action
Ext.
sensor
External
Analysis
Analysis
map
Map
45
Evolution following Embodied
Cognition approach
Stage3: Distinction between internal and external state
leads to generation of a consciousness (distinction
between itself and other from itself)
External spatial map
Command
Feedback
Interacting Entity
46
Embodied cognition:
a possible definition
Following Anderson definition:
“it focuses the attention on the fact that most real-world thinking
occurs in very particular (and often very complex) environments, is
employed for very practical ends, and exploits the possibility of
interaction with and manipulation of external props. It thereby
foregrounds the fact that cognition is a highly embodied or situated
activity and suggests that thinking beings ought therefore be
considered first and foremost as acting beings ”
47
Evaluation of Embodied Cognition
Recent studies of neurophisiology have confirmed, ata
biological level, the effectiveness of this approach.
Eg. one of the primary goal of intelligent multicellular organisms
evolving toward higher level organisms is to use contextual
information obtained through sensing to move in the surrounding
environment to reach a safer or a food reacher point.
In the human brain, these kind of motions are generated
by specific groups of neurons called Fixed Action
Patterns (FAPs), whose output is able to modulate
motor muscles actions according to a codified sequence
of effector signals. Sequences are modulated by FAPs,
basing on the contextual information acquired through
the senses.
48
Embodied Systems
The representation of the internal knowledge in
embodied systems, and hence the description of
context, is strictly linked with the motion possibilities of
the entity itself.
In general the body of the system has an important role
in the evolution of the entity  The body can be
considered not only as the instrument to perform the
only action phase but this concept can be extended for
every phase of the cycle.
E.g. Make more fine the sensing phase lead to a different
representation of the knowledge (more information to manage)
respect to a coarse sensing phase
49
A Bio-inspired Model
Up to now some fundamental concepts have been
pointed out:
 How a conscience can be represented  Damasio’s
approach  core self, proto self, autobiographical
memories, autobiographical self
 The relations between actions and consciousness 
Llinas approach  FAP
 How the knowledge representation can influence the
cognition process  embodied cognition
 How can be represented a living entity related to its
environment  cognitive cycle
50
A Bio-inspired Model
All of this concept can be fused together to supply a
coherent model to represent a cognitive system
51
Application examples in
cognitive radio
Application example in
cognitive radio - Motivations
Growing success of wireless communications systems
Fundamental problem: lack of available spectrum
53
Application example in
cognitive radio - Motivations
A lot of studies supported by FCC:
• point out a scarce effective utilization of the wireless
spectrum
• encourage new solutions to exploit underutilized
bands
• demand for the overcoming of the exclusive use of
the allocated frequencies
• COGNITIVE RADIO (CR) paradigm is the proposed
solution
54
Application example in
cognitive radio - Motivations
Which are the possible applications of a CR system?
• Exploitation of unused frequencies (or opportunities,
in general) e. g. spectrum holes
55
Application example in
cognitive radio - Apps
Which are the possible applications of a CR system?
• Self-interconnection and self-management of different
systems with different standards
• Homeland security
• Signal interception and identification
• Military applications
• Emergency situations
56
Application example in
cognitive radio
One of the most important characteristics of a CR is the
LEARNING CAPABILITY
Provide the CR with a sort of intelligence increase its
adaptivity and flexibility
Hence the CR can be used not only in well known
situations but also in unexpected or unforeseen
scenarios
57
Application example in
cognitive radio – Cognitive Cycle
Haykin and Mitola’s cognitive cycles can be simplified in
order to obtain the proposed bio-inspired cognitive
cycle:
ACTION
DECISION
LEARN
ANALYSIS
SENSING
58
Application example in
cognitive radio
It is also important to
underline that the
flexibility guaranteed by
the previously described
bio-inspired approaches
can be extended not
only at the physical layer
of the communication
(as in Haykin’s vision)
but at the entire ISO-OSI
communication model.
59
Application example in
cognitive radio
These kinds of applications are aimed for obtaining a
global optimization of the configuration parameters of
the wireless systems by
 overcoming the traditional limits among different levels
 realizing a cross layer optimization management systems
This is the case of the most important european project
E2R.
60
E2R
61
E2R
End-to-End Reconfigurability (E2R) is the key enabler
for providing a seamless experience to the end-user
and the operators:



Managing and increasing resilience growingly complex
architectures
Reducing costs deployment, evolution and operation of
large communication systems
Providing opportunities develop and experiment rapidly
new services and applications
62
Application example in
cognitive radio
 Most of the developed CR project are characterized by fixed
or slightly flexible models.
 In general, these models has to be accurate and often it is
necessary to include in them some severe constraints.
To overcome these limits it is possible to apply the
theory related to bio-inspired cognitive radios
63
Application example in
cognitive radio: Infomobility
• The chosen cognitive
system is composed by a
Cognitive Base
Transceiver Station
(CBTS) for mobile
applications.
• Task of CBTS is to
manage communication
with a set of mobile
stations in a vehicular
context
• CBTS is equipped by a
“smart antenna” system
64
Application example in
cognitive radio : Infomobility
The cognitive cycle proposed in the general theory is
well suited to chosen application
DECISION
Physical
World
ACTION
Learning
ANALYSIS
SENSING
65
Application example in
cognitive radio : Infomobility
Let us describe the
cognitive cycle mapped for
the chosen application:
• Sensing: the smart
antenna system performs a
scanning of the
environment
66
Application example in
cognitive radio : Infomobility
Let us describe the
cognitive cycle mapped for
the chosen application:
• Analysis: extract from
the context the presence of
the users in the domain of
interest
67
Application example in
cognitive radio : Infomobility
Let us describe the
cognitive cycle mapped for
the chosen application:
• Decision: allocate
available resources
68
Application example in
cognitive radio : Infomobility
Let us describe the
cognitive cycle mapped for
the chosen application:
• Action: reconfigure the
beamformer in order to
provide a reliable
communication link to the a
user
69
Application example in
cognitive radio : Infomobility
• In the next few slides we’ll focus our attention on two
important phase of the cognitive cycle:
• ANALYSIS PHASE
• DECISION PHASE
Analysis Phase
• Most important phase for radio
scene analysis is carried out by
ANALYSIS PHASE
• Input: oversampled filtered received
signal (from Sensing phase)
• Output: transmission mode of each
user enriched with context
information (to Decision phase)
• Objective: identify the transmission
mode of each user
Analysis Phase
• The analysis phase of the
proposed Cognitive Cycle is
composed by two sub-phases:
 FEATURES EXTRACTION
 CLASSIFICATION
Analysis Phase
• Different algorithm can be applied to analysis phase
• Stand-alone techniques




Energy detector
Matched Filter
Feature detection (the one applied in this example)
Transformation
• Cooperative and Distributes
 Distributed Detection with Fusion
 Distributed Detection without Fusion
Analysis Phase – features extraction
• Features extraction: Discrete Fourier Transform
(DFT) of the input signal
• normalized DFT is used as probability density
function (pdf)
• extracted features x are conditional moments of
evaluated pdf
 mean
 standard deviation
 kurtosis
Analysis Phase – map
The analysis map:
75
Analysis Phase – features extraction
The features space:
76
Analysis Phase – features extraction
The extracted features: kurtosis
77
Analysis Phase – classification
• In order to classify signal based
on extracted features it is
possible to apply different
existing tools:




Bayesian/Hidden Markov Models
Neural Networks
Support Vector Machines
K-NN, Parzen Windows
Analysis Phase – classification results
example
Decision Phase – Algorithm
• For the proposed intelligent system a Reinforcement
Learning (RL) approach it is chosen for decision
phase.
• RL is a machine learning technique that unlike other
machine learning approaches is:
 model free
 unsupervised
 able to learn on line
• RL approach allows the system to learn the correct
strategies for interactions with the environment
80
Decision Phase – General Algorithm
Decision phase, carried out
using the RL, can be described
by using:
•Input: high level description of
the environment status xc (from
analysis)
•Output: establish new
configuration xp (to action)
•Objective: choose action that
maximizes r (reward)
•Experience = capability of
predicting r given xc and xp
81
Decision Phase – Algorithm in the
proposed apps
• DECISION AGENT provides intelligent control for
the system
• Input: external state




number of detected users
associated direction
SNR of each established link
Transmission modality used
• Output: new internal state
 beamformer configuration
 power for each link
• Goal: choose an action that maximizes the reward
• Experience is the capability to predict the reward
given an internal and an external states
Decision Phase – Algorithm in the
proposed apps
Learning task: estimation of the Q-function
Q(xc, xp) = E{ r | xc, xp} :
 at the beginning Q = 0 for every couple (xc, xp);
 if you encounter (xc, xp) and receive a reward r then
Q(xc, xp) = αr + (1 - α )Q(xc, xp)
83
Decision Phase – Algorithm in the
proposed apps
Decision task: balance exploration and exploitation 
ε-greedy policy is the chosen strategy:
 with probability 1 - ε, exploits: choose
xp = arg[maxxp (Q(xck, xp))]
 with probability ε, explores: pick xp randomly
It is a simple algorithms but theoretical results
guarantee convergence
84
Decision Phase – results
Reward: maximize the SNR (equally
minimize the steering error)
85
Decision Phase – results
Reward: maximize the SNR (equally
minimize the steering error)
86
Decision Phase – results
Reward: maximize the SNR (equally
minimize the steering error)
87
Decision Phase – results
Reward: maximize the SNR (equally
minimize the steering error)
88
Decision Phase – remarks
• Since the reward does not penalize the used power
the base station maximize the SNR using almost all
available power
• It is possible to change the reward thanks to
adaptivity provided by reinforcement learning
approach
Decision Phase – results
Reward: maximize the SNR (equally
minimize the steering error) keeping
as low as possible tx power
90
Decision Phase – results
Reward: maximize the SNR (equally
minimize the steering error) keeping
as low as possible tx power
91
Decision Phase – conclusive remarks
• In this case results regarding steering error are
omitted because they are similar to the previous one
• Changing the reward the system maximize the
steering error as before keeping as low as possible
the used power