Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Framing the Image
S. Allen Broughton - Rose-Hulman Institute of Technology
Rose Math Seminar, February 4, 2015
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Overview
Signals as Vectors - define signals and describe the vector
spaces they live in.
Analysis/Synthesis - Determination of “Fourier coefficients”
and reconstruction of signals from their Fourier coefficients.
Orthogonal reconstruction - Orthogonal bases of basic
waveforms
Frame reconstruction - General frames of basic waveforms
Wrap up
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
What is a signal?
Physical examples of signals
Sound or audio signal - a 1D signal
Images e.g., photographs X-rays - a 2D signal
Movies a 2D image changing over time - a 3D signal
A 3D MRI image of a body part - a 3D signal
A time varying 3D medical image of a beating heart - a 4D
signal
A data sample in a multivariate statistical study - multi
dimensional signal
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
What is a signal?
Definition of a signal
We will keep our definition of signals simple.
Definition
A (scalar valued) signal is a function f : Ω → R, where Ω is an
object of interest. The dimensionality of the signal is the
dimension of Ω as an object. Typically Ω is a bounded subset of
the space-time continuum.
A vector valued signal is a sequence of functions
f1 , . . . , fs : Ω → R. For a point ω ∈ Ω the vector (f1 (ω), . . . , fs (ω))
is the vector value of the signal at the point ω.
We are only going to look at scalar signals.
All our signals will be real-valued though great advantage
can be achieved by looking at complex-valued signals.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
What is a signal?
Examples of the definition -1
Example
An audio clip is a function f (t) where t varies in an interval
a ≤ t ≤ b. The magnitude |f (t)| presents the amplitude of the
sound and the frequency of oscillation of f represents the
frequency of the sound.
Example
An monochrome image is function f (x, y ) defined for
a ≤ x ≤ b, c ≤ y ≤ d. The value f (x, y ) represents the intensity
level of light. For example if we impose 0 ≤ f (x, y ) ≤ 1, then
our 0 might represent black, and 1 represent white and the
intermediate values represent shades of grey.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
What is a signal?
Examples of the definition - 2
Example
A color image is given by vector-valued signal
(R(x, y ), G(x, y ), B(x, y )) defined for a ≤ x ≤ b, c ≤ y ≤ d.
The R, G, B give the intensity level of the red, green and blue
components of the colour at (x, y ). The functions R, G, B are
called the red, blue and green channels, respectively.
Example
A monochrome movie is function f (x, y , t) defined for
a ≤ x ≤ b, c ≤ y ≤ d, t0 ≤ t ≤ t1 . For a given fixed t 0 the
function f (x, y , t 0 ) defines a monochomatic image or frame.
This is not the frame of the title!
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
What is a signal?
Communication/presentation of signals
1D signals - sound or a graph
2D signals - image
3D signal - a movie, series of images
3D signal - a series of 2D slices of a 3D object, dependent
on perspective
3D signal - a varying projected 2D image of a 3D object,
dependent on perspective
4D signal - 2D projection movie of a beating heart
Here are some Wikipedia MRI images
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Signal spaces as vector spaces
Continuous vs discrete signals - 1
Continuous signals: We have assumed that f signal is
defined at all ω ∈ Ω and so there are infinitely many values.
Discrete signals: So that signals be handled by computers,
digital cameras etc., we discretize the signal. Pick a
appropriate, finite set {ω1 , . . . , ωd } ⊂ Ω and form the vector
Xf = (f (ω1 ), . . . , f (ωd )) ∈ Rd
Uniform 1D linear, 2D rectangular, and 3D "box" grids are
obvious choices for {ω1 , . . . , ωd } ⊂ Ω.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Signal spaces as vector spaces
Continuous vs discrete signals - 2
One can easily imagine uses for polar and spherical
sampling grids.
The number and the selection of the points is important in
the fidelity of the discretized signal and the implementation
of sampling.
E.g., aliasing of frequencies signal can be an issue.
There are other methods of discretization, such as local
averaging for cameras.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Signal spaces as vector spaces
Energy of a signal
The energy of a signal is defined as follows.
Continuous case:
kf k2 = hf , f i =
Z
f 2 (ω)dω
Ω
where dω is a suitable measure on Ω, usually the ordinary
volume measure.
Discrete case:
kX k2 = hX , X i =
d
X
i=1
xi2
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Signal spaces as vector spaces
Superposition and scaling of signals
Given two signals f and g defined on Ω they can be added
(superimposed) and scaled
f,g → f + g
a, f → af , a ∈ R
The set of all signals forms a vector space.
In the discrete case these operations are just the ordinary
vector operations in Rd .
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Signal spaces as vector spaces
Inner products - 1
Given two signals f and g defined on Ω we can compute
their inner product.
Z
hf , gi =
f (ω)g(ω)dω
Ω
In the discrete case for vectors X and Y the inner product
is:
d
X
hX , Y i =
xi yi
i=1
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Signal spaces as vector spaces
Inner products - 2
If X Y are represented as column vectors then the inner
product is a matrix product.
hX , Y i = Y t X
If the second vector is a “vector of interest”, or basic
waveform, then the inner products vector hf , gi or hX , Y i
measures the strength of characteristics of g or Y present
in the vector f or X respectively.
More later.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Signal spaces as vector spaces
Finite energy signals
We define the two main types of vectors paces of finite energy
signals.
Continuous case: The space of finite energy signals on Ω
is the space
L2 (Ω) = {f : kf k < ∞}
Discrete Case: Rd is the totality of finite energy signals of
length d.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Example: Fourier Analysis
Fourier Coefficients - 1
We move on to analysis and operation on signals, starting with
a familiar example.
The Fourier coefficient analyse the various frequencies in a
signal.
1
an =
π
Z
1
bn =
π
Z
2π
f (x) cos(nx)dx = hf , cos(nx)i
0
2π
f (x) sin(nx)dx = hf , sin(nx)i
0
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Example: Fourier Analysis
Fourier Coefficients - 2
Conversely given the sequence {a0 , a1 , b1 , a2 , b2 , . . .} we
may construct the synthesized signal
∞
∞
n=1
n=1
X
a0 X
+
an cos(nx) +
bn sin(nx)
2
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Example: Fourier Analysis
Fourier Coefficients - 3
The map
f → {a0 , a1 , b1 , a2 , b2 , . . .}
is called the analysis operator with respect to the
sequence {1, cos(x), sin(x), cos(2x), sin(2x), . . .}
The map
{a0 , a1 , b1 , a2 , b2 . . .} →
∞
∞
n=1
n=1
X
a0 X
+
an cos(nx)+
bn sin(nx)
2
is called the synthesis operator.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Example: Fourier Analysis
Fourier Coefficients - 4
The composition of the two maps is given by
f →
∞
∞
X
hf , 1i X
hf , cos(nx)i cos(nx)+
hf , sin(nx)i sin(nx).
+
2
n=1
n=1
The fact that
f =
∞
∞
X
hf , 1i X
+
hf , cos(nx)i cos(nx)+
hf , sin(nx)i sin(nx),
2
n=0
n=1
for f ∈ L2 [0, 2π], is called perfect reconstruction for the
sequence {1, cos(x), sin(x), cos(2x), sin(2x), . . .}.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Finite dimensional analysis/synthesis
Finite analysis/synthesis operators - 1
Let X ∈ Rd be a signal and let Φ = {φ1 , . . . , φN ∈ Rd } be
any sequence of vectors.
we think of the {φ1 , . . . , φN ∈ Rd } as basic waveforms.
We will call Φ a frame if it satisfies additional conditions,
specified later.
We will do analysis and synthesis operations using the
vectors in Φ.
To this end, let F be the matrix
F = φ1 φ2 · · ·
φN
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Finite dimensional analysis/synthesis
Finite analysis/synthesis operators - 2
The “Fourier coefficients” of X with respect to F are given
by
hX , φn i = φtn X .
b ∈ RN ,
The vector consisting of the hX , φn i is denoted by X
and in matrix form is given by

  t

hX , φ1 i
φ1 X
 hX , φ2 i   φt X 

  2 
b
X =
 =  ..  = F t X .
..

  . 
.
hX , φN i
φtN X
The operator X → F t X is called the analysis operator.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Finite dimensional analysis/synthesis
Finite analysis/synthesis operators - 3
Now pick
Y =
a1 a2 · · ·
aN
t
as a proposed sequence of Fourier coefficients.
The synthesized signal is
a1 φ1 + a2 φ2 + · · · + aN φN = FY .
The operator Y → FY is called the synthesis operator.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Reconstruction and dual frame
Frame operator, dual frame and reconstruction - 1
The frame operator S = FF t is the combined operator
b = F t X → FF t X = SX .
X →X
S is a d × d matrix, operating on signals X ∈ Rd .
Suppose S is invertible, then define
Ψ = {ψ1 = S −1 φ1 , . . . , ψN = S −1 φN }.
The frame Ψ, if it exists, is called the dual frame to Φ.
In matrix form
S −1 F =
ψ1 ψ2 · · ·
ψN
.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Reconstruction and dual frame
Frame operator, dual frame and reconstruction - 2
Rewriting X = S −1 FF t X we get

b = ψ1 ψ2 · · ·
X = S −1 F X
ψN




hX , φ1 i
hX , φ2 i
..
.



,

hX , φN i
yielding the following reconstruction formula.
Proposition
Let notation be as above and assume that the frame operator is
invertible. Then
N
X
X =
hX , φi i ψi
i=1
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Applications and Examples
Why synthesis and analysis?
Feature extraction - Fourier analysis can tell us frequency
distribution.
Reconstruction
b → S −1
X →X
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
The Frame Operator and Grammian Matrix
More on the frame operator S - 1
The frame operator S has the following interpretation as a sum
of outer products.
S = FF t

=
φ1 φ2 · · ·
φN




φt1
φt2
..
.
φtN
= φ1 φt1 + φ2 φt2 + · · · + φN φtN





Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
The Frame Operator and Grammian Matrix
More on the frame operator S - 2
The outer product φn φtn is a d × d rank one matrix in
contrast to the scalar inner product φtn φn .
Each outer product satisfies
φn φtn X = hX , φ1 i φn
The partial sum
φ1 φt1
+
φ2 φt2
+ ··· +
φn φtn
X =
n
X
hX , φi i φi
i=1
is an approximation to SX , an analogue of the partial
Fourier sum
n
X
m=0
hf , cos(mx)i cos(mx) +
n
X
m=1
hf , sin(mx)i sin(mx)
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
The Frame Operator and Grammian Matrix
The Grammian matrix G - 1
The Grammian matrix records inner products
G = FtF
 t
φ1
 φt
 2
= .
 ..
φtN



=




 φ1 φ2 · · ·

φt1 φ1
φt2 φ1
..
.
φt1 φ2
φt2 φ2
..
.
φN
···
···
..
.
φt1 φN
φt2 φN
..
.
φtN φ1 φtN φ2 · · ·
φtN φN





Frame expansion
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
The Frame Operator and Grammian Matrix
The Grammian G matrix - 2
The columns of F form an orthonormal set if and only if
G = IN the N × N identity matrix, since G tells us that the
column dot products are what they are supposed to be.
The set Φ is an orthonormal basis of Rd if and only if
N = d and G = IN = Id
If Φ is an orthonormal basis of Rd then
S = Id
For
G = F t F = Id ⇒ F t = F −1
S = FF t = FF −1 = I
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Perfect reconstruction
Reconstruction formula
Proposition
If Φ is an orthonormal basis of Rd then we have perfect
reconstruction
N
X
X =
hX , φn i φn
n=1
Proof.
The righthand sum is
FF t = SX = Id X = X .
Frame expansion
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Perfect reconstruction
Parseval’s equality
Proposition
If Φ is an orthonormal basis of Rd then Parseval’s equality holds
kX k2 =
N
X
hX , φn i2
n=1
Proof.
N
X
n=1
hX , φn i2 = F t X , F t X = FF t X , X = hIN X , X i = kX k2
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
What is a frame?
Relaxing conditions on frames and bases
Now suppose that we relax some conditions on the set Φ
Φ is a spanning set but need not be orthogonal or even a
basis.
We do lose unique expansion, i.e., Y in the expansion
below is no longer unique.
a1 φ1 + a2 φ2 + · · · + aN φN = FY
However we still retain perfect reconstruction
X =
N
X
hX , φn i ψn
n=1
at the expense of additional terms and computing the dual
frame Ψ.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
What is a frame?
Frame definition
We finally define a frame.
Definition
Let Φ be a finite set in Rd as previously described and suppose
that there are constants 0 < A ≤ B such that for all X ∈ Rd
2
A kX k ≤
N
X
hX , φn i2 ≤ B kX k2 .
n=1
Then Φ is called a frame with frame bounds A and B.
If A = B then Φ is called a tight frame.
If kφn k = 1 for all n then Φ is called a unit norm frame.
If hφm , φn i = c for all n 6= m then Φ is called an equiangular
frame.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Frame operator again
Conditions on the frame operator - 1
Suppose Φ is a frame with frame bound A and B.
Then
A kX k2 ≤ hSX , X i ≤ B kX k2
Hence, S is invertible.
The optimum frame bounds are the smallest and largest
eigenvalues of S.
If Φ is a tight frame, then hSX , X i = A hX , X i.
If Φ is a tight frame, then S = AId .
If Φ is a unit norm tight frame, then A =
N
d.
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Frame operator again
Conditions on the frame operator - 2
We just prove some of the bullets on the previous slide.
From the definitions:
N
X
hX , φn i2 = F t X , F t X = FF t X , X = hSX , X i .
n=1
From the definition of tight frame hSX , X i = A hX , X i. From
the above bullet we get:
hS(X + Y ), X + Y i = hSX , X i + 2 hSX , Y i + hSY , Y i
= A hX , X i + 2 hSX , Y i + A hY , Y i
hS(X + Y ), X + Y i = A hX + Y , X + Y i
= A hX , X i + 2A hX , Y i + A hY , Y i
Continued on next slide
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame expansion
Frame operator again
Conditions on the frame operator - 3
From the last slide, for all X and Y
hSX , Y i = A hX , Y i
and
hSX − AX , Y i = hSX , Y i−hAX , Y i = A hX , Y i−A hX , Y i = 0
yielding SX = AX for all X .
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
Frame operator again
Conditions on the frame operator - 4
Proof of constant for a tight unit normal frame.
Proof.
Suppose that Φ is a unit norm frame, tight frame.
Then, the Grammian matrix G = F t F has 1’s on the
diagonal.
So
N = trace(F t F ) = trace(FF t ) = trace(AId ) = dA
N
A=
d
Frame expansion
Overview
Signals as Vectors
Analysis/Synthesis
Orthogonal expansion
done
Any Questions?
Frame expansion