Mathematical Preliminaries and Review
—– Ruye Wang for E59
1
Complex Variables and Sinusoids
A complex number can be represented in either Cartesian or polar coordinate system:
z = x + j y = |z| ! z = r ejθ = r(cos θ + j sin θ)
where
!
!
Re[z] = x = r cos θ
,
Im[z] = y = r sin θ
√
r = |z| = x2 + y 2
θ = ! z = tan−1 y/x
The complex conjugate of z = x + jy is defined as:
z ∗ = (x + jy)∗ = x − jy = (r ejθ )∗ = r e−jθ
Arithmetic operations of complex variables (vectors in complex space):
•
z + z ∗ = (x + jy) + (x − jy) = 2x = 2 Re [z]
z − z ∗ = (x + jy) − (x − jy) = 2jy = 2j Im [z]
zz ∗ = (x + jy)(x − jy) = x2 + y 2 = |z|2
•
z1 ± z2 = (x1 + jy1 ) ± (x2 + jy2 ) = (x1 ± x2 ) + j(y1 ± y2 )
•
z1 z2 = r1 ejθ1 r2 ejθ2 = r1 r2 ej(θ1 +θ2 )
•
r1 ejθ1
r1 j(θ1 −θ2 )
z1
=
=
e
jθ
z2
r2 e 2
r2
•
z 1/n = (rej(θ+2kπ) )1/n = r 1/n ej(θ/n+2kπ/n) ,
(k = 0, 1, · · · , n − 1)
Some important values on the complex plane:
ej2kπ = 1, ej(2kπ+π/2) = j,
Euler’s Formula:
!
ej(2kπ±π) = −1,
ej(2kπ−π/2) = −j,
!
ejθ = cos θ + j sin θ
,
e−jθ = cos θ − j sin θ
1
(k = 0, ±1, ±2, · · ·)
cos θ = (ejθ + e−jθ )/2
sin θ = (ejθ − e−jθ )/2j
A sinusoidal time function can be represented in either of the two forms:
• The real and imaginary parts (horizontal and vertical projections) of a complex exponential (a vector
rotating in CCW direction) represent a cosine and sine function, respectively:
Re[Aej(ωt+ϕ) ] = Re[A cos(ωt + ϕ) + jA sin(ωt + ϕ)] = A cos(ωt + ϕ)
Im[Aej(ωt+ϕ) ] = Im[A cos(ωt + ϕ) + jA sin(ωt + ϕ)] = A sin(ωt + ϕ)
The rate of the rotation is determined by the angular frequency ω.
• The vector sum of two complex exponentials for two vectors rotating in CW and CCW directions,
corresponding to a positive frequency ω and a negative one −ω, respectively:
A cos(ωt + ϕ) =
A j(ωt+ϕ)
[e
+ e−j(ωt+ϕ) ]
2
Similarly, a sine function can also be obtained as a vector sum:
jA cos(ωt + ϕ) =
2
A j(ωt+ϕ)
[e
− e−j(ωt+ϕ) ],
2
i.e.
A cos(ωt + ϕ) =
A j(ωt+ϕ)
[e
− e−j(ωt+ϕ) ]
2j
Linear Algebra
• Vectors and matrices:
An n-D vector is a column vector and its transpose is a row vector:


x1
 . 

x =  .. 
,
xn
xT = [x1 , · · · , xn ]
An m by n matrix is an array of m rows and n columns, which can also be represented in terms of
its column (or row) vectors:
Am×n


a11 · · · a1n
 .
.. 
..
.
=
.
. 
 .
 = [a1 , · · · , an ]
am1 · · · amn
2
where aj is the jth column (j = 1, · · · , n):


a1j
 . 

aj =  .. 

amj
– The transpose of A is:

T




a11 · · · a1n
a11 · · · am1
aT1






.
.
.
.
T
..
..

 . 
.. 
.. 
AT = 
.
.
 ..
 =  ..
 = [a1 , · · · , an ] =  .. 
am1 · · · amn
a1n · · · anm
aTn
– The inverse A−1 of A satisfies:
A−1 A = AA−1 = I
where I is the identity or unit matrix.
• Inner (dot) product:
The inner (dot) product of two vectors x and y is a scaler defined as:


y1∗
n
(

.. 
=
xj yj∗
< x, y >= xT y∗ = [x1 , · · · , xn ] 
 . 
j=1
yn∗
The following concepts are defined based on inner product:
– The norm (or length) of a vector is defined as
||x|| =< x, x >1/2 =
)
*(
* n
+
xj x∗
j
j=1
=
)
*(
* n
+
|xj |2
j=1
– A vector x is normalized if ||x|| = 1.
– Two vectors x and y are orthogonal if < x, y >= 0.
– Two orthogonal vectors are orthonormal if they are both normalized.
– The angle between two vectors is defined as:
θ = cos−1 (
< x, y >
),
||x|| ||y||
i.e.,
< x, y >= ||x|| ||y|| cos θ
– The scaler projection of x on y is defined as:
Py (x) =
< x, y >
= ||x|| cos θ,
||y||
if ||y|| = 1, then Py (x) =< x, y >
(The projection of x on y could also be defined as a vector in the direction of y, which is not
considered here.)
3
• Vector and matrix multiplication:
An n-D column vector xn can be pre-multiplied by an matrix Am×n to result an m-D column vector
ym :
ym = Am×n xn
which can be represented in element form as:





y1
a11 · · · a1n
x1
 . 

 . 
.. 
..
 .  =  ..


.
.   .. 
 . 
 .

am1 · · · amn
ym
xn
and the ith element is (i = 1, · · · , m):
yi =
n
(
aij xj
j=1
Given y and A, this linear system can be solved for x (by pre-multiplying both sides of the equation
by A−1 ):
A−1 y = A−1 Ax = x,
i.e. x = A−1 y
The product of two matrices Am×k and Bk×n is also a matrix:
Cm×n = [c1 , · · · , cn ] = Am×k Bk×n = A[b1 , · · · , bn ]
where the jth column is:
cj = Abj ,
i.e.,
and its ith element is:
cij =





c1j
a11 · · · a1k
b1j
 . 

 . 
.. 
..
 .  =  ..


.
.   .. 
 . 
 .

cmj
am1 · · · amk
bkj
k
(
ail blj ,
l=1
4
(i = 1, · · · , m)
• Orthogonal and unitary matrices
Some special matrices of interest are defined below:
– A is symmetric if AT = A.
– A is Hermitian if A∗T = A, where A∗T is the conjugate transpose of A. If A∗ = A is real, it is
symmetric.
– A is orthogonal if AT = A−1 , i.e., AT A = AAT = I.
– A is unitary if A∗T = A−1 . If A∗ = A is real, it is orthogonal.
All column (or row) vectors of a unitary matrix A = [a1 , · · · , an ] are orthonormal:
< ai , aj >= aTi a∗j = δ[i − j] =
!
1 i=j
0 i=
# j
where δ[n] is the delta function defined as:
δ[n] =
3
!
1 n=0
0 n=
# 0
Inner Product (Hilbert) Space
• Euclidean space:
An n-D inner product vector space, called a Euclidean space, which is a set of all n-D vectors with
inner product defined. This space can be spanned by a set of n linearly independent basis vectors
{b1 , · · · , bn } (none of them can be represented as a linear combination of the rest), so that any vector
x in the space can be expressed as a linear combination of these basis vectors:



c1
n
(
 . 



ck bk = c1 b1 + · · · + cn bn = [b1 , · · · , bn ]  .. 
x=
=
k=1
cn


b11 · · · b1n
c1
 . 
.. . .
.. 


. .   .. 
.
 = Bc
bn1 · · · bnn
cn
where B = [b1 , · · · , bn ] is an n by n matrix with the n basis vectors as its columns, and c =
[c1 , · · · , cn ]T is a column vector composed of n coefficients or weights for the basis vector. These
coefficients can be obtained by solving this linear system (by pre-multiplying B−1 on both sides):
c = B−1 x
In particular, if the basis vectors are orthonormal:
< bi , bj >= bTi b∗j = δ[i − j]
then B∗T = B−1 is a unitary matrix (or orthogonal if B∗ = B
becomes:



c1
b∗T
1
 . 

−1
∗T



.
c =  .  = B x = B x =  ...
cn
b∗T
n
and the kth element
ck =< x, bk >= xT b∗k = Pbk (x)
is the projection of vector x onto the kth basis vector bk .
5
is real), and the equation above


x

– Any set of n linearly independent vectors can be used as a basis of an n-D vector space.
– Any given basis of n vectors can be converted in to a set of orthogonal basis vectors by GramSchmidt process.
– Any rotation of an orthogonal basis results another orthogonal basis, i.e., any two different sets
of orthogonal bases are related by a rotation corresponding to an orthogonal matrix.
– All discussions above for n-D Euclidean space can be generalized to an infinite dimensional
vector space by letting n −→ ∞.
• Function space:
The concept of n-D vector space can be generalized to n-D function space, which a set of all functions
x(t) defined over a particular range 0 < t < T with the inner product defined as:
< x(t), y(t) >=
,
T
x(t)y ∗ (t)dt,
(compared to < x, y >=
The norm of a function is defined as:
||x(t)|| =< x(t), x(t) >
1/2
.,
=
T
x(t)x∗ (t)dt =
.,
T
-
i
xi yi∗ )
|x(t)|2 dt
A function x(t) is normalized if it has unity norm ||x(t)|| = 1.
Two functions x(t) and y(t) are orthogonal if the inner product is zero:
< x(t), y(t) >=
,
T
x(t)y ∗ (t)dt = 0
This function space can also be spanned by a set of basis functions bk (t) so that any given function
x(t) in the space can be expressed as a linear combination of these basis functions:
x(t) =
(
ck bk (t)
k
In particular, if the basis functions are orthogonal:
< bk (t), bl (t) >=
,
T
bk (t)b∗l (t)dt = 0
(k #= l)
then the coefficient ck can be obtained by taking an inner product with bl (t) on both sides of the
equation above:
< x(t), bl (t) >=<
(
ck bk (t), bl (t) >=
k
(
k
ck < bk (t), bl (t) >= cl < bl (t), bl (t) >= cl ||bl (t)||2
The kth coefficient becomes:
< x(t), bk (t) >
=
ck =
< bk (t), bk (t) >
/
T
x(t)b∗k (t)dt
||bk (t)||2
Moreover, if the basis functions are orthonormal (orthogonal and normalized) ||bk (t)|| = 1, then we
get
,
ck =< x(t), bk (t) >= x(t)b∗k (t)dt
T
i.e., ck is the projection of x(t) onto the kth basis function bk (t).
6
4
Miscellaneous
• Integration of these expressions:
,
sin2 (at)dt,
,
t e−t dt,
,
t sin(at)dt
• Solution of 1st and 2nd-order ordinary differential equations:
1
ẏ(t) + y(t) = af (t),
τ
ÿ(t) + 2ζωn ẏ(t) + ωn2 y(t) = af (t)
where τ , ζ, ωn and a are real constants, and the input f (t) can be:
– f (t) = 0 [free response];
– f (t) = u(t) [unit step response];
– f (t) = sin ωt [sinusoidal response];
– f (t) = ejωt [harmonic (complex exponential) response];
• Geometric series:
– Finite:
n−1
(
k=0
– Infinite (|x| < 1):
xk = 1 + x + x2 + · · · + xn−1 =
∞
(
k=0
xk = 1 + x + x2 + · · · =
1 − xn
1−x
1
1−x
• Trigonometry identities:
–
sin(α ± β) = sin α cos β ± cos α sin β
cos(α ± β) = cos α cos β ∓ sin α sin β
–
sin2 α = (1 − cos 2α)/2
cos2 α = (1 + cos 2α)/2
–
sin 2α = 2 sin α cos α
cos 2α = cos2 α − sin2 α
–
0
sin(α/2) = ± (1 − cos α)/2
0
cos(α/2) = ± (1 + cos α)/2
7