Vector Mathematics - Basic Definitions
Remember about the Cartesian co-ordinate plane with X and Y-axis? Each of the coordinates can be considered as a different VECTOR consists of 2 variables.
A vector is an entity, which may consist of 1 or more variables.
Variable in vector is also referred as dimension. A vector consists of 1 variable is also
called 1 dimensional vector. A vector consists of 2 variables is also called 2 dimensional
vector.
A vector that consists of n variables is also referred as n-dimensional vector.
Example: A 3x3 image consist of 9 pixels can be regarded as 9 dimensional vector
How do we regard a 5x5 image in vector mathematics?
[Ans.: A 5x5 image can be regard as 25 dimensional vector]
On the Cartesian co-ordinate plane, there are many 2-dimensional vectors consist of 2
variables (X-co-ordinate and Y-co-ordinate). All these vectors within the Cartesian coordinate plane form a VECTOR SPACE.
Definitions: A vector space is the set or the collection of all the possible vectors within a
particular system.
Example: Let's have 2x2 binary image. A vector space of such 2x2 binary image consists
of all the possible combinations as below
0
0

1
0

0  0 0  0
0 0 1 1
0 1 0 1
0 0 1 1
0   0 0  0
0 1 1 0
0 1 0 1
0 1 1 0
1 0 1 0
0 0 1 1
1 1 1 1
0 0 1 1
1 0 1
0 1 1
1 1 1
0 1 1
What is the total number of vector that forms the vector space of 2x2 gray-scale image?
[Ans.: 232]
Definitions: Vector dot product or vector inner product is defined as the sum of the
product of individual components of the vectors as illustrated below:
 a1 
 b1 
a 
 
~  2
~ b2 
Let A 
and B 
:
:
 
 
a n 
bn 
~ ~
A  B  a1b1  a2 b2  .....  an bn
~
Definitions: Magnitude or norm of a vector V is defined as the square root of sum of the
square of individual components of the vector as follows:
~
2
2
2
V  v1  v2  ..  vn
Basis Vector
Let's assume that within a vector space, there exists a set of vectors (for simplicity in
future references, we call these vectors collectively as basis vector) such that all the
vectors within that vector space can be represented in term of these basis vectors.
Definition: Basis vector is a member of a set of vectors within a vector space that fulfil
the requirements stated below:
1. They SPAN the vector space; i.e., any vector within the vector space can be
represented as the LINEAR COMBINATION or WEIGHT SUM of these basis vectors
as follows:
~ ~ ~ ~
Let the basis vector set = {B1 , B2 , B3 ....Bn } in a vector space where each vector is
~
denoted as V . Then, there exists a set of numbers = {a1 , a2 ...., an } (we call these
numbers as coefficients future reference) such that
~
~
~
~
V  a1 B1  a2 B2  ....  an Bn
The basis vector does not contain any other basis vector but itself (This is consistent
to the logic that when something is called a basic element, it should not contain other
element but itself); they are independent; they are ORTHOGONAL to each other;
inner product of one vector and another in the basis vector set is always zero.
~ ~
Bi  B j 
{ B~0
i
2
if i  j
if i  j
Example of basis vector in 2-dimensional vector space
y
~
A
~
V
~
B
~
By
x
~
Bx
0 
~ 1
~
Let's examine if Bx    and B y    fulfils the requirement as basis vector
0 
1 
~ x 
Let's V    be any vector in the 2-dimensional vector space.
 y
~
~
According to the definition, in order for B x and B y to be qualified as basis vector, then for
~
each vector V , there must exist the coefficients a x and a y associated with each basis
vector such that
~
~
~
V  a x Bx  a y B y
x 
1 
0 
    ax    a y  
 y
0 
1 
We can easily see from the above equation that the solution for coefficients a x and a y do
exist: a x  x and a y  y
What is the advantage of having the basis vector set
The advantage lies on when one is given a vector and also the a basis vector set, one may
easily find out the "amount" or the coefficients of each basis vector using very simple
operation of vector inner product.
Consider a vector
~
~
~
~
V  a1 B1  a2 B2  ....  an Bn
~
Notice the result from performing vector inner product between the vector V and also
~
one of its basis vectors B1 :
~ ~
B1  V
~
~
~
~
 B1  (a1 B1  a 2 B2  ....  a n Bn )
~ ~
~ ~
~ ~
 a1 B1  B1  a 2 B1  B2  ....  a n B1  Bn
~ 2
 a1 B1  a 2 (0)  ....  a n (0)
Inner product of different
~ 2
basis vector is always zero
 a1 B1
0 if i  j
~ ~
Bi  B j 
~ 2
Bi if i  j
{
~
The result is the coefficients a1 of basis vector B1 multiply with the square norm of the
~ 2
basis vector B1 . Since we can calculate the norm of the basis vector (see definition of
vector norm), we are ready to find the actual value of the coefficient as follows:
~ ~
~
Bi  V  ai Bi
~ ~
Bi  V
 ai 
~ 2
Bi
2
~ ~ ~ ~
~
Conclusion: Given a vector V and also the basis vector set {B1 , B2 , B3 ....Bn } of the vector
~
space where V resides in, one can easily use vector inner product to find out the
coefficients {a1 , a2 ...., an } associated with each basis vector which can later be used to
~
reconstruct V as follows:
~
~
~
~
V  a1 B1  a2 B2  ....  an Bn
Exercises
~
1. Given a vector V  3 4 and the basis vectors of vector space of general 2
~
~
~
~
~
dimensional vectors: B1  0 1 and B2  1 0 . Find the coefficients of B1 and B 2 in V
~
2. Given a vector V  x y  and the basis vectors of vector space of general 2
~
~
~
~
dimensional vectors: B1  1 1 and B2  1  1 . Find the coefficients of B1 and B 2
~
in V
~
~
3. Verify if these two vectors B1   1 1 and B2  1 1 forms a basis vector set in a
~
vector space where each vector resides is a 2 dimensional vector denoted as V  x y  .
~
~
4. Verify if these two vectors B1   2 1 and B2   3  6 forms a basis vector set in
the same vector space mentioned in question 3
5. From the result of questions 3 and 4, what can you say about whether in a particular
vector space, there is only one basis vector set or there can be many set of basis vector?
What is the significance of finding the coefficients and reconstructing a vector?
Let's consider about this situation. You are given a piece of concrete. You find that the
concrete is almost perfect except that it is rough. You know that this is due to too much
sand added into the mixture of the concrete and it is very difficult to perform any
modification (reduce the amount of sand) on the concrete in its existing form.
Now, what you hoping for most likely is to find out what's the amount of the individual
basic elements such as cement, water and sand that make up the concrete. By knowing
the recipe, you can reproduce another concrete similar to the earlier one, except that now,
you add less sand into the mixture in order to get a smoother concrete. In another word,
you are hoping for a way to find out the recipe or the amount of the individual basic
element that forms the concrete.
In this example, the concrete that is a mixture of its basic elements such as cement, sand
~
and water is analogous to the vector V , which is a linear combination of the basis vector
~ ~ ~ ~
set {B1 , B2 , B3 ....Bn } . The amount of the basic elements is equivalent to the coefficient of
the basis vectors {a1 , a2 ...., an } . The only difference here is one have a simple but
powerful method called vector inner product to calculate the coefficient (the recipe).
As discussed in the earlier section (Vector Mathematics - Basic Definitions), one can in
fact, regards an image as a vector. As vector can be represented as a combination of basis
vector, image can also be a combination of a set of basic pattern or better known as basis
image.
There can be many set of basis vector that could possibly form the vectors in a particular
vector space. Therefore, an image can also be composed using many different set of basis
image. Different basis image set allows one to see the image from different perspective
and allow one to abstract different type of info, which would otherwise, difficult to be
obtained from other perspective. One may see image as composition of basis image
associated with line or edge pattern (as in the Frei-Chen masks). This helps one to
determine if there is a line or edge in an image. One may also see image as composed of
basis image with various degree of smoothness or better known as spatial frequencies
(how rapid brightness change across space). Here is an example of a set of basis image in
which the patterns are associated with spatial frequency
Certain aspect of human visual system has been found closely related to spatial
frequency. For example, edge and details of image is often revealed in high frequency
components while contrast of image is often revealed in low frequency components.
Therefore, it is often useful to have image represented in terms of their frequency
components for further analysis and manipulation.
In the original form of image, each pixel’s value represents the brightness in a particular
location; it reveals the intensity of light associated with space; the image is represented in
spatial domain. On the other hand, we can also represent an image with the coefficients
of basis image associated with various spatial frequencies Here, the value of the
coefficient represents the amount of that particular basis image in the image; it reveals the
intensity of pattern associated with spatial frequency; the image is represented in
frequency domain.
Transformation
The process of changing the representation of an image from one domain to another
domain is called TRANSFORMATION. Changing from the original (spatial) domain to
some other domain is called Forward Transform while the reverse is called Inverse
Transform.
In operation wise, forward transformation means to find the coefficient of basis image;
i.e., to perform inner product between basis image and also the image to be transformed.
Example:
Consider transforming a 2 x 2 images is transformed into coefficients of a set of basis
images listed as follows
1 1 1
~
B00  
2 1 1
1
11
~
B10  
2  1  1
1 1  1
~
B01  
2 1  1
1  1  1
~
B11  
2  1 1 
~ 8 2 
Let's the image I  

 4 2
~
~
1. Perform vector inner product between I and B00 . The coefficients is the value of the
component indexed (0,0), in transformed image (top left corner)
~
~
2. Perform vector inner product between I and B01 . The coefficients is the value of the
component indexed (0,1) in transformed image (top right corner)
~
~
3. Perform vector inner product between I and B10 . The coefficients is the value of the
component indexed (1,0) in transformed image (bottom left corner)
~
~
4. Perform vector inner product between I and B11 . The coefficients is the value of the
component indexed (1,1) in transformed image (bottom right corner)
~ 8 4
5. The resulting transformed image which is usually denoted as T  

 2 2
In general the forward transform can be expressed by the following equation
H 1 W 1
t (u , v)   i (r , c)buv (r , c)
r 0 c 0
where
r – row, c- column,
u – vertical frequency index, v – horizontal frequency index
H – image height, W – image width
~
t (u , v ) as the component indexed (u,v) in T ,
~
i(r , c) as the component indexed (r,c) in I ,
~
buv (r , c) as the component indexed (r,c) in Buv
In operation wise, inverse transform is simply calculating the sum of the product of the
coefficient and its corresponding basis image as illustrated below:
~
~
~
~
~
I  t (0,0) B00  t (0,1) B01  t (1,0) B10  t (1,1) B11
1 1 1
1 1  1
11 1 
1  1  1 8 2
 (8) 
 (4) 
 (2) 
 (2) 




2 1 1
2 1  1
2  1  1
2  1 1  4 2
Note that in the process of forward and inverse transform, no information is lost; the data
can be fully recovered
In general the inverse transform can be expressed by the following equation
H 1 W 1
i (r , c)    t (u , v)buv (r , c)
u 0 v 0
where
r – row, c- column,
u – vertical frequency index, v – horizontal frequency index
H – image height, W – image width
~
t (u , v ) as the component indexed (u,v) in T ,
~
i(r , c) as the component indexed (r,c) in I ,
~
buv (r , c) as the component indexed (r,c) in Buv
Interpretation of Transformed Image
~
It is important to note that the values in the transformed image T no longer represent
brightness. Rather, it represents the intensity of various spatial frequencies. Notice that
the coefficients have been arrange accordingly such that coefficient associated with
higher frequency is placed farther away from the origin (top left corner where the
coefficient of the basis image is associated with zero frequency). In most text, zero
frequency is often referred as DC (Direct Current – no oscillations) while non-zero
frequency is called AC (Alternative Current – with oscillations)
If the transformed image is displayed, then the brightness shall reflect the intensity. Since
most image in real word consists of mainly low frequencies, one can expect that the
transformed image would follow a general fashion where the brightness concentrates on
the top left corner (corresponds to low frequency) and become darker as it moves towards
the bottom right corner (corresponds to higher frequency).
By knowing how to interpret a transformed image properly, one is ready to manipulate
the transformed image according to the needs. For example, in a certain application
where one need to automate the process of separating images of marble floor and also
images of cement floor. This could be done easily in frequency representation of image
where the one with higher intensity in high frequency region (right bottom) shall be the
smooth marble image, while the one with higher intensity in low frequency region (top
left) shall be the more coarse cement floor.
In the analogies to concrete, the forward transform is analogous to finding the recipe of
the concrete (image) which is the mixture of various basic elements such as cement,
water and sand (basis images). The coefficient obtained is analogous to the amount of
basic element in the concrete. Inverse transform is analogous to reproducing another
concrete using the recipe. One may change (increase) the smoothness of concrete by
changing the recipe (reducing amount of sand). This is analogous to changing the texture
of the image (smoothen) by modifying the magnitude of the coefficients (reduce the
magnitude of coefficient of basis image associated with high spatial frequency);
Filtering in Frequency Domain
Changing the properties of an image by means of taking out/reducing unwanted
components is called FILTERING. Since the filtering is performed in the frequency
domain, one need to forward transforms the image into its equivalent frequency
representation. Next, multiply a scale factor range from zero to one with the coefficient.
Multiplying with zero will completely wipe out a component. Multiplying some
intermediate value within the range will effectively reduce the amount of the component.
In order to retain a component, simply multiply its coefficient with one. The resulting
modified version of the transformed image can then be inverse transformed to obtain the
texture-modified image.
There are a few type of Filtering.:
Lowpass filtering – wide out the high frequency greater than a specified threshold( or
better known as cut-off frequency) while retaining the low frequency components.
Highpass filtering–taking out/reducing the amount of low frequency components while
retaining the high frequency components.
Lowpass filtering mask
Highpass filtering mask
The frequency associated with a coefficient in the transformed image is usually expressed
by its Euclidean distance u 2  v 2 from the origin. For example, in lowpass filtering
where the cut-off frequency is specified as 9, then all the coefficients index (u ,v)
where u 2  v 2  9 would be made to zero.
In an image, details and edges are composed of high frequency components. Therefore,
lowpass filtering tends to remove/reduce details and edges and hence, causing the image
to become blur.
On the other hand, contrast of an image is composed of low frequency components.
Therefore, highpass filtering tends to reduce contrast of image, leaving only the edges
and details.
What’s next? What are the differences among DFT, WHT, DCT. How to generate
the basis images for various transform?