ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 1
Complex Number and
Complex Variables
Objectives
After completing this chapter, you will be able to:
▪
▪
▪
Differentiate Complex Number with Complex
Number.
Learn the different operations and relations
involving Complex Number.
Learn what Polar Form is.
Introduction
Functions of (x, y) that depend only on the combination (x + iy) are called functions of
a complex variable and functions of this kind that can be expanded in power series in
this variable are of particular interest.
This combination (x + iy) is generally called z, and we can define such functions as z n,
exp(z), sin z, and all the standard functions of z as well as of x.
They are defined in exactly the same way the only difference being that they are
actually complex valued functions, that is, they are vectors in this two dimensional
complex number space, each with a real and an imaginary part (or component).
Most of the standard functions we have previously discussed have the property that
their values are real when their arguments are real. The obvious exception is the square
root function, which becomes imaginary for negative arguments.
Since we can multiply z by itself and by any other complex number, we can form any
polynomial in z and any power series as well. We define the exponential and sine
functions of z by their power series expansions which converge everywhere in the
complex plane.
Since all the operations that produce standard functions can be applied to complex
functions we can produce all the standard functions of a complex variable by the same
steps as go to producing standard functions of real variables.
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Complex Number
A complex number is a number that can be expressed in the form a + bi,
where a and b are real numbers, and i represents the imaginary unit, satisfying the
equation i2 = −1. Because no real number satisfies this equation, i is called an imaginary
number. For the complex number a + bi, a is called the real part, and b is called
the imaginary part.
Complex numbers allow solutions to certain equations that have no solutions in real
numbers. For example, the equation
has no real solution, since the square of a real number cannot be negative. Complex
numbers, however, provide a solution to this problem. The idea is to extend the real
numbers with an indeterminate i (sometimes called the imaginary unit) taken to satisfy
the relation i2 = −1, so that solutions to equations like the preceding one can be found.
In this case, the solutions are −1 + 3i and −1 − 3i, as can be verified using the fact
that i2 = −1:
A complex number can be visually represented as a pair of
numbers (a, b) forming a vector on a diagram called
an Argand diagram, representing the complex plane. "Re"
is the real axis, "Im" is the imaginary axis,
and i satisfies i2 = −1.
According to the fundamental theorem of algebra, all polynomial equations with
real or complex coefficients in a single variable have a solution in complex numbers. In
contrast, some polynomial equations with real coefficients have no solution in real
numbers. The 16th-century Italian mathematician Gerolamo Cardano is credited with
introducing complex numbers—in his attempts to find solutions to cubic equations.
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Formally, the complex number system can be defined as the algebraic extension of the
ordinary real numbers by an imaginary number i. This means that complex numbers can
be added, subtracted and multiplied as polynomials in the variable i, under the rule
that i2 = −1. Furthermore, complex numbers can also be divided by nonzero complex
numbers.
Based on the concept of real numbers, a complex number is a number of the
form a + bi, where a and b are real numbers and i is an indeterminate satisfying i2 = −1.
For example, 2 + 3i is a complex number.
This way, a complex number is defined as a polynomial with real coefficients in the
single indeterminate i, for which the relation i2 + 1 = 0 is imposed. Based on this
definition, complex numbers can be added and multiplied, using the addition and
multiplication for polynomials. The relation i2 + 1 = 0 induces the equalities i4k =
1, i4k+1 = i, i4k+2 = −1, and i4k+3 = −i, which hold for all integers k; these allow the
reduction of any polynomial that results from the addition and multiplication of complex
numbers to a linear polynomial in i, again of the form a + bi with real coefficients a, b.
The real number a is called the real part of the complex number a + bi; the real
number b is called its imaginary part. To emphasize, the imaginary part does not
include a factor i; that is, the imaginary part is b, not bi.
Cartesian Complex Plane
A complex number z can thus be identified with
an ordered pair (Re(z), Im(z)) of real numbers, which in
turn may be interpreted as coordinates of a point in a
two-dimensional space. The most immediate space is
the Euclidean plane with suitable coordinates, which is
then called complex plane or Argand diagram named
after Jean-Robert Argand. Another prominent space on
which the coordinates may be projected is the twodimensional surface of a sphere, which is then
called Riemann sphere.
The definition of the complex numbers involving two arbitrary real values
immediately suggests the use of Cartesian coordinates in the complex plane. The
horizontal (real) axis is generally used to display the real part, with increasing values to
the right, and the imaginary part marks the vertical (imaginary) axis, with increasing
values upwards.
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
A charted number may be either viewed as the coordinatized point, or as a position
vector from the origin to this point. The coordinate values of a complex number z can
hence be expressed its Cartesian, rectangular, or algebraic form.
Notably, the operations of addition and multiplication take on a very natural geometric
character, when complex numbers are viewed as position vectors: addition corresponds
to vector addition, while multiplication corresponds to multiplying their magnitudes and
adding the angles they make with the real axis. Viewed in this way, the multiplication of
a complex number by i corresponds to rotating the position vector counterclockwise by
a quarter turn (90°) about the origin—a fact which can be expressed algebraically as
follows:
Polar Complex Plane
It is a two-dimensional coordinate system in which
each point on a plane is determined by a distance from a
reference point and an angle from a reference direction.
The reference point (analogous to the origin of
a Cartesian coordinate system) is called the pole, and
the ray from the pole in the reference direction is the polar
axis. The distance from the pole is called the radial coordinate, radial distance or
simply radius, and the angle is called the angular coordinate, polar angle, or azimuth.
The radial coordinate is often denoted by r or ρ, and the angular coordinate by φ, θ, or t.
Angles in polar notation are generally expressed in either degrees or radians (2π rad
being equal to 360°).
Converting Between Polar And Cartesian Coordinates
The polar coordinates r and φ can be converted to the Cartesian
coordinates x and y by using the trigonometric functions sine and cosine:
The Cartesian coordinates x and y can be converted to polar
coordinates r and φ with r ≥ 0 and φ in the interval (−π, π] by:
(as in the Pythagorean theorem or the Euclidean norm), and
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
where atan2 is a common variation on the arctangent function defined as
If r is calculated first as above, then this formula for φ may be stated a little more
simply using the standard arccosine function:
The value of φ above is the principal value of the complex number
function arg applied to x + iy. An angle in the range [0, 2π) may be obtained by adding
2π to the value in case it is negative (in other words when y is negative).
Modulus And Argument
An alternative option for coordinates in the complex
plane is the polar coordinate system that uses the distance of
the point z from the origin (O), and the angle subtended
between the positive real axis and the line segment Oz in a
counterclockwise sense. This leads to the polar form of
complex numbers.
The absolute value (or modulus or magnitude) of a complex
number z = x + yi is
If z is a real number (that is, if y = 0), then r = |x|. That is, the absolute value of a
real number equals its absolute value as a complex number.
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
By Pythagoras' theorem, the absolute value of a complex number is the distance
to the origin of the point representing the complex number in the complex plane.
The argument of z (in many applications referred to as the "phase" φ) is the
angle of the radius Oz with the positive real axis, and is written as arg(z). As with the
modulus, the argument can be found from the rectangular form x+yi —by applying the
inverse tangent to the quotient of imaginary-by-real parts. By using a half-angle identity,
a single branch of the arctan suffices to cover the range of the arg-function, (−π, π], and
avoids a more subtle case-by-case analysis
Normally, as given above, the principal value in the interval (−π, π] is
chosen. Values in the range [0, 2π) are obtained by adding 2π—if the value is
negative. The value of φ is expressed in radians in this article. It can increase by
any integer multiple of 2π and still give the same angle, viewed as subtended by
the rays of the positive real axis and from the origin through z. Hence, the arg
function is sometimes considered as multivalued. The polar angle for the
complex number 0 is indeterminate, but arbitrary choice of the polar angle 0 is
common.
The value of φ equals the result of atan2:
Together, r and φ give another way of representing complex numbers, the polar
form, as the combination of modulus and argument fully specify the position of a point
on the plane. Recovering the original rectangular co-ordinates from the polar form is
done by the formula called trigonometric form
Using Euler's formula this can be written as
Using the cis function, this is sometimes abbreviated to
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
In angle notation, often used in electronics to represent a phasor with
amplitude r and phase φ, it is written as
Equality
Two complex numbers are equal if and only if both their real and imaginary parts are
equal. That is, complex numbers z1 and z2 are equal if and only if Re(z1) =
Re(z2) and Im(z1) = Im(z2). Nonzero complex numbers written in polar form are equal if
and only if they have the same magnitude and their arguments differ by an integer
multiple of 2π.
Ordering
Since complex numbers are naturally thought of as existing on a two-dimensional plane,
there is no natural linear ordering on the set of complex numbers. In fact, there is
no linear ordering on the complex numbers that is compatible with addition and
multiplication – the complex numbers cannot have the structure of an ordered field. This
is because any square in an ordered field is at least 0, but i2 = −1.
Conjugate
The complex conjugate of the complex number z = x + yi is given by x − yi. It is
denoted by either 𝑧 or z*. This unary operation on complex numbers cannot be
expressed by applying only their basic operations addition, subtraction, multiplication
and division.
Geometrically, 𝑧 is the "reflection" of z about the real axis. Conjugating twice gives the
original complex number
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The product of a complex number z = x + y and its conjugate is known as the absolute
square. It is always a positive real number and equals the square of the magnitude of
each:
This property can be used to convert a fraction with a complex denominator to an
equivalent fraction with a real denominator by expanding both numerator and
denominator of the fraction by the conjugate of the given denominator. This process is
sometimes called "rationalization" of the denominator (although the denominator in the
final expression might be an irrational real number), because it resembles the method to
remove roots from simple expressions in a denominator.
The real and imaginary parts of a complex number z can be extracted using the
conjugation:
Moreover, a complex number is real if and only if it equals its own conjugate.
Conjugation distributes over the basic complex arithmetic operations:
Conjugation is also employed in inversive geometry, a branch of geometry studying
reflections more general than ones about a line. In the network analysis of electrical
circuits, the complex conjugate is used in finding the equivalent impedance when
the maximum power transfer theorem is looked for.
Addition And Subtraction
Two complex numbers a and b are most easily added by separately adding their
real and imaginary parts of the summands. That is to say:
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Multiplication
Since the real part, the imaginary part, and the indeterminate i in a complex number are
all considered as numbers in themselves, two complex numbers, given as z = x + yi
and w = u + vi are multiplied under the rules of the distributive property,
the commutative properties and the defining property i2 = -1 in the following way
Reciprocal And Division
Multiplication And Division In Polar Form
Formulas for multiplication, division and exponentiation are simpler in polar form
than the corresponding formulas in Cartesian coordinates. Given two complex
numbers z1 = r1(cos φ1 + i sin φ1) and z2 = r2(cos φ2 + i sin φ2), because of the
trigonometric identities
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
In other words, the absolute values are multiplied and the arguments are added to yield
the polar form of the product. For example, multiplying by i corresponds to a quarterturn counter-clockwise, which gives back i2 = −1. The picture at the right illustrates the
multiplication of
Since the real and imaginary parts of 5 + 5i are equal, the argument of that number is
45 degrees, or π/4 (in radian). On the other hand, it is also the sum of the angles at the
origin of the red and blue triangles are arctan (1/3) and arctan(1/2), respectively. Thus,
the formula
holds. As the arctan function can be approximated highly efficiently, formulas like this –
known as Machin-like formulas – are used for high-precision approximations of π.
Similarly, division is given by
Video Links
Introduction to complex numbers
• https://www.youtube.com/watch?v=SP-YJe7Vldo
Complex Variables and Functions
• https://www.youtube.com/watch?v=iUhwCfz18os&lis
t=PLdgVBOaXkb9CNMqbsL9GTWwU542DiRrPB
Complex Variables and Functions
• https://www.youtube.com/watch?v=iUhwCfz18os&lis
t=PLdgVBOaXkb9CNMqbsL9GTWwU542DiRrPB
References
•
https://ocw.mit.edu/ans7870/18/18.013a/textbook
/HTML/chapter18/section02.html
•
https://en.wikipedia.org/wiki/Complex_number#:
~:text=A%20complex%20number%20is%20a,is%
20called%20an%20imaginary%20number.
10
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 2
Laplace and Inverse
Laplace Transform
Objectives
After completing this chapter, you will be able to:
▪
▪
▪
Identify and Understand the Laplace and Inverse
Laplace Transform
Lists and apply the Properties and Theorem of
Laplace Transform
Convert Laplace Transform into Inverse Laplace
Transform
Introduction
In mathematics, the Laplace transform, named after its inventor Pierre-Simon
Laplace (/ləˈplɑːs/), is an integral transform that converts a function of a real
variable t (often time) to a function of a complex variable s complex frequency). The
transform has many applications in science and engineering because it is a tool for
solving differential equations. In particular, it transforms differential equations into
algebraic equations and convolution into multiplication.
The Laplace transform is named after mathematician and astronomer Pierre-Simon
Laplace, who used a similar transform in his work on probability theory. Laplace wrote
extensively about the use of generating functions in Essai philosophique sur les
probabilités (1814), and the integral form of the Laplace transform evolved naturally as a
result.
Laplace's use of generating functions was similar to what is now known as the ztransform, and he gave little attention to the continuous variable case which was
discussed by Niels Henrik Abel. The theory was further developed in the 19th and early
20th centuries by Mathias Lerch, Oliver Heaviside, and Thomas Bromwich.
The current widespread use of the transform (mainly in engineering) came about during
and soon after World War II, replacing the earlier Heaviside operational calculus. The
advantages of the Laplace transform had been emphasized by Gustav Doetsch to whom
the name Laplace Transform is apparently due.
Laplace Transform
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The Laplace transform of a function f(t), defined for all real numbers t ≥ 0, is the
function F(s), which is a unilateral transform defined by
where s is a complex number frequency parameter
An alternate notation for the Laplace transform is ℒ {𝑓} instead of F.
The meaning of the integral depends on types of functions of interest. A necessary
condition for existence of the integral is that f must be locally integrable on [0, ∞). For
locally integrable functions that decay at infinity or are of exponential type, the integral
can be understood to be a (proper) Lebesgue integral. However, for many applications it
is necessary to regard it as a conditionally convergent improper integral at ∞. Still more
generally, the integral can be understood in a weak sense, and this is dealt with below.
One can define the Laplace transform of a finite Borel measure μ by the Lebesgue
integral
An important special case is where μ is a probability measure, for example, the Dirac
delta functions. In operational calculus, the Laplace transform of a measure is often
treated as though the measure came from a probability density function f. In that case, to
avoid potential confusion, one often writes
This limit emphasizes that any point mass located at 0 is entirely captured by the Laplace
transform. Although with the Lebesgue integral, it is not necessary to take such a limit, it
does appear more naturally in connection with the Laplace–Stieltjes transform.
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Bilateral Laplace transform
When one says "the Laplace transform" without qualification, the unilateral or one-sided
transform is usually intended. The Laplace transform can be alternatively defined as
the bilateral Laplace transform, or two-sided Laplace transform, by extending the limits of
integration to be the entire real axis. If that is done, the common unilateral transform
simply becomes a special case of the bilateral transform, where the definition of the
function being transformed is multiplied by the Heaviside step function.
The bilateral Laplace transform F(s) is defined as follows:
Probability theory
In pure and applied probability, the Laplace transform is defined as an expected
value. If X is a random variable with probability density function f, then the Laplace
transform of f is given by the expectation
By convention, this is referred to as the Laplace transform of the random variable X itself.
Here, replacing s by −t gives the moment generating function of X. The Laplace
transform has applications throughout probability theory, including first passage
times of stochastic processes such as Markov chains, and renewal theory.
Of particular use is the ability to recover the cumulative distribution function of a
continuous random variable X, by means of the Laplace transform as follows:
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Properties And Theorems
The Laplace transform has a number of properties that make it useful for analyzing
linear dynamical systems. The most significant advantage is that differentiation becomes
multiplication,
and integration becomes
division,
by s (reminiscent
of
the
way logarithms change multiplication to addition of logarithms).
Because of this property, the Laplace variable s is also known as operator variable in
the L domain: either derivative operator or (for s−1) integration operator. The transform
turns integral equations and differential equations to polynomial equations, which are
much easier to solve. Once solved, use of the inverse Laplace transform reverts to the
original domain.
Given the functions f(t) and g(t), and their respective Laplace transforms F(s) and G(s),
Computation Of The Laplace Transform Of A Function's Derivative
It is often convenient to use the differentiation property of the Laplace transform to find
the transform of a function's derivative. This can be derived from the basic expression
for a Laplace transform as follows:
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Table Of Selected Laplace Transforms
The following table provides Laplace transforms for many common functions of a single
variable. For definitions and explanations, see the Explanatory Notes at the end of the
table.
Because the Laplace transform is a linear operator,
Using this linearity, and various trigonometric, hyperbolic, and complex number (etc.)
properties and/or identities, some Laplace transforms can be obtained from others more
quickly than by using the definition directly.
The unilateral Laplace transform takes as input a function whose time domain is the nonnegative reals, which is why all of the time domain functions in the table below are
multiples of the Heaviside step function, u(t).
The entries of the table that involve a time delay τ are required to be causal (meaning
that τ > 0). A causal system is a system where the impulse response h(t) is zero for all
time t prior to t = 0. In general, the region of convergence for causal systems is not the
same as that of anticausal systems.
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Inverse Laplace Transform
In mathematics, the inverse Laplace transform of a function F(s) is the piecewisecontinuous and exponentially-restricted real function f(t) which has the property:
It can be proven that, if a function F(s) has the inverse Laplace transform f(t), then f(t) is
uniquely determined (considering functions which differ from each other only on a point
set having Lebesgue measure zero as the same). This result was first proven
by Mathias Lerch in 1903 and is known as Lerch's theorem.
The Laplace transform and the inverse Laplace transform together have a number of
properties that make them useful for analyzing linear dynamical systems.
Mellin's Inverse Formula
An integral formula for the inverse Laplace transform, called the Mellin's inverse
formula, the Bromwich integral, or the Fourier–Mellin integral, is given by the line
integral:
where the integration is done along the vertical line Re(s) = γ in the complex plane such
that γ is greater than the real part of all singularities of F(s) and F(s) is bounded on the
line, for example if contour path is in the region of convergence. If all singularities are in
the left half-plane, or F(s) is an entire function, then γ can be set to zero and the above
inverse integral formula becomes identical to the inverse Fourier transform.
In practice, computing the complex integral can be done by using the Cauchy residue
theorem.
Post's Inversion Formula
Post's inversion formula for Laplace transforms, named after Emil Post, is a simplelooking but usually impractical formula for evaluating an inverse Laplace transform.
The statement of the formula is as follows: Let f(t) be a continuous function on the
interval [0, ∞) of exponential order, i.e.
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
for some real number b. Then for all s > b, the Laplace transform for f(t) exists and is
infinitely differentiable with respect to s. Furthermore, if F(s) is the Laplace transform
of f(t), then the inverse Laplace transform of F(s) is given by
for t > 0, where F(k) is the k-th derivative of F with respect to s.
As can be seen from the formula, the need to evaluate derivatives of arbitrarily high
orders renders this formula impractical for most purposes.
With the advent of powerful personal computers, the main efforts to use this formula have
come from dealing with approximations or asymptotic analysis of the Inverse Laplace
transform, using the Grunwald–Letnikov differ integral to evaluate the derivatives.
Post's inversion has attracted interest due to the improvement in computational science
and the fact that it is not necessary to know where the poles of F(s) lie, which make it
possible to calculate the asymptotic behavior for big x using inverse Mellin transforms for
several arithmetical functions related to the Riemann hypothesis.
Video Links
Laplace transform
• https://www.youtube.com/watch?v=OiNh2DswFt4
Inverse Laplace Transform Example
• https://www.youtube.com/watch?v=c6YnYr8KsSo
Inverse Laplace Transform Example
• https://www.youtube.com/watch?v=c6YnYr8KsSo
References
•
•
•
https://en.wikipedia.org/wiki/Laplace_transform
https://en.wikipedia.org/wiki/Inverse_Laplace_tra
nsform
https://web.stanford.edu/~boyd/ee102/laplace.pd
f
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 3
Power Series
Objectives
After completing this chapter, you will be able to:
•
Illustrate the interval of convergence for a
power series.
• Differentiate and integrate a power series to
obtain other power series.
• Find Maclaurin series for a function.
• Find Taylor series for a function.
Introduction
In this module you will learn to represent power series algebraically
and graphically. The graphical representation of power series can be used to
illustrate the amazing concept that certain power series converge to well-known
functions on certain intervals. In the first lesson you will start with a power series
and determine the function represented by the series. In the last two lessons
you will begin with a function and find its power series representation.
Power Series
In this lesson you will study several power series and discover that on the
intervals where they converge, they are equal to certain well known functions.
Defining Power Series
A power series is a series in which each term is a constant times a power
of x or a power of (x - a) where a is a constant.
Suppose each ck represents some constant. Then the infinite series
is a power series centered at x = 0 and the infinite series
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
is a power series centered at x = a.
Finding Partial Sums Of A Power Ser Ies
Consider the power series
Although you cannot enter infinitely many terms of this series in the Y=
Editor, you can graph partial sums of the series because each partial sum is a
polynomial with a finite number of terms.
Defining An Infinite Geometric Series
Recall that an infinite geometric series can be written as a + ar + ar2 + ar3
+ ... + ark…, where a represents the first term and r represents the common
ratio of the series. If | r | < 1, the infinite geometric series a + ar + ar 2 + ar3 + ... +
ark + ... converges to a/(1-r).
The power series is a geometric series with first term 1 and common ratio x.
This means that the power series converges when | x | < 1 and converges to
1/(1-x) on the interval (-1, 1).
Visualizing Convergence
The graphs of several partial sums can illustrate the interval of
convergence for an infinite series.
•
Graph the second-, third-, and fourth-degree polynomials that
•
represent partial sums of
Graph y=1/(1-x)
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
•
Change the graphing style for y=1/(1-x)
from the partial sums.
to "thick" to distinguish it
On the interval (-1,1) the partial sums are close to y=1/(1-x). The interval
(-1,1) is called the interval of convergence for this power series because as the
number of terms in the partial sums increases, the partial sums converge to
y=1/(1-x) on that interval.
Maclaurin Series
In the previous lesson you explored several power series and their
relationships to the functions to which they converge. In this lesson you will start
with a function and find the power series that best converges to that function for
values of x near zero.
Suppose f is some function. A second-degree polynomial p(x) = ax2 + bx + c that
satisfies p(0)
= f(0),
p'(0) = f'(0), and p"(0) = f"(0), gives a good approximation of f near x = 0. The procedure
below illustrates the method by finding a quadratic polynomial that satisfies these
conditions for the function f(x) = ex.
Let f(x) = ex and p(x) = ax2 + bx + c.
Use the fact that f and p are equal at x = 0 to find the value of c.
f(x) = ex
f(0) = e0 = 1
p(x) = ax2 + bx + c
p(0) = a(0)2 + b(0) + c = c
If c = 1, the function and the polynomial have the same value at x = 0, so p(x)
= ax2 + bx + 1.
Using a similar procedure, set the first derivatives equal when x = 0 and solve for b.
f '(x) = ex
f '(0) = e0 = 1
p '(x) = 2ax + b
p '(0) = 2a(0) + b = b
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
If b = 1, the function and the polynomial will have the same slope at x = 0, so p(x)
= ax2 + x + 1.
Set the second derivatives equal when x = 0 and solve for a.
f "(x) = ex
f "(0) = e0 = 1
p "(x) = 2a
p "(0) = 2a
If a = 1/2, the function and the polynomial will have the same concavity at x = 0.
approximates y = ex near x = 0.
So
Graphs Of The Function And The Approximating Quadratic Polynomial
•
Graph the functions Y1 = e^X and Y2 = (1/2)X2 + X + 1 in a [-5, 5, 1] x [-2, 10,1]
window.
The parabola has the same value, the same slope, and the same concavity
as y = ex when x = 0, and the quadratic polynomial is a good approximation
for y = ex when x is near 0.
Taylor Series
In the previous lesson, you found Maclaurin series that approximate
functions near x = 0. This lesson investigates how to find a series that
approximates a function near x = a, where a is any real number.
Given a function f that has all its higher order derivatives, the series
, where
is called the Taylor series for f centered at a. The Taylor series is a power series that
approximates the function f near x = a.
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The partial sum is called the nth-order Taylor polynomial for f centered at a.
Every Maclaurin series is a Taylor series centered at zero.
The Taylor Polynomial Of Ex Centered At 1
The second-order Taylor polynomial centered at 1 for the function f(x) = ex can
be found by using a procedure similar to the procedure given.
The coefficient of the term (x - 1)k in the Taylor polynomial is given by
.
This formula is very similar to the formula for finding the coefficient of xk in a Maclaurin
polynomial where the derivative is evaluated at 0. In this Taylor polynomial, the
derivative is evaluated at 1, the center of the series.
The coefficients of the second-order Taylor polynomial centered at 1 for ex are
f(1) = e
f '(1) = e
So the second-order Taylor polynomial for ex centered at 1 is
and near x = 1, ex P2(x).
,
The Taylor series for ex centered at 1 is similar to the Maclaurin series
for e found in last topic. However, the terms in the Taylor series have powers of (x - 1)
rather than powers of x and the coefficients contain the values of the derivatives
evaluated at x = 1 rather than evaluated at x = 0.
x
Graphing the function and the polynomial illustrate that the polynomial is a good
approximation near x = 1.
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
•
Graph y = ex and
in a [-2, 3, 1] x [-3, 10, 1] window.
The second-order Maclaurin polynomial you found in last topic,
,
x
x
is tangent to f(x) = e at x = 0 and has the same concavity as f(x) = e at that point. The
polynomial
, which is centered at x = 1, is tangent to f(x)
x
= e at x = 1 and has the same concavity as f(x) = ex at that point.
Video Links
Power Series
•
•
https://www.youtube.com/watch?v=EGni2-m5yxM
https://www.youtube.com/watch?v=DlBQcj_zQk0
Maclaurin and Taylor Series
•
•
https://www.youtube.com/watch?v=LDBnS4c7YbA
https://www.youtube.com/watch?v=3d6DsjIBzJ4
References
•
http://education.ti.com/html/t3_free_courses/calcu
lus84_online/mod24/mod24_1.html
•
Kreyszig, Erwin (2011). Advanced Engineering
Mathematics, 10th ed. Wiley
https://blogs.ubc.ca/infiniteseriesmodule/units/unit3-power-series/taylor-series/the-taylor-series/
https://blogs.ubc.ca/infiniteseriesmodule/units/unit3-power-series/power-series/the-power-series/
•
•
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 4
Fourier Series
Objectives
After completing this chapter, you will be able to:
▪
▪
▪
Define Fourier Series.
Identify Complex-valued Function.
Identify a superposition of an infinite number of sine
and cosine function.
INTRODUCTION
In mathematics, a Fourier series is a periodic function composed of harmonically
related sinusoids, combined by a weighted summation. With appropriate weights, one
cycle (or period) of the summation can be made to approximate an arbitrary function in
that interval. As such, the summation is a synthesis of another function. The discretetime Fourier transform is an example of Fourier series. The process of deriving the
weights that describe a given function is a form of Fourier analysis. For functions on
unbounded intervals, the analysis and synthesis analogies are Fourier transform and
inverse transform.
Definition of Fourier Series
Consider a real-valued function, s(x) that is integrable on an interval of length P,
which will be the period of the Fourier series. Common examples of analysis intervals
are:
The analysis process determines the weights, indexed by integer n, which is also the
number of cycles of the 𝒏𝒕𝒉 harmonic in the analysis interval. Therefore, the length of a
cycle, in the units of x, is P/n. And the corresponding harmonic frequency is n/P. The
𝑛
𝑛
𝒏𝒕𝒉 harmonic are sin(2𝜋𝑥 𝑃) andcos(2𝜋𝑥 𝑃), and their amplitudes (weights) are found by
integration over the interval of length P.
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
•
If s(x) is P-periodic, then any interval of that length is sufficient.
•
𝑎0 and 𝑏0 can be reduced to 𝑎0 =𝑃 ∫𝑝 𝑠(𝑥 )𝑑𝑥 and 𝑏0 = 0.
•
Many texts choose P=2𝝅 to simplify the argument of the sinusoid functions.
2
The synthesis process (the actual Fourier series) is:
In general, integer N is theoretically infinite. Even so, the series might not
converge or exactly equate to s(x) at all values of x in the analysis interval. For the
"well-behaved" functions typical of physical processes, equality is customarily assumed.
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
And definitions
the sine and cosine pairs
can be expressed as a single sinusoid with a phase offset, analogous to the conversion
between orthogonal (Cartesian) and polar coordinates:
The customary form for generalizing to complex-valued
(next section) is obtained
using Euler's formula to split the cosine function into complex exponentials.
Here, complex conjugation is denoted by an asterisk:
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Complex-valued functions
If s(x) is a complex-valued function of a real variable x both components (real
and imaginary part) are real-valued functions that can be represented by a Fourier
series. The two sets of coefficients and the partial sum are given by:
Video Links
Fourier Series
•
https://www.youtube.com/watch?v=vA9dfINW4Rg
Computing Fourier Series
•
https://www.youtube.com/watch?v=KfRE744AFEE
Fourier Series Introduction
•
https://www.khanacademy.org/science/electricalengineering/ee-signals/ee-fourier-series/v/eefourier-series-intro
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
References
•
•
•
https://eng.libretexts.org/Bookshelves/Electrical
_Engineering/Book%3A_Electrical_Engineering_
(Johnson)/04%3A_Frequency_Domain/4.02%3A_
Complex_Fourier_Series
https://en.wikipedia.org/wiki/Fourier_series
https://eng.libretexts.org/Bookshelves/Electrical
_Engineering/Book%3A_Electrical_Engineering_
(Johnson)/04%3A_Frequency_Domain/4.03%3A_
Classic_Fourier_Series
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 5
Fourier Transform
Objectives
After completing this chapter, you will be able to:
▪
▪
▪
Identify the Fourier
Understand the characteristic of a Fourier Transform
Identify the branches of Fourier
Introduction
We’re about to make the transition from Fourier series to the Fourier transform.
“Transition” is the appropriate word, for in the approach we’ll take the Fourier transform
emerges as we pass from periodic to nonperiodic functions. To make the trip we’ll view
a nonperiodic function (which can be just about anything) as a limiting case of a periodic
function as the period becomes longer and longer. Actually, this process doesn’t
immediately produce the desired result. It takes a little extra tinkering to coax the
Fourier transform out of the Fourier series, but it’s an interesting approach. Fourier was
elected to the Académie des Sciences in 1817. During Fourier's eight remaining years
in Paris, he resumed his mathematical researches, publishing a number of important
articles. Fourier's work triggered later contributions on trigonometric series and the
theory of functions of real variable.
The Fourier transform is crucial to any discussion of time series analysis, and this
chapter discusses the definition of the transform and begins introducing some of the
ways it is useful.
As an aside, I don’t know if this is the best way of motivating the definition of the Fourier
transform, but I don’t know a better way and most sources you’re likely to check will just
present the formula as a done deal. It’s true that, in the end, it’s the formula and what
we can do with it that we want to get to, so if you don’t find the (brief) discussion to
follow to your tastes, I am not offended. called, variously, the top hat function (because
of its graph), the indicator function, or the characteristic function for the interval (−1/2,
1/2). While we have defined Π(±1/2) = 0, other common conventions are either to have
Π(±1/2) = 1 or Π(±1/2) = 1/2. And some people don’t define Π at ±1/2 at all, leaving two
holes in the domain. I don’t want to get dragged into this dispute. It almost never
matters, though for some purposes the choice Π(±1/2) = 1/2 makes the most sense.
We’ll deal with this on an exceptional basis if and when it comes up.
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Π(t) is not periodic. It doesn’t have a Fourier series. In problems you experimented a
little with periodizations, and I want to do that with Π but for a specific purpose. As a
periodic version of Π(t) we repeat the nonzero part of the function at regular intervals,
separated by (long) intervals where the function is zero. We can think of such a function
arising when we flip a switch on for a second at a time, and do so repeatedly, and we
keep it off for a long time in between the times it’s on. (One often hears the term duty
cycle associated with this sort of thing.) Here’s a plot of Π(t) periodized to have period
15.
Here are some plots of the Fourier coefficients of periodized rectangle functions with
periods 2, 4, and 16, respectively. Because the function is real and even, in each case
the Fourier coefficients are real, so these are plots of the actual coefficients, not their
square magnitudes.
We see that as the period increases the frequencies are getting closer and closer
together and it looks as though the coefficients are tracking some definite curve. (But
we’ll see that there’s an important issue here of vertical scaling.) We can analyze what’s
going on in this particular example, and combine that with some general statements to
lead us on. Recall that for a general function f(t) of period T the Fourier series has the
form
so that the frequencies are 0, ±1/T, ±2/T, . . .. Points in the spectrum are spaced 1/T
apart and, indeed, in the pictures above the spectrum is getting more tightly packed as
the period T increases. The n-th Fourier coefficient is given by
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
We see that as the period increases the frequencies are getting closer and closer
together and it looks as though the coefficients are tracking some definite curve. (But
we’ll see that there’s an important issue here of vertical scaling.) We can analyze what’s
going on in this particular example, and combine that with some general statements to
lead us on.
Recall that for a general function f(t) of period T the Fourier series has the form
so that the frequencies are 0, ±1/T, ±2/T, . . .. Points in the spectrum are spaced 1/T
apart and, indeed, in the pictures above the spectrum is getting more tightly packed as
the period T increases. The n-th Fourier coefficient is given by
Here’s a graph. You can now certainly see the continuous curve that the plots of
the discrete, scaled Fourier coefficients are shadowing.
The function sin πx/πx (written now with a generic variable x) comes up so often
in this subject that it’s given a name, sinc:
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
How general is this? We would be led to the same idea — scale the Fourier
coefficients by T — if we had started off periodizing just about any function with the
intention of letting T → ∞. Suppose f(t) is zero outside of |t| ≤ 1/2. (Any interval will do,
we just want to suppose a function is zero outside some interval so we can periodize.)
We periodize f(t) to have period T and compute the Fourier coefficients:
How big is this? We can estimate
Where
Fourier transform defined There you have it. We now define the Fourier
transform of a function f(t) to be
For now, just take this as a formal definition; we’ll discuss later when such an
integral exists. We assume that f(t) is defined for all real numbers t. For any s ∈ R,
integrating f(t) against e−2πist with respect to t produces a complex valued function of
s, that is, the Fourier transform ˆf(s) is a complex-valued function of s ∈ R. If t has
dimension time then to make st dimensionless in the exponential e−2πist s must have
dimension 1/time.
While the Fourier transform takes flight from the desire to find spectral
information on a nonperiodic function, the extra complications and extra richness of
what results will soon make it seem like we’re in a much different world. The definition
just given is a good one because of the richness and despite the complications. Periodic
functions are great, but there’s more bang than buzz in the world to analyze. The
spectrum of a periodic function is a discrete set of frequencies, possibly an infinite set
(when there’s a corner) but always a discrete set. By contrast, the Fourier transform of a
nonperiodic signal produces a continuous spectrum, or a continuum of frequencies. It
may be that ˆf(s) is identically zero for |s| sufficiently large — an important class of
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
signals called bandlimited — or it may be that the nonzero values of ˆf(s) extend to ±∞,
or it may be that ˆf(s) is zero for just a few values of s. The Fourier transform analyzes a
signal into its frequency components. We haven’t yet considered how the corresponding
synthesis goes. How can we recover f(t) in the time domain from ˆf(s) in the frequency
domain?
Recovering f(t) from ˆf(s) We can push the ideas on nonperiodic functions as
limits of periodic functions a little further and discover how we might obtain f(t) from its
transform ˆf(s). Again suppose f(t) is zero outside some interval and periodize it to have
(large) period T. We expand f(t) in a Fourier series,
The inverse Fourier transform defined, and Fourier inversion, too The
integral we’ve just come up with can stand on its own as a “transform”, and so we define
the inverse Fourier transform of a function g(s) to be
Again, we’re treating this formally for the moment, withholding a discussion of
conditions under which the integral makes sense. In the same spirit, we’ve also
produced the Fourier inversion theorem. That is
A quick summary Let’s summarize what we’ve done here, partly as a guide to
what we’d like to do next. There’s so much involved, all of importance, that it’s hard to
avoid saying everything at once. Realize that it will take some time before everything is
in place.
• The domain of the Fourier transform is the set of real numbers s. One says that
ˆf is defined on the frequency domain, and that the original signal f(t) is defined on the
time domain (or the spatial domain, depending on the context). For a (nonperiodic)
signal defined on the whole real line we generally do not have a discrete set of
frequencies, as in the periodic case, but rather a continuum of frequencies.2 (We still do
call them “frequencies”, however.) The set of all frequencies is the spectrum of f(t).
◦ Not all frequencies need occur, i.e., ˆf(s) might be zero for some values of s.
Furthermore, it might be that there aren’t any frequencies outside of a certain range,
i.e.,
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The Inverse Fourier Transform is defined by
Now remember that ˆf (s) is a transformed, complex-valued function, and while it
may be “equivalent” to f(t) it has very different properties. Is it really true that when ˆf(s)
exists we can just plug it into the formula for the inverse Fourier transform — which is
also an improper integral that looks the same as the forward transform except for the
minus sign — and really get back f(t)? Really? That’s worth wondering about.
• The square magnitude | ˆf(s)| 2 is called the power spectrum (especially in
connection with its use in communications) or the spectral power density (especially in
connection with its use in optics) or the energy spectrum (especially in every other
connection). An important relation between the energy of the signal in the time domain
and the energy spectrum in the frequency domain is given by Parseval’s identity for
Fourier transforms:
A warning on notations: None is perfect, all are in use Depending on the
operation to be performed, or on the context, it’s often useful to have alternate notations
for the Fourier transform. But here’s a warning, which is the start of a complaint, which
is the prelude to a full blown rant. Diddling with notation seems to be an unavoidable
hassle in this subject. Flipping back and forth between a transform and its inverse,
naming the variables in the different domains (even writing or not writing the variables),
changing plus signs to minus signs, taking complex conjugates, these are all routine
day-to-day operations and they can cause endless muddles if you are not careful, and
sometimes even if you are careful. You will believe me when we have some examples,
and you will hear me complain about it frequently
Here’s one example of a common convention:
If the function is called f then one often uses the corresponding capital letter, F,
to denote the Fourier transform. So one sees a and A, z and Z, and everything in
between. Note, however, that one typically uses different names for the variable for the
two functions, as in f(x) (or f(t)) and F(s). This ‘capital letter notation’ is very common in
engineering but often confuses people when ‘duality’ is invoked, to be explained below
And then there’s this:
Since taking the Fourier transform is an operation that is applied to a function to
produce a new function, it’s also sometimes convenient to indicate this by a kind of
“operational” notation. For example, it’s common to write Ff(s) for ˆf(s), and so, to repeat
the full definition
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
This is often the most unambiguous notation. Similarly, the operation of taking
the inverse Fourier transform is then denoted by F −1, and so
Finally, a function and its Fourier transform are said to constitute a “Fourier pair”,;
this is concept of ‘duality’ to be explained more precisely later. There have been various
notations devised to indicate this sibling relationship. One is
A warning on definitions Our definition of the Fourier transform is a standard one,
but it’s not the only one. The question is where to put the 2π: in the exponential, as we
have done; or perhaps as a factor out front; or perhaps left out completely. There’s also
a question of which the Fourier transform is and which is the inverse, i.e., which gets the
minus sign in the exponential. All of the various conventions are in day-to-day use in the
professions, and I only mention this now because when you’re talking with a friend over
drinks about the Fourier transform, be sure you both know which conventions are being
followed. I’d hate to see that kind of misunderstanding get in the way of a beautiful
friendship. Following the helpful summary provided by T. W. K¨orner in his book Fourier
analysis, I will summarize the many irritating variations. To be general, let’s write
Getting To Know Your Fourier Transform
In one way, at least, our study of the Fourier transform will run the same course as your
study of calculus. When you learned calculus it was necessary to learn the derivative
and integral formulas for specific functions and types of functions (powers, exponentials,
trig functions), and also to learn the general principles and rules of differentiation and
integration that allow you to work with combinations of functions (product rule, chain
rule, inverse functions). It will be the same thing for us now. We’ll need to have a
storehouse of specific functions and their transforms that we can call on, and we’ll need
to develop general principles and results on how the Fourier transform operates.
The triangle function Consider next the “triangle function”, defined by
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
For the Fourier transform we compute (using integration by parts, and the factoring trick
for the sine function):
It’s no accident that the Fourier transform of the triangle function turns out to be
the square of the Fourier transform of the rect function. It has to do with convolution, an
operation we have seen for Fourier series and will see anew for Fourier transforms in
the next chapter.
The exponential decay another commonly occurring function is the (one-sided)
exponential decay, defined by where a is a positive constant. This function models a
signal that is zero, switched on, and then decays exponentially. Here are graphs for a =
2, 1.5, 1.0, 0.5, 0.25
Which is which? If you can’t say, see the discussion on scaling the independent
variable at the end of this section. Back to the exponential decay, we can calculate its
Fourier transform directly
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Which is which? You’ll soon learn to spot that immediately, relative to the pictures in the
time domain, and it’s an important issue. Also note that |Ff(s)| 2 is an even function of s
even though Ff(s) is not. We’ll see why later. The shape of |Ff(s)| 2 is that of a “bell
curve”, though this is not Gaussian, a function we’ll discuss just below. The curve is
known as a Lorenz profile and comes up in analyzing the transition probabilities and
lifetime of the excited state in atoms.
How does the graph of f(ax) compare with the graph of f(x)? Let me remind you of
some elementary lore on scaling the independent variable in a function and how scaling
affect its graph. The question is how the graph of f(ax) compares with the graph of f(x)
when 0 <a< 1 and when a > 1; I’m talking about any generic function f(x) here. This is
very simple, especially compared to what we’ve done and what we’re going to do, but
you’ll want it at your fingertips and everyone has to think about it for a few seconds.
Here’s how to spend those few seconds.
Consider, for example, the graph of f(2x). The graph of f(2x), compared with the graph
of f(x), is squeezed. Why? Think about what happens when you plot the graph of f(2x)
over, say, −1 ≤ x ≤ 1. When x goes from −1 to 1, 2x goes from −2 to 2, so while you’re
plotting f(2x) over the interval from −1 to 1 you have to compute the values of f(x) from
−2 to 2. That’s more of the function in less space, as it were, so the graph of f(2x) is a
squeezed version of the graph of f(x). Clear? Similar reasoning shows that the graph of
f(x/2) is stretched. If x goes from −1 to 1 then x/2 goes from −1/2 to 1/2, so while you’re
plotting f(x/2) over the interval −1 to 1 you have to compute the values of f(x) from −1/2
to 1/2. That’s less of the function in more space, so the graph of f(x/2) is a stretched
version of the graph of f(x).
For Whom The Bell Curve Tolls
Let’s next consider the Gaussian
function and its Fourier transform. We’ll
need this for many examples and problems.
This function, the famous “bell shaped
curve”, was used by Gauss for various
statistical problems. It has some striking
properties with respect to the Fourier
transform which, on the one hand, give it a
special role within Fourier analysis, and on
the other hand allow Fourier methods to be
applied to other areas where the function
comes up. We’ll see an application to
probability and statistics in this graph.
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
For various applications one throws in extra factors to modify particular
properties of the function. We’ll do this too, and there’s not a complete agreement on
what’s best. There is an agreement that before anything else happens, one has to know
the amazing equation.
Now, the function f(x) = e−x2 does not have an elementary antiderivative, so this
integral cannot be found directly by an appeal to the Fundamental Theorem of Calculus.
The fact that it can be evaluated exactly is one of the most famous tricks in
mathematics. It’s due to Euler, and you shouldn’t go through life not having seen it. And
even if you have seen it, it’s worth seeing again; see the discussion following this
section.
Evaluation Of The Gaussian Integral
We want to evaluate Now we make a change of variables, introducing polar
coordinates, (r, θ). First, what about the limits of integration? To let both x and y range
from −∞ to ∞ is to describe the entire plane, and to describe the entire plane in polar
coordinates is to let r go from 0 to ∞ and θ go from 0 to 2π. Next, e−(x2+y2) becomes
e−r2 and the area element dx dy becomes r dr dθ. It’s the extra factor of r in the area
element that makes all the difference. With the change to polar coordinates we have
General Properties And Formulas
We’ve started to build a storehouse of specific transforms. Let’s now proceed along the
other path awhile and develop some general properties. For this discussion — and
indeed for much of our work over the next few lectures — we are going to abandon all
worries about transforms existing, integrals converging, and whatever other worries you
might be carrying. Relax and enjoy the ride.
Fourier Transform Pairs And Duality
One striking feature of the Fourier transform and the inverse Fourier transform is the
symmetry between the two formulas, something you don’t see for Fourier series. For
Fourier series the coefficients are given by an integral (a transform of f(t) into ˆf(n)), but
the “inverse transform” is the series itself. The Fourier transforms F and F −1 are the
same except for the minus sign in the exponential.5 In words, we can say that if you
replace s by −s in the formula for the Fourier transform then you’re taking the inverse
Fourier transform. Likewise, if you replace t by −t in the formula for the inverse Fourier
transform then you’re taking the Fourier transform. That is
10
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
This might be a little confusing because you generally want to think of the two variables,
s and t, as somehow associated with separate and different domains, one domain for
the forward transform and one for the inverse transform, one for time and one for
frequency, while in each of these formulas one variable is used in both domains. You
have to get over this kind of confusion, because it’s going to come up again. Think
purely in terms of the math: The transform is an operation on a function that produces a
new function. To write down the formula I have to evaluate the transform at a variable,
but it’s only a variable and it doesn’t matter what I call it as long as I keep its role in the
formula straight. Also be observant what the notation in the formula says and, just as
important, what it doesn’t say. The first formula, for example, says what happens when
you first take the Fourier transform of f and then evaluate it at −s, it’s not a formula for
F(f(−s)) as in “first change s to −s in the formula for f and then take the transform”. I
could have written the first displayed equation as (Ff)(−s) = F −1f(s), with an extra
parentheses around the Ff to emphasize this, but I thought that looked too clumsy. Just
be careful, please.
are sometimes referred to as the “duality” property of the transforms. One also says that
“the Fourier transform pair f and Ff are related by duality”, meaning exactly these
relations. They look like different statements but you can get from one to the other. We’ll
set this up a little differently in the next section.
Duality and reversed signals There’s a slightly different take on duality that I prefer
because it suppresses the variables and so I find it easier to remember. Starting with a
signal f(t) define the reversed signal f – by
Duality and reversed signals There’s a slightly different take on duality that I prefer
because it suppresses the variables and so I find it easier to remember. Starting with a
signal f(t) define the reversed signal f – by
This identity is somewhat interesting in itself, as a variant of Fourier inversion. You can
check it directly from the integral definitions, or from our earlier duality results.6 Of
course then also
11
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Even And Odd Symmetries And The Fourier Transform
We’ve already had a number of occasions to use even and odd symmetries of
functions. In the case of real-valued functions the conditions have obvious
interpretations in terms of the symmetries of the graphs; the graph of an even function is
symmetric about the y-axis and the graph of an odd function is symmetric through the
origin. The (algebraic) definitions of even and odd apply to complex-valued as well as to
realvalued functions, however, though the geometric picture is lacking when the function
is complex-valued because we can’t draw the graph. A function can be even, odd, or
neither, but it can’t be both unless it’s identically zero. How are symmetries of a function
reflected in properties of its Fourier transform? I won’t give a complete accounting, but
here are a few important cases.
We can refine this if the function f(t) itself has symmetry. For example, combining the
last two results and remembering that a complex number is real if it’s equal to its
conjugate and is purely imaginary if it’s equal to minus its conjugate, we have:
• If f is real valued and even then its Fourier transform is even and real valued.
• If f is real valued and odd function then its Fourier transform is odd and purely
imaginary.
We saw this first point in action for Fourier transform of the rect function Π(t) and for the
triangle function Λ(t). Both functions are even and their Fourier transforms, sinc and
sinc2, respectively, are even and real. Good thing it worked out that way.
Linearity
One of the simplest and most frequently invoked properties of the Fourier transform is
that it is linear (operating on functions). This means:
The Shift Theorem
A shift of the variable t (a delay in time) has a simple effect on the Fourier transform. We
would expect the magnitude of the Fourier transform |Ff(s)| to stay the same, since
shifting the original signal in time should not change the energy at any point in the
spectrum. Hence the only change should be a phase shift in Ff(s), and that’s exactly
what happens.
12
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The Stretch (Similarity) Theorem
How does the Fourier transform change if we stretch or shrink the variable in the time
domain? More precisely, we want to know if we scale t to at what happens to the
Fourier transform of f(at). First suppose a > 0. Then
If a < 0 the limits of integration are reversed when we make the substitution u = ax, and
so the resulting transform is (−1/a)Ff(s/a). Since −a is positive when a is negative, we
can combine the two cases and present the Stretch Theorem in its full glory:
This is also sometimes called the Similarity Theorem because changing the variable
from x to ax is a change of scale, also known as a similarity
There’s an important observation that goes with the stretch theorem. Let’s take a to be
positive, just to be definite. If a is large (bigger than 1, at least) then the graph of f(at) is
squeezed horizontally compared to f(t). Something different is happening in the
frequency domain, in fact in two ways. The Fourier transform is (1/a)F(s/a). If a is large
then F(s/a) is stretched out compared to F(s), rather than squeezed in. Furthermore,
multiplying by 1/a, since the transform is (1/a)F(a/s), also squashes down the values of
the transform. The opposite happens if a is small (less than 1). In that case the graph of
f(at) is stretched out horizontally compared to f(t), while the Fourier transform is
compressed horizontally and stretched vertically. The phrase that’s often used to
describe this phenomenon is that a signal cannot be localized (meaning concentrated at
a point) in both the time domain and the frequency domain. We will see more precise
formulations of this principle.
To sum up, a function stretched out in the time domain is squeezed in the frequency
domain, and vice versa. This is somewhat analogous to what happens to the spectrum
of a periodic function for long or short periods. Say the period is T, and recall that the
points in the spectrum are spaced 1/T apart, a fact we’ve used several times. If T is
large then it’s fair to think of the function as spread out in the time domain — it goes a
long time before repeating. But then since 1/T is small, the spectrum is squeezed. On
the other hand, if T is small then the function is squeezed in the time domain — it goes
13
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
only a short time before repeating — while the spectrum is spread out, since 1/T is
large.
Video Links
What is the Fourier Transform
•
https://www.youtube.com/watch?v=spUNpyF58BY
Fourier series From heat flow to circle drawings
•
https://www.youtube.com/watch?v=r6sGWTCMz2k
Fourier Transform
•
https://www.youtube.com/watch?v=ykNtIbtCR-8
References
•
•
•
•
•
https://ieeexplore.ieee.org/stamp/stamp.jsp?arn
umber=7389485
file:///C:/Users/Jhoy/Downloads/Chapter%202_F
ourier%20Transform.pdf
http://web.ipac.caltech.edu/staff/fmasci/home/as
tro_refs/TheFourierTransform.pdf
http://www.math.ncku.edu.tw/~rchen/2016%20Te
aching/Chapter%202_Fourier%20Transform.pdf
http://www0.cs.ucl.ac.uk/teaching/GZ05/03fourier.pdf
14
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 6
Power Series Solution Of
Differential Equations
Objectives
After completing this chapter, you will be able to:
▪
▪
Understand the Power Series
Solve Power Series of Differential Equations
In mathematics, the power series method is used to seek a power series solution to
certain differential equations. In general, such a solution assumes a power series with
unknown coefficients, then substitutes that solution into the differential equation to find
a recurrence relation for the coefficients.
We want to continue the discussion of the previous lecture with the focus now on power
series, i.e., where the terms in the series contain powers of some parameter, typically a
dynamical variable, and we are expanding about the point where that parameter
vanishes. (A special case is the geometric series we discussed in Lecture 2.) The power
series then defines a function of that parameter. The standard examples are the Taylor
and Maclaurin series mentioned at the beginning of the previous lecture corresponding
to expanding in a series about x = 0 or 0 x x = , The coefficients n b are related to the
derivatives of the underlying, at least when the series is convergent, i.e., when the
series actually defines the function. Some specific examples of power series are
These series uniquely define the corresponding functions, whenever (i.e., at values of x
where) the series converges. When we apply the convergence tests of the previous
lecture to these expressions, we will define a range for the variable x within which the
series converges, i.e., a range in which the series expansion makes mathematical (and
physical) sense. This range for x is called the interval of convergence (for the power
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
series expansion). For values of x outside of this interval we will need to find different
expressions to define the underlying functions, i.e., the power series serves to define
the function only in the interval of convergence.
So the series absolutely converges for x  2 . At the endpoints,  =1, x = 2 , the series
does not absolutely converge since both the p and s parameters of Eq. (2.29) vanish.
For x =−2 all terms in the series are positive and the series diverges ( S S 1 1 (− = − = 
2 2 ) ( ) ). At the other endpoint,  =1, x = 2 , we must be more careful due to the
alternating signs. We apply test 5) (from Lecture 2) to this series with (2 1 ) ( ) n n n n a
b = = − . Since lim 0 n n → a  , the series again diverges. We can see both of these
results explicitly by looking at the series
Hence the function S x 1 ( ) is well defined (by the series, i.e., the series converges)
only on the open interval −   2 2 x (open means excluding the end points). For S x 2 (
) we find from the ratio test that
So the ratio test says that the series absolutely converges for x 1. At the endpoint x
=−1 the signs are all negative and the series diverges ) , just the negative Harmonic
series). At the other endpoint, x =1 , the signs alternate, test 5) is satisfied (and the
series converges conditionally. Hence the function is well defined (by the power series)
only on the semi-open interval. Note again that we must treat the endpoints carefully.
The third series may look like it is missing every other term (the even powers) and
therefore you may be confused about how to apply the ratio test. This is not an issue.
The idea of the ratio test is to always consider 2 contiguous terms in the series.
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Alternatively we can (re)write the series as (note that now every value of n contributes
to the sum)
The series converges absolutely everywhere, −    x ( S x 3 ( ) is just the
power series expansion of the sine function sin x ).
Thus the series absolutely converges for x the open interval x . Since p =  1 2 1, 4 S
diverges at the end points. For x =−1 there are no alternating signs so S4 (−1) also
diverges. At x =−3 we have ( 1 1 ), which satisfies test 5) and the series conditionally
converges. Hence the interval of convergence for S x 4 ( ) is x .
Now that we have verified that we know how to determine the interval of convergence,
we should restate what is important about the interval. For values of x within that
interval of convergence the following statements are all true (and equivalent).
To further develop this discussion we state the following theorems (without proof), which
are true in the interval of convergence where the (infinite) power series can be treated
like a polynomial.
1. We can integrate or differentiate the series term-by-term to find a definition of the
corresponding integral or derivative of the function defined by the original series,
i.e., S x ( ) or S x dx ( )  . The resulting series has the same interval of
convergence as the original series, except perhaps for the behavior at the
endpoints (see below). [This is why series are so useful. Note that the effect of
integrating or taking a derivative is to introduce a factor of 1/(n+1) or n in the term
bn x n±1 , which does not change the interval of convergence (you should
convince yourself of this fact).]
2. If we have two series defining two functions with known intervals of convergence,
we can add, subtract or multiply the two series to define the corresponding
functions, S x S x 1 2 ( )  ( ) or S x S x 1 2 ( ) ( ) . The new series
convergences in the overlap (or common) interval of the original 2 intervals of
convergence. We can also think about the function defined by dividing the two
series, which with some manipulation we can express as a corresponding power
series. This function is well defined in the overlap interval of the original series
except points where the series in the denominator vanishes. At these points the
function may still be defined if the series in the numerator also vanishes
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
(appropriately quickly). We are guaranteed that there is an interval where the
new function and its series expansion make sense, but we must explicitly
calculate it.
3. We can substitute one series into another series if the value of the first series is
within the interval of convergence of the second,
4. The power series expansion of a function is UNIQUE! No matter how you
determine the coefficients n b in n n b x , once you have found them you are
done. [This is a really important point for the Lazy but Smart!]
So let’s develop power series expansions for the functions we know and love.
We know the general form is given by Eq. (3.1). If we apply the Maclaurin
expansion definition to x e , we find (recall the essential property of the
exponential, x x de dx e = )
For this power series the interval of convergence is all values of x, x  ,
In a similar fashion we can quickly demonstrate that
Again the interval of convergence is all values of x, x  . ASIDE: Note that with
the definition 2 i −1, i  −1 , it follows from Eqs. (3.9) and (3.11) that
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
This last result is called Euler’s formula (and is extremely useful!). We can obtain
more useful relations by using the four theorems above. Recall our discussion in the last
lecture of the geometric and Harmonic series. We had
Note that the integral changes the behavior at one endpoint from divergent in the
geometric case to conditionally convergent in the harmonic case, an example of the
change at the endpoints noted above in item I).
which we could have obtained directly by differentiation of (1 ) p + x . In this context the
reader is encouraged to think some about this use of the factorial function even when
the argument is not an integer, i.e., the expansion in Eq. (3.19) is useful even when n is
not an integer (but m is). Another tool for obtaining a series expansion arises from
switching from a Maclaurin series expansion to a Taylor series expansion, i.e., expand
about a point other than the origin. Consider the function ln x . It is poorly behaved at x
= 0 , but is well behaved at x =1 . So we can use Eq. (3.15) in the form
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The interval of convergence is then given, i.e., the series converges absolutely for 0 2 
x and converges conditionally at x = 2. To repeat, the function defined by a power
series expansion is well behaved within the interval of convergence. But clearly it is
important to consider what it means when the series expansion for a function diverges.
In general, the series will diverge where the original function is singular as is the case
for ln x at x = 0 in Eq. (3.20). However, the relationship between the series and the
function does not always work going the other way. A divergent series does not
necessarily mean a singular function. The various possibilities are the following.
1. The series diverges and the function is singular at the same point. This typically
occurs at the boundary of the interval of convergence as in the example of ln x at
x = 0.
2. The series may be divergent, but the function is well behaved. For example ln x
is well behaved for x  2 but the (specific) power series expansion in Eq. (3.20)
diverges. Similarly the function 1 1( + x) is well behaved everywhere except the
single point x =−1 while the power series expansion in Eq. (3.14) diverges for x
1 . The mathematics behind this behavior is most easily understood in the
formalism of complex variables as we will discuss next (where we will develop
the concept of a radius of convergence to replace the interval of convergence). In
any case, it is clear as we have already noted that outside of the interval of
convergence the series is no longer useful to define the function. However, it is
often possible to find a different power series expansion that is useful
(convergent) in a different interval of convergence.
3. The logic of how a power series expansion is used typically runs like the
following. We solve a differential equation by using the equation to solve for the
coefficients in a power series expansion of the solution Within the interval of
convergence of that series we succeed in summing the series and writing the
solution in closed form We use that closed form to define the solution to the
original differential equation over a much larger range in the variable. (The
essential subtext here is the uniqueness of the solution and the power series.)
4. A third and truly devious possibility is that the series looks “OK” but does not
accurately describe the function. This behavior is associated with points where
the function exhibits an essential singularity. Consider the function ( ) 2 1 x f x e−
= . The function and all of its derivatives vanish at x = 0 suggesting a power
series expansion about x = 0 of the (naïve) form f x( ) = 0 . But this is only true at
the isolated point x = 0 and has no finite interval of validity, because the function
has an essential singularity at the origin. A more useful series arises from
expanding about the point at infinity In this form it is clear that special care must
be taken at the point x = 0. If we know that a power series converges, a related
important issue, especially for physicists, is the question of how rapidly the series
is converging, i.e., what is the magnitude of the error if we truncate the series
after N terms? There are several useful relations that address this question,
which we present without derivation.
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Consider a Taylor series expansion about the point 0 x x = of the function f x( ) . We can
define a remainder as
where the point x lies somewhere in the interval between 0 x and x, i.e., the final sum in
Eq. (3.21) can be written in terms of the first term in the sum but with the derivative
evaluated at the point x rather than 0 x . While it may be difficult to determine the
precise value of the special point x , this expression provides an easy way to obtain an
approximation to the remainder.
This result is often derived as part of the introductory calculus course, but recent
experience suggests that we should review its derivation here. First note that functions
defined by powers series (as here) are smooth and continuous (and bounded, i.e., not
infinite) within the interval of convergence of the series in the sense that all derivatives
of the function are (well) defined by related power series expansions which converge in
the same interval (except possibly at the endpoints – see point I) on page 4). Next
consider performing some nested integrals of the N +1st derivative of f(x), which is
defined by its Taylor series. We have
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
We can evaluate this integral by using another calculus result – the mean value
theorem. Consider a general function g(x) that is continuous and smooth in an interval
that includes x0 to x. Thus in this interval the function g(y) will exhibit (unique) maximum
and minimum values such that g g y g min max   ( ) for all y in the interval from x0 to
x. Further g(y) will take on all values in this range ( g g y g min max   ( ) ) at least
once as y varies from x0 to x. Then it follows that the mean value for this interval,
defined by
There must at least one value of the variable y within the interval where the
function passes through its mean value. Now we can apply this result to Eq. (3.23) and
establish the existence of a value x such that
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
We see clearly that, as we approach the edge of the interval of convergence, the
convergence is slower, i.e., it takes more terms to obtain the same size remainder.
Finally let’s discuss how to use such power series expansions.
1. They are extremely useful in numerical work, e.g., when working on a computer.
Imagine that we wish to evaluate the difference between two very similar
functions, e.g., the difference between The functions are so similar that trying to
take the difference by simply evaluating the functions separately numerically
requires incredible numerical accuracy, while taking the difference first
analytically greatly simplifies the problem. Consider their
2. We can also turn our discussion around and use the power series expansions of
known functions in order to evaluate sums of interest. This application is
straightforward in principle, but requires some creativity. We used this approach
in Lecture 2 to evaluate the Harmonic series with alternating signs to find ln 2 .
Let’s do something more challenging here. Imagine that we want to sum the
series
Clearly we want to consider a function whose power series expansion has similar
coefficients, but at the same time we want a power series that we can sum after,
perhaps, simplifying the series using some simple operation like taking a derivative. In
this case define a new function by the power series
Now we simplify by taking a derivative, then perform the sum, and finally
integrate (by parts) to “undo” the derivative,
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Showing that this sum is 1 is not at all straightforward by other means. Similar
manipulations can often be used to sum a power series and obtain a closed expression
for the function defined by the power series. The point is to use manipulations allowed
within the interval of convergence. Consider the function defined by
The ratio test tells us that this series is convergent for x 1 . The explicit form
clearly suggests that integrating will yield a simpler expression, which we can sum, i.e.,
which we can recognize,
10
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Power series are also useful for performing integrals, at least numerically.
Consider the following example,
while the true answer to the same number of significant figures is 1.3780. Of course,
these days one is seldom far from a computer with Mathematica or Maple.
Power series expansions are also useful for evaluating indeterminate
mathematical forms. This application is essentially (a careful application of) L’Hopital’s
rule. Consider the example
Now expand the numerator (and denominator) in power series keeping the first
nonzero terms,
11
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Finally power series are useful in simplifying physics problems. The standard
example is that of a pendulum. With the pendulum’s orientation specified by a polar
angle  (measured from the “down” direction) Newton’s equation is
Using just the first term in the power series for sin we obtain the (linearized) Harmonic
Oscillator problem
12
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Video Links
Power Series Solutions
• https://www.youtube.com/watch?v=oY0ItxI9xTk
Introduction to Power Series
• https://www.youtube.com/watch?v=DlBQcj_zQk0
Introduction to Power Series
• https://www.youtube.com/watch?v=DlBQcj_zQk0
References
•
•
•
https://sites.math.northwestern.edu/~sweng/teachin
g/2018su/math224/notes/lecture12.pdf
https://www3.nd.edu/~apilking/Math10560/Lectures/
Lecture%2030.pdf
http://courses.washington.edu/ph227814/227/ellis/P
hys_227_08_Lec_3.pdf
13
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 7
Simultaneous Linear
and Non Linear
Objectives
After completing this chapter, you will be able to:
▪
▪
▪
▪
Understand Simultaneous Linear and Non Linear
Equation
Differentiate Linear and Non-Linear Equation
Know about some back history of Simultaneous Linear
and Non Linear Equation
Solve Simultaneous Linear and Non Linear Equation
Introduction
Sometimes, one is just not enough, and we need two variables to get the job done.
That is when simultaneous linear equations come into play. Imagine this situation, 2 bars
of Cadbury chocolate and 3 Mars bars cost Rs.270. At the same store, someone else
buys 4 bars of Cadbury chocolate and 2 Mars bars and pays Rs.300. What is the cost of
one bar of Cadbury chocolate and one Mars bar? These are the types of problems that
require you to solve for 2 variables and therefore the need arises for simultaneous
equations.
The training wheels are off from this point on and it’s time to dive into the deep end
of the middle school math pool.
The Big Idea: What are Simultaneous Linear Equations?
A system of two equations with the same variables is called simultaneous
linear equations. The important thing to remember is that there are 2 unknowns (or
variables) and 2 equations (which are linear) in this system. The aim here is to get the
pair of values of the variables which satisfy both the equations. For example, both the
following equations are linear equations in two variables:
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
As you can see, if we treat these equations with two variables as two
independent entities then we can find an infinite number of solutions for both xx and yy.
That is not helpful at all! Therefore, we write them as a system of simultaneous
equations. That means that the variables are the same and can have unique solutions
for both x and y, for example:
Now, we want to find the pair (x,y)(x,y) that satisfies both the equations at the same
time. Basically, what that means is that for all the infinite solutions to equation 1, there will
be a pair (x,y)(x,y) that also satisfies equation 2. That is what we are looking for. We can
visualize a system of simultaneous linear equations by drawing 2 linear graphs and
finding out their intersection point.
The red line represents all the solutions for equation 1, and the blue line,
solutions for equation 2. The intersection is the unique (x,y)(x,y) that we are looking for
which will satisfy both the equations.
There are three major methods to solve simultaneous linear equations:
•
•
•
Graphically
Substitution
Elimination
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Take a look at the example below to understand the Substitution method: you can
observe how the values of the variables are obtained by substituting the variable in one
equation by its solution of the other equation. The key is to represent one variable as an
expression of the other variable.
Take a look at the image below to understand the Elimination method: In this method we
can eliminate one variable by multiplying each of the equations so that when one equation
is subtracted from the other, the resulting equation contains only one variable as the
coefficient of the other variable reduces to zero.
Some useful tips to help you speed up the process of solving simultaneous linear
equations:
•
•
You can use the graphical, substitution or elimination methods to solve a system
of linear equations.
Simultaneous linear equations may have a unique solution if they intersect at one
point
•
Infinite solutions if the two lines are overlapping:
•
No solutions if the two lines are parallel:
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
•
•
One Unique solution is obtained if the 2 lines intersect at one particular point: this
will give an ordered pair (x,y)(x,y) of solution. If the 2 lines lie on top of each other,
it is simple to see that the 2 equations are just multiples of one another and they
will have an infinite number of ordered pairs (x,y)(x,y) as their solution. In some
cases, you may find that the 2 equations actually represent a set of parallel lines;
this will mean that the system has no solutions as the 2 lines will never intersect
one another.
It is always recommended that you should substitute the ordered pair of
values (x,y)(x,y) back into the simultaneous linear equations to check whether it is
indeed the correct solution.
Geometric Or Graphical Interpretation
The graph of a linear equation ax+by = c is a straight line.
Two distinct lines always intersect at exactly one point unless they are parallel (have
the same slope).
The coordinates of the intersection point of the lines is the solution to the simultaneous
linear equations describing the lines. So we would normally expect a pair of
simultaneous equations to have just one solution.
Let's look at an example graphically:
2x + 3y = 7
4x + y = 9
From the graph we see that the point of intersection of the two lines is (2, 1)
Hence, the solution of the simultaneous equations is x = 2, y =1.
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
If you solved the two equations using either Gaussian elimination or substitution, you
would obtain the same result.
Recall that the slope of the line ax+by = c is
Two lines are parallel if they have the same slope.
(b 0).
Thus if a1x + b1y = c1 and a2x + b2y = c2 are parallel lines then
And
.
So, if the above equation is true, the lines are parallel, they do not intersect, and the
system of linear equations has no solution.
Consider the following system of linear equations:
x - y = -2
x - y = 1.
Using the method of substitution, we subtract the second equation from the first to
obtain: 0 = -3. This is a false statement and the system, therefore, has no solution. If
we look closer at the
lines we see that they satisfy the condition
and are therefore parallel (as can be seen below). They do not intersect explaining why
the system of linear equations has no solution.
What if the two equations represent the same line?
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Consider the equations
x-y=1
2x - 2y = 2
Multiply the first equation by 2 to put the equations in the form
2x - 2y = 2
2x - 2y = 2
Now subtraction gives 0 = 0, which is true no matter what values x and y may have!
This time the two equations represent the same line, since both can be written in the
form y = x - 1.
Any point on this line has coordinates which will satisfy both equations, so there are an
infinite number of solutions!
In general, two equations represent the same line if one equation is a multiple of the
other. That is
There are then three possibilities for a pair of simultaneous linear equations:
(i) Just one solution (the usual situation - both lines are unique and not parallel to each
other)
(ii) No solution ( the lines are parallel,
)
(iii) Infinitely many solutions (the equations represent the same line,
)
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The Elimination Method
This method for solving a pair of simultaneous linear equations reduces one equation to
one that has only a single variable. Once this has been done, the solution is the same
as that for when one line was vertical or parallel. This method is known as
the Gaussian elimination method.
Example 2.
Solve the following pair of simultaneous linear equations:
Equation 1:
Equation 2:
2x + 3y = 8
3x + 2y = 7
Step 1: Multiply each equation by a suitable number so that the two equations have the
same leading coefficient. An easy choice is to multiply Equation 1 by 3, the coefficient
of x in Equation 2, and multiply Equation 2 by 2, the x coefficient in Equation 1:
3 * (Eqn 1) ---> 3 * (2x + 3y = 8)--->
6x + 9y = 24
2 * (Eqn 2) ---> 2 * (3x + 2y = 7)--->
6x + 4y = 14
Both equations now have the same leading coefficient = 6
Step 2: Subtract the second equation from the first.
-(6x + 9y = 24
-(6x + 4y = 14)
5y = 10
Step 3: Solve this new equation for y.
y = 10/5 = 2
Step 4: Substitute y = 2 into either Equation 1 or Equation 2 above and solve for x. We'll
use Equation 1.
2x + 3(2) = 8
2x + 6 = 8
Subtract 6 from both sides
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
2x = 2
Divide both sides by 2
x=1
Solution: x = 1, y = 2 or (1,2).
Simultaneous Non-Linear Equations
A system of nonlinear equations is two or more equations, at least one of which
is not a linear equation, that are being solved simultaneously.
Note that in a nonlinear system, one of your equations can be linear, just not all of them.
In this tutorial, we will be looking at systems that have only two equations and two
unknowns.
A system of nonlinear equations is a system of two or more equations in two or more
variables containing at least one equation that is not linear. Recall that a linear equation
can take the form
Ax + By + C = 0
Any equation that cannot be written in this form in nonlinear. The substitution method we
used for linear systems is the same method we will use for nonlinear systems. We solve
one equation for one variable and then substitute the result into the second equation to
solve for another variable, and so on. There is, however, a variation in the possible
outcomes.
In general, a solution of a system in two variables is an ordered pair that
makes BOTH equations true.
In other words, it is where the two graphs intersect, what they have in
common. So if an ordered pair is a solution to one equation, but not the other,
then it is NOT a solution to the system.
Since we are looking at nonlinear systems, in some cases, there may be more
than one ordered pair that satisfies all equations in the system.
A consistent system is a system that has at least one solution.
An inconsistent system is a system that has no solution.
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The equations of a system are dependent if ALL the solutions of one equation
are also solutions of the other equation. In other words, they end up being the
same graph.
The equations of a system are independent if they do not share ALL
solutions. They can have one point in common, just not all of them.
There are three possible outcomes that you may encounter when working
with these systems:
1. a finite number of solutions
2. no solution
3. infinite solutions
Infinite Solutions
If the two graphs end up lying on top of each other, then there is an
infinite number of solutions. In this situation, they would end up being the
same graph, so any solution that would work in one equation is going to work
in the other.
If you get an infinite number of solutions for your final answer, is this system
consistent or inconsistent?
If you said consistent, you are right!
If you get an infinite number of solutions for your final answer, would the
equations be dependent or independent?
If you said dependent, you are correct!
The graph below illustrates a system of two equations and two unknowns
that has an infinite number of solutions:
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Solve by Substitution Method
Step 1: Simplify if needed.
This would involve things like removing ( ) and removing fractions.
To remove ( ): just use the distributive property.
To remove fractions: since fractions are another way to write division,
and the inverse of divide is to multiply, you remove fractions by
multiplying both sides by the LCD of all of your fractions.
Step 2: Solve one equation for either variable.
It doesn't matter which equation you use or which variable you choose
to solve for.
You want to make it as simple as possible. If one of the equations is
already solved for one of the variables, that is a quick and easy way to
go.
If you need to solve for a variable, then try to pick one that has a 1 or 1 as a coefficient. That way when you go to solve for it, you won't
have to divide by a number and run the risk of having to work with a
fraction (yuck!!).
Also, it is easier to solve for a variable that is to the 1 power, as
opposed to being squared, cubed, etc.
Step 3: Substitute what you get for step 2 into the other equation.
This is why it is called the substitution method. Make sure that you
substitute the expression into the OTHER equation, the one you didn't
use in step 2.
This will give you one equation with one unknown.
Step 4: Solve for the remaining variable.
Solve the equation set up in step 3 for the variable that is left.
Most of the equations in this step will end up being either linear or
quadratic. Once in awhile you will run into a different type of equation.
10
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Keep in mind that when you go to solve for this variable that you may
end up with no solution for your answer. For example, you may end
up with your variable equaling the square root of a negative number,
which is not a real number, which means there would be no solution.
If your variable drops out and you have a FALSE statement, that
means your answer is no solution.
If your variable drops out and you have a TRUE statement, that
means your answer is infinite solutions, which would be the
equation of the line.
Step 5: Solve for second variable.
If you come up with a finite number of values for the variable in
step 4, that means the two equations have a finite number of
solutions. Plug the value(s) found in step 4 into any of the equations
in the problem and solve for the other variable.
Step 6: Check the proposed ordered pair solution(s) in BOTH original
equations.
You can plug in the proposed solution(s) into BOTH equations. If it
makes BOTH equations true, then you have your solution to the
system.
If it makes at least one of them false, you need to go back and redo
the problem.
Intersection Of A Parabola And A Line
There are three possible types of solutions for a system of nonlinear equations involving
a parabola and a line
A GENERAL NOTE: POSSIBLE TYPES OF SOLUTIONS FOR POINTS OF
INTERSECTION OF A PARABOLA AND A LINE
The graphs below illustrate possible solution sets for a system of equations involving a
parabola and a line.
•
No solution. The line will never intersect the parabola.
•
One solution. The line is tangent to the parabola and intersects the parabola at
exactly one point.
•
Two solutions. The line crosses on the inside of the parabola and intersects the
parabola at two points.
11
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
EXAMPLE: SOLVING A SYSTEM OF NONLINEAR EQUATIONS REPRESENTING A
PARABOLA AND A LINE
Solve the system of equations.
Solve the first equation for x and then substitute the resulting expression into the
second equation.
Solving for y gives y = 2 and y = 1. Next, substitute each value for y into the first
equation to solve for x. Always substitute the value into the linear equation to check for
extraneous solutions.
12
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The solutions are (1,2) and (0,1), which can be verified by substituting these (x,
y)values into both of the original equations
13
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Video Links
How to - Solve Simultaneous Equation
•
https://www.youtube.com/watch?v=YoVY5JwAF9o
Simultaneous equations (linear and non-linear)
•
https://www.youtube.com/watch?v=ozP-vf99DK4
Simultaneous Equations
• https://www.youtube.com/watch?v=cWbZqWgsuY8
References
•
•
•
https://www.cuemath.com/algebra/simultaneous
-linear-equations/
https://courses.lumenlearning.com/waymakercol
legealgebra/chapter/methods-for-solving-asystem-of-nonlinear-equations/
http://mathsfirst.massey.ac.nz/Algebra/Systems
ofLinEq.htm
14
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 8
Numerical Differentiation
and Integration
Objectives
After completing this chapter, you will be able to:
▪
▪
▪
▪
Understand Differentiation and Integration
Differentiate Differentiation and Integration
Know about some back history of Numerical
Differentiation and Integration
Solve Differentiation and Integration problem
In numerical
analysis, numerical
differentiation describes algorithms for
estimating the derivative of a mathematical function or function subroutine using values
of the function and perhaps other knowledge about the function.
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Basic Concepts
This chapter deals with numerical approximations of derivatives. The first questions that
comes up to mind is: why do we need to approximate derivatives at all? After all, we do
know how to analytically differentiate every function. Nevertheless, there are several
reasons as of why we still need to approximate derivatives:
• Even if there exists an underlying function that we need to differentiate, we might
know its values only at a sampled data set without knowing the function itself.
• There are some cases where it may not be obvious that an underlying function
exists and all that we have is a discrete data set. We may still be interested in
studying changes in the data, which are related, of course, to derivatives.
• There are times in which exact formulas are available but they are very
complicated to the point that an exact computation of the derivative requires a lot
of function evaluations. It might be significantly simpler to approximate the
derivative instead of computing its exact value.
• When approximating solutions to ordinary (or partial) differential equations, we
typically represent the solution as a discrete approximation that is defined on a
grid. Since we then have to evaluate derivatives at the grid points, we need to be
able to come up with methods for approximating the derivatives at these points,
and again, this will typically be done using only values that are defined on a lattice.
The underlying function itself (which in this cased is the solution of the equation)
is unknown.
A simple approximation of the first derivative is
where we assume that h > 0. What do we mean when we say that the expression
on the right-hand-side of (5.1) is an approximation of the derivative? For linear functions
(5.1) is actually an exact expression for the derivative. For almost all other functions, (5.1)
is not the exact derivative.
Let’s compute the approximation error. We write a Taylor expansion of f(x + h)
about x, i.e.,
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
is called a backward differencing (which is obviously also a one-sided differencing
formula).
The second term on the right-hand-side of (5.3) is the error term. Since the
approximation (5.1) can be thought of as being obtained by truncating this term from the
exact formula (5.3), this error is called the truncation error. The small parameter h denotes
the distance between the two points x and x+h. As this distance tends to zero, i.e., h →
0, the two points approach each other and we expect the approximation (5.1) to improve.
This is indeed the case if the truncation error goes to zero, which in turn is the case if f
00(ξ) is well defined in the interval (x, x+h). The “speed” in which the error goes to zero
as h → 0 is called the rate of convergence. When the truncation error is of the order of
O(h), we say that the method is a first order method. We refer to a methods as a p thorder method if the truncation error is of the order of O(h p ).
It is possible to write more accurate formulas than (5.3) for the first derivative. For
example, a more accurate approximation for the first derivative that is based on the values
of the function at the points f(x−h) and f(x+h) is the centered differencing formula
Let’s verify that this is indeed a more accurate formula than (5.1). Taylor
expansions of the terms on the right-hand-side of (5.4) are
which means that the truncation error in the approximation (5.4) is
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
If the third-order derivative f 000(x) is a continuous function in the interval [x − h, x
+ h], then the intermediate value theorem implies that there exists a point ξ ∈ (x − h, x +
h) such that
which means that the expression (5.4) is a second-order approximation of the first
derivative. In a similar way we can approximate the values of higher-order derivatives.
For example, it is easy to verify that the following is a second-order approximation of the
second derivative
To verify the consistency and the order of approximation of (5.6) we expand
where we assume that ξ ∈ (x − h, x + h) and that f(x) has four continuous derivatives in
the interval. Hence, the approximation (5.6) is indeed a second-order approximation of
the derivative, with a truncation error that is given b
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Numerical Integration
In analysis, numerical integration comprises a broad family of algorithms for
calculating the numerical value of a definite integral, and by extension, the term is also
sometimes used to describe the numerical solution of differential equations. This article
focuses on calculation of definite integrals. The term numerical quadrature (often
abbreviated to quadrature) is more or less a synonym for numerical integration, especially
as applied to one-dimensional integrals. Some authors refer to numerical integration over
more than one dimension as cubature; others take quadrature to include higherdimensional integration.
The basic problem in numerical integration is to compute an approximate solution
to a definite integral
to a given degree of accuracy. If f(x) is a smooth function integrated over a small number
of dimensions, and the domain of integration is bounded, there are many methods for
approximating the integral to the desired precision.
History
The term "numerical integration" first appears in 1915 in the publication A Course
in Interpolation and Numeric Integration for the Mathematical Laboratory by David Gibb.
Quadrature is a historical mathematical term that means calculating area. Quadrature
problems have served as one of the main sources of mathematical
analysis. Mathematicians of Ancient Greece, according to the Pythagorean doctrine,
understood calculation of area as the process of constructing geometrically
a square having the same area (squaring). That is why the process was
named quadrature. For example, a quadrature of the circle, Lune of Hippocrates, The
Quadrature of the Parabola. This construction must be performed only by means
of compass and straightedge.
The ancient Babylonians used the trapezoidal rule to integrate the motion of Jupiter along
the ecliptic
For a quadrature of a rectangle with the sides a and b it is necessary to construct
a square with the side (the Geometric mean of a and b). For this purpose it is possible to
use the following fact: if we draw the circle with the sum of a and b as the diameter, then
the height BH (from a point of their connection to crossing with a circle) equals their
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
geometric mean. The similar geometrical construction solves a problem of a quadrature
for a parallelogram and a triangle.
The Area Segment of Parabola
Problems of quadrature for curvilinear figures are much more difficult. The quadrature of
the circle with compass and straightedge had been proved in the 19th century to be
impossible. Nevertheless, for some figures (for example the Lune of Hippocrates) a
quadrature can be performed. The quadratures of a sphere surface and a parabola
segment done by Archimedes became the highest achievement of the antique analysis.
•
•
The area of the surface of a sphere is equal to quadruple the area of a great circle of
this sphere.
The area of a segment of the parabola cut from it by a straight line is 4/3 the area of
the triangle inscribed in this segment.
For the proof of the results Archimedes used the Method of exhaustion of Eudoxus.
In medieval Europe the quadrature meant calculation of area by any method. More
often the Method of indivisibles was used; it was less rigorous, but more simple and
powerful. With its help Galileo Galilei and Gilles de Roberval found the area of
a cycloid arch, Grégoire de Saint-Vincent investigated the area under a hyperbola (Opus
Geometricum, 1647), and Alphonse Antonio de Sarasa, de Saint-Vincent's pupil and
commentator, noted the relation of this area to logarithms.
John Wallis algebrised this method: he wrote in his Arithmetica Infinitorum (1656) series
that we now call the definite integral, and he calculated their values. Isaac
Barrow and James Gregory made further progress: quadratures for some algebraic
curves and spirals. Christiaan Huygens successfully performed a quadrature of
some Solids of revolution.
The quadrature of the hyperbola by Saint-Vincent and de Sarasa provided a
new function, the natural logarithm, of critical importance.
With the invention of integral calculus came a universal method for area
calculation. In response, the term quadrature has become traditional, and instead the
modern phrase "computation of a univariate definite integral" is more common.
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Reasons for numerical integration
Here are several reasons for carrying out numerical integration.
1. The integrand f(x) may be known only at certain points, such as obtained
by sampling. Some embedded systems and other computer applications may
need numerical integration for this reason.
2. A formula for the integrand may be known, but it may be difficult or impossible to
find an antiderivative that is an elementary function. An example of such an
integrand is f(x) = exp(−x2), the antiderivative of which (the error function, times a
constant) cannot be written in elementary form.
3. It may be possible to find an antiderivative symbolically, but it may be easier to
compute a numerical approximation than to compute the antiderivative. That may
be the case if the antiderivative is given as an infinite series or product, or if its
evaluation requires a special function that is not available.
Methods for one-dimensional integrals
Numerical integration methods can generally be described as combining
evaluations of the integrand to get an approximation to the integral. The integrand is
evaluated at a finite set of points called integration points and a weighted sum of these
values is used to approximate the integral. The integration points and weights depend on
the specific method used and the accuracy required from the approximation.
An important part of the analysis of any numerical integration method is to study the
behavior of the approximation error as a function of the number of integrand evaluations.
A method that yields a small error for a small number of evaluations is usually considered
superior. Reducing the number of evaluations of the integrand reduces the number of
arithmetic operations involved, and therefore reduces the total round-off error. Also, each
evaluation takes time, and the integrand may be arbitrarily complicated.
A 'brute force' kind of numerical integration can be done, if the integrand is reasonably
well-behaved (i.e. piecewise continuous and of bounded variation), by evaluating the
integrand with very small increments.
We have now seen some of the most generally useful methods for discovering
antiderivatives, and there are others. Unfortunately, some functions have no simple
antiderivatives; in such cases if the value of a definite integral is needed it will have to be
approximated. We will see two methods that work reasonably well and yet are fairly
simple; in some cases more sophisticated techniques will be needed.
Of course, we already know one way to approximate an integral: if we think of the
integral as computing an area, we can add up the areas of some rectangles. While this is
quite simple, it is usually the case that a large number of rectangles is needed to get
acceptable accuracy. A similar approach is much better: we approximate the area under
a curve over a small interval as the area of a trapezoid. In figure 8.6.1 we see an area
under a curve approximated by rectangles and by trapezoids; it is apparent that the
trapezoids give a substantially better approximation on each subinterval.
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Integration via Interpolation
One direct way of obtaining quadratures from given samples of a function is by
integrating an interpolant. As always, our goal is to evaluate I = R b a f(x)dx. We assume
that the values of the function f(x) are given at n + 1 points: x0, . . . , xn ∈ [a, b]. Note that
we do not require the first point x0 to be equal to a, and the same holds for the right side
of the interval. Given the values f(x0), . . . f(xn), we can write the interpolating polynomial
of degree 6 n, which in the Largenge form is
The integral of f(x) can then be approximated by the integral of Pn(x), i.e.,
Note that if we want to integrate several different functions, and use their values at
the same points (x0, . . . , xn), the quadrature coefficients (6.8) should be computed only
once, since they do not depend on the function that is being integrated. If we change the
interpolation/integration points, then we must recompute the quadrature coefficients. For
equally spaced points, x0, . . . , xn, a numerical integration formula of the form.
is called a Newton-Cotes formula.
Composite Integration Rules
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
In a composite quadrature, we divide the interval into subintervals and apply an
integration rule to each subinterval. We demonstrate this idea with a couple of
Example
Consider the points
The composite trapezoidal rule is obtained by applying the trapezoidal rule in each
subinterval [xi−1, xi ], i = 1, . . . , n, i.e.,
A composite trapezoidal rule
A particular case is when these points are uniformly spaced, i.e., when all intervals
have an equal length. For example, if
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The notation of a sum with two primes,
, means that we sum over all the terms
with the exception of the first and last terms that are being divided by 2. We can also
compute the error term as a function of the distance between neighboring points, h. We
know from (6.11) that in every subinterval the quadrature error is
Hence, the overall error is obtained by summing over n such terms:
Clearly
If we assume that f 00(x) is continuous in [a, b] (which we anyhow do in order for
the interpolation error formula to be valid) then there exists a point ξ ∈ [a, b] such that
This means that the composite trapezoidal rule is second-order accurate.
10
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Video Links
Numerical Differentiation Methods
•
https://www.youtube.com/watch?v=tcqsLqIyjmk
Numerical Integration With Trapezoidal and Simpson's
Rule
•
https://www.youtube.com/watch?v=RTX-ik_8i-k
Simpson’s Rule & Numerical Integration
•
https://www.youtube.com/watch?v=7EqRRuh-5Lk
References
•
•
•
•
•
https://en.wikipedia.org/wiki/Numerical_differentiatio
n
http://www2.math.umd.edu/~dlevy/classes/amsc466
/lecture-notes/differentiation-chap.pdf
https://www.whitman.edu/mathematics/calculus_onli
ne/section14.01.html
https://mathworld.wolfram.com/NumericalDifferentiat
ion.html
https://en.wikipedia.org/wiki/Numerical_integration#:
~:text=In%20analysis%2C%20numerical%20integra
tion%20comprises,on%20calculation%20of%20defi
nite%20integrals.
11
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 9
Ordinary and Partial
Differential Equations
Objectives
After completing this chapter, you will be able to:
▪
▪
▪
Solve Ordinary and Partial Differential Equations
Understand the characteristic of a Ordinary and
Partial Differential Equations
Identify the Classification of Ordinary and Partial
Differential Equations
Introduction
In mathematics,
an Ordinary
Differential
Equation (ODE)
is
a differential
equation containing one or more functions of one independent variable and
the derivatives of those functions. The term ordinary is used in contrast with the
term partial differential equation which may be with respect to more than one
independent variable.
A linear differential equation is a differential equation that is defined by a linear
polynomial in the unknown function and its derivatives, that is an equation of the form
among ordinary differential equations, linear differential equations play a prominent role
for several reasons. Most elementary and special functions that are encountered
in physics and applied mathematics are solutions of linear differential equations
(see Holonomic function). When physical phenomena are modeled with non-linear
equations, they are generally approximated by linear differential equations for an easier
solution. The few non-linear ODEs that can be solved explicitly are generally solved by
transforming the equation into an equivalent linear ODE (see, for example Riccati
equation).
Some ODEs can be solved explicitly in terms of known functions and integrals. When
that is not possible, the equation for computing the Taylor series of the solutions may be
useful. For applied problems, numerical methods for ordinary differential equations can
supply an approximation of the solution.
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Ordinary differential equations (ODEs) arise in many contexts of mathematics
and social and natural sciences. Mathematical descriptions of change use differentials
and derivatives. Various differentials, derivatives, and functions become related via
equations, such that a differential equation is a result that describes dynamically changing
phenomena, evolution, and variation. Often, quantities are defined as the rate of change
of other quantities (for example, derivatives of displacement with respect to time), or
gradients of quantities, which is how they enter differential equations.
Specific mathematical fields include geometry and analytical mechanics. Scientific fields
include much of physics and astronomy (celestial mechanics), meteorology (weather
modeling), chemistry (reaction
rates),biology (infectious
diseases,
genetic
variation), ecology and population modeling (population competition), economics (stock
trends, interest rates and the market equilibrium price changes).
Many mathematicians have studied differential equations and contributed to the field,
including Newton, Leibniz, the Bernoulli family, Riccati, Clairaut, d'Alembert, and Euler.
A simple example is Newton's second law of motion — the relationship between the
displacement x and the time t of an object under the force F, is given by the differential
equation
which constrains the motion of a particle of constant mass m. In general, F is a
function of the position x(t) of the particle at time t. The unknown function x(t) appears
on both sides of the differential equation, and is indicated in the notation F(x(t)).
In what follows, let y be a dependent variable and x an independent variable,
and y = f(x) is an unknown function of x. The notation for differentiation varies
depending upon the author and upon which notation is most useful for the task at hand.
In this context, the Leibniz's notation (dy/dx,d2y/dx2,...,dny/dxn) is more useful for
differentiation and integration, whereas Lagrange's notation (y′,y′′, ..., y(n)) is more useful
for representing derivatives of any order compactly, and Newton's notation is often used
in physics for representing derivatives of low order with respect to time.
Given F, a function of x, y, and derivatives of y. Then an equation of the form
called an Explicit Ordinary Differential Equation of order n.
More generally, an Implicit Ordinary Differential Equation of order n takes the form
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
There are further classifications:
Autonomous
A differential equation not depending on x is called autonomous.
Linear
A differential equation is said to be linear if F can be written as a linear
combination of the derivatives of y:
where a i (x) and r (x) are continuous functions of x.The function r(x) is called the source
term, leading to two further important classifications:
Homogeneous
If r(x) = 0, and consequently one "automatic" solution is the trivial solution, y = 0.
The solution of a linear homogeneous equation is a complementary function, denoted
here by yc.
Nonhomogeneous (Or Inhomogeneous)
If r(x) ≠ 0. The additional solution to the complementary function is the particular
integral, denoted here by yp.
The general solution to a linear equation can be written as y = yc + yp.
Non-Linear
A differential equation that cannot be written in the form of a linear combination.
System Of ODEs
A number of coupled differential equations form a system of equations. If y is a
vector whose elements are functions; y(x) = [y1(x), y2(x),..., ym(x)], and F is a vectorvalued function of y and its derivatives, then
is an explicit system of ordinary differential equations of order n and dimension m.
In column vector form:
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
These are
The implicit analogue is:
not necessarily linear.
where 0 = (0, 0, ..., 0) is the zero vector. In matrix form
For a system of the form , some sources also require that the Jacobian
matrix be non-singular in order to call this an implicit ODE [system]; an implicit ODE
system satisfying this Jacobian non-singularity condition can be transformed into an
explicit ODE system. In the same sources, implicit ODE systems with a singular Jacobian
are termed differential algebraic equations (DAEs). This distinction is not merely one of
terminology; DAEs have fundamentally different characteristics and are generally more
involved to solve than (nonsingular) ODE systems. Presumably for additional derivatives,
the Hessian matrix and so forth are also assumed non-singular according to this scheme,
although note that order, which makes the Jacobian singularity criterion sufficient for this
taxonomy to be comprehensive at all orders.
The behavior of a system of ODEs can be visualized through the use of a phase
portrait.
Singular Solutions
The theory of singular solutions of ordinary and partial differential equations was a
subject of research from the time of Leibniz, but only since the middle of the nineteenth
century has it received special attention. A valuable but little-known work on the subject
is that of Houtain (1854). Darboux (from 1873) was a leader in the theory, and in the
geometric interpretation of these solutions he opened a field worked by various writers,
notably Casorati and Cayley. To the latter is due (1872) the theory of singular solutions
of differential equations of the first order as accepted circa 1900.
Reduction To Quadratures
The primitive attempt in dealing with differential equations had in view a reduction
to quadratures. As it had been the hope of eighteenth-century algebraists to find a method
for solving the general equation of the nth degree, so it was the hope of analysts to find
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
a general method for integrating any differential equation. Gauss (1799) showed,
however, that complex differential equations require complex numbers. Hence, analysts
began to substitute the study of functions, thus opening a new and fertile
field. Cauchy was the first to appreciate the importance of this view. Thereafter, the real
question was no longer whether a solution is possible by means of known functions or
their integrals, but whether a given differential equation suffices for the definition of a
function of the independent variable or variables, and, if so, what are the characteristic
properties.
Fuchsian Theory
Two memoirs by Fuchs inspired a novel approach, subsequently elaborated by
Thomé and Frobenius. Collet was a prominent contributor beginning in 1869. His method
for integrating a non-linear system was communicated to Bertrand in
1868. Clebsch (1873) attacked the theory along lines parallel to those in his theory
of Abelian integrals. As the latter can be classified according to the properties of the
fundamental curve that remains unchanged under a rational transformation, Clebsch
proposed to classify the transcendent functions defined by differential equations
according to the invariant properties of the corresponding surfaces f = 0 under rational
one-to-one transformations.
Lie's Theory
From 1870, Sophus Lie's work put the theory of differential equations on a better
foundation. He showed that the integration theories of the older mathematicians can,
using Lie groups, be referred to a common source, and that ordinary differential equations
that admit the same infinitesimal transformations present comparable integration
difficulties. He also emphasized the subject of transformations of contact.
Lie's group theory of differential equations has been certified, namely: (1) that it unifies
the many ad hoc methods known for solving differential equations, and (2) that it provides
powerful new ways to find solutions. The theory has applications to both ordinary and
partial differential equations.
Sturm–Liouville Theory
Sturm–Liouville theory is a theory of a special type of second order linear ordinary
differential
equation.
Their
solutions
are
based
on eigenvalues and
corresponding eigenfunctions of linear operators defined via second-order homogeneous
linear equations. The problems are identified as Sturm-Liouville Problems (SLP) and are
named after J.C.F. Sturm and J. Liouville, who studied them in the mid-1800s. SLPs have
an infinite number of eigenvalues, and the corresponding eigenfunctions form a complete,
orthogonal set, which makes orthogonal expansions possible. This is a key idea in applied
mathematics, physics, and engineering. SLPs are also useful in the analysis of certain
partial differential equations.
Existence And Uniqueness Of Solutions
Local Existence And Uniqueness Theorem Simplified
The theorem can be stated simply as follows. For the equation and initial value
problem:
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
if F and ∂F/∂y are continuous in a closed rectangle
in the x-y plane, where a and b are real (symbolically: a, b ∈ ℝ) and × denotes
the cartesian product, square brackets denote closed intervals, then there is an interval
for some h ∈ ℝ where the solution to the above equation and initial value problem can
be found. That is, there is a solution and it is unique. Since there is no restriction on F to
be linear, this applies to non-linear equations that take the form F(x, y), and it can also
be applied to systems of equations.
Global Uniqueness And Maximum Domain Of Solution
When the hypotheses of the Picard–Lindale theorem are satisfied, then local
existence and uniqueness can be extended to a global result. More precisely:
For each initial condition (x0, y0) there exists a unique maximum (possibly infinite) open
interval
such that any solution that satisfies this initial condition is a restriction of the solution
that satisfies this initial condition with domain In the case that , there are exactly two
possibilities.
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Partial Differential Equation
Partial differential equation, in mathematics, equation relating a function of
several variables to its partial derivatives. A partial derivative of a function of several
variables expresses how fast the function changes when one of its variables is changed,
the others being held constant (compare ordinary differential equation). The partial
derivative of a function is again a function, and, if f(x, y) denotes the original function of
the variables x and y, the partial derivative with respect to x—i.e., when only x is allowed
to vary—is typically written as fx(x, y) or ∂f/∂x. The operation of finding a partial derivative
can be applied to a function that is itself a partial derivative of another function to get what
is called a second-order partial derivative. For example, taking the partial derivative
of fx(x, y) with respect to y produces a new function fxy(x, y), or ∂2f/∂y∂x. The order and
degree of partial differential equations are defined the same as for ordinary differential
equations.
A partial differential equation (or briefly a PDE) is a mathematical equation that
involves two or more independent variables, an unknown function (dependent on those
variables), and partial derivatives of the unknown function with respect to the independent
variables. The order of a partial differential equation is the order of the highest derivative
involved. A solution (or a particular solution) to a partial differential equation is a function
that solves the equation or, in other words, turns it into an identity when substituted into
the equation. A solution is called general if it contains all particular solutions of the
equation concerned.
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
The term exact solution is often used for second- and higher-order nonlinear PDEs to
denote a particular solution (see also Preliminary remarks at Second-Order Partial
Differential Equations).
Partial differential equations are used to mathematically formulate, and thus aid the
solution of, physical and other problems involving functions of several variables, such as
the propagation of heat or sound, fluid flow, elasticity, electrostatics, electrodynamics, etc.
General Form Of First-Order Partial Differential Equation
A first-order partial differential equation with n independent variables has the
general form
where w=w(x1,x2,…,xn) is the unknown function and F(…) is a given function.
Quasilinear Equations. Characteristic System. General Solution
A first-order quasilinear partial differential equation with two independent
variables has the general form
Such equations are encountered in various applications (continuum mechanics,
gas dynamics, hydrodynamics, heat and mass transfer, wave theory, acoustics,
multiphase flows, chemical engineering, etc.).
If the functions f, g, and h are independent of the unknown w, then equation is
called linear.
Characteristic System. General Solution
The system of ordinary differential equations
is known as the characteristic system of equation. Suppose that two independent
particular solutions of this system have been found in the form
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
where C1 and C2 are arbitrary constants; such particular solutions are known
as integrals of system. Then the general solution to equation can be written as
where Φ is an arbitrary function of two variables. With equation (4) solved for u2, one
often specifies the general solution in the form u2=Ψ(u1), where Ψ(u) is an arbitrary
function of one variable.
Remark. If h(x,y,w)≡0, then w=C2 can be used as the second integral in (3).
Example. Consider the linear equation
The associated characteristic system of ordinary differential equations
has two integrals
Therefore, the general solution to this PDE can be written as w−bx=Ψ(y−ax), or
where Ψ(z) is an arbitrary function.
Second-Order Partial Differential Equations
Linear second-order PDEs and their properties. Principle of linear superposition
A second-order linear partial differential equation with two independent variables has
the form
Some Linear Equations Encountered In Applications
Three basic types of linear partial differential equations are distinguished—
parabolic, hyperbolic, and elliptic (for details, see below). The solutions of the equations
pertaining to each of the types have their own characteristic qualitative differences.
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Heat equation (a parabolic equation)
The simplest example of a parabolic equation is the heat equation
Wave Equation (a hyperbolic equation)
The simplest example of a hyperbolic equation is the wave equation
Laplace Equation (an elliptic equation)
The simplest example of an elliptic equation is the Laplace equation
Classification Of Second-Order Partial Differential Equations
Types Of Equations
Any Semi linear partial differential equation of the second-order with two
independent variables can be reduced, by appropriate manipulations, to a simpler
equation that has one of the three highest derivative combinations specified above in
examples and given a point (x,y), equation is said to be.
Canonical form of parabolic equations
(case b2−ac=0)
Two canonical forms of hyperbolic equations
(case b2−ac>0)
Canonical form of elliptic equations
(case b2−ac<0)
10
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
References
•
https://en.wikipedia.org/wiki/Ordinary_differentia
l_equation#CITEREFSimmons1972
•
http://www.scholarpedia.org/article/Partial_differ
ential_equation
•
https://www.britannica.com/science/partialdifferential-eq
Video Links
PDEs: Ordinary versus Partial Differential Equations
•
https://www.youtube.com/watch?v=y3dHKNl-q-k
ODEs: Introduction to Ordinary Differential Equations
•
https://www.youtube.com/watch?v=bP2Ug7Mm5RE
PDEs: Classification of Partial Differential Equations
•
https://www.youtube.com/watch?v=oICjqJwRbqY&li
st=PLYdroRCLMg5MLOnfJMyu4nLxtKOD7gGpz&in
dex=2
11
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Chapter 10
Optimization
Objectives
After completing this chapter, you will be able to:
▪ Understand about Optimization
▪
Identify and understand the Characterize the major
subfields of Optimization
▪
Classification of critical points and extrema
▪
Apply the Computational optimization techniques
and Application of Optimization
Introduction
Optimization is a general term used to describe types of problems and solution
techniques that are concerned with the best (“optimal”) allocation of limited resources in
projects. The problems are called optimization problems and the methods optimization
methods. Typical problems are concerned with planning and making decisions, such as
selecting an optimal production plan. A company has to decide how many units of each
product from a choice of (distinct) products it should make. The objective of the company
may be to maximize overall profit when the different products have different individual
profits.
Mathematical optimization (alternatively spelled optimisation) or mathematical
programming is the selection of a best element (with regard to some criterion) from some
set of available alternatives Optimization problems of sorts arise in all quantitative
disciplines
from computer
science and engineering to operations
research and economics, and the development of solution methods has been of interest
in mathematics for centuries.
In the simplest case, an optimization problem consists of maximizing or minimizing a real
function by systematically choosing input values from within an allowed set and
computing the value of the function. The generalization of optimization theory and
techniques to other formulations constitutes a large area of applied mathematics. More
generally, optimization includes finding "best available" values of some objective function
given a defined domain (or input), including a variety of different types of objective
functions and different types of domains.
1
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Optimization, also known as mathematical programming, collection of mathematical
principles and methods used for solving quantitative problems in many disciplines,
including physics, biology, engineering, economics, and business. The subject grew from
a realization that quantitative problems in manifestly different disciplines have important
mathematical elements in common. Because of this commonality, many problems can be
formulated and solved by using the unified set of ideas and methods that make up the
field of optimization.
The historic term mathematical programming, broadly synonymous with optimization,
was coined in the 1940s before programming became equated with computer
programming. Mathematical programming includes the study of the mathematical
structure of optimization problems, the invention of methods for solving these problems,
the study of the mathematical properties of these methods, and the implementation of
these methods on computers. Faster computers have greatly expanded the size and
complexity of optimization problems that can be solved. The development of optimization
techniques has paralleled advances not only in computer science but also in operations
research, numerical analysis, game theory, mathematical economics, control theory,
and combinatorics.
An important class of optimization is known as linear programming. Linear indicates that
no variables are raised to higher powers, such as squares. For this class, the problems
involve minimizing (or maximizing) a linear objective function whose variables are real
numbers that are constrained to satisfy a system of linear equalities and inequalities.
Another important class of optimization is known as nonlinear programming. In nonlinear
programming the variables are real numbers, and the objective or some of the constraints
are nonlinear functions (possibly involving squares, square roots, trigonometric functions,
or products of the variables). Both linear and nonlinear programmings are discussed in
this article. Other important classes of optimization problems not covered in this article
include stochastic programming, in which the objective function or the constraints depend
on random variables, so that the optimum is found in some “expected,” or probabilistic,
sense; network optimization, which involves optimization of some property of a flow
through a network, such as the maximization of the amount of material that can be
transported between two given locations in the network; and combinatorial optimization,
in which the solution must be found among a finite but very large set of possible values,
such as the many possible ways to assign 20 manufacturing plants to 20 locations.
Linear Programming
Interest in linear programming has also extended to economics. In 1937 the Hungarianborn mathematician John von Neumann analyzed a steadily expanding economy based
on alternative methods of production and fixed technological coefficients. As far as
mathematical history is concerned, the study of linear inequality systems excited virtually
no interest before 1936. In 1911 a vertex-to-vertex movement along edges of
a polyhedron (as is done in the simplex method) was suggested as a way to solve a
problem that involved optimization, and in 1941 movement along edges was proposed
for a problem involving transportation. Credit for laying much of the mathematical
foundations should probably go to von Neumann. In 1928 he published his famous paper
2
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
on game theory, and his work culminated in 1944 with the publication, in collaboration
with the Austrian economist Oskar Morgenstern, of the classic Theory of Games and
Economic Behaviour. In 1947 von Neumann conjectured the equivalence of linear
programs and matrix games, introduced the important concept of duality, and made
several proposals for the numerical solution of linear programming and game problems.
Serious interest by other mathematicians began in 1948 with the rigorous development
of duality and related matters.
Basic ideas
A simple problem in linear programming is one in which it is necessary to find the
maximum (or minimum) value of a simple function subject to certain constraints. An
example might be that of a factory producing two commodities. In any production run, the
factory produces x1 of the first type and x2 of the second. If the profit on the second type
is twice that on the first, then x1 + 2x2 represents the total profit. The function x1 + 2x2 is
known as the objective function.
The simplex method
To illustrate the simplex method, the example from the preceding section will be
solved again. The problem is first put into canonical form by converting the linear
inequalities into equalities by introducing “slack variables” x3 ≥ 0 (so that x1 + x3 = 8), x4 ≥
0 (so that x2 + x4 = 5), x5 ≥ 0 (so that x1 + x2 + x5 = 10), and the variable x0 for the value
of the objective function (so that x1 + 2x2 − x0 = 0). The problem may then be restated as
that of finding nonnegative quantities x1, …, x5 and the largest possible x0 satisfying the
resulting equations. One obvious solution is to set the objective variables x1 = x2 = 0,
which corresponds to the extreme point at the origin. If one of the objective variables is
increased from zero while the other one is fixed at zero, the objective value x0 will
increase as desired (subject to the slack variables satisfying the equality constraints). The
variable x2 produces the largest increase of x0 per unit change; so it is used first. Its
increase is limited by the nonnegativity requirement on the variables. In particular, if x2 is
increased beyond 5, x4 becomes negative.
Standard formulation
In practice, optimization problems are formulated in terms of matrices—a compact
symbolism for manipulating the constraints and testing the objective function
algebraically. The original (or “primal”) optimization problem was given its standard
formulation by von Neumann in 1947. In the primal problem the objective is replaced by
the product (px) of a vector x = (x1, x2, x3, …, xn)T, whose components are the objective
variables and where the superscript “transpose” symbol indicates that the vector should
be written vertically, and another vector p = (p1, p2, p3, …, pn), whose components are the
coefficients of each of the objective variables. In addition, the system
of inequality constraints is replaced by Ax ≤ b, where the m by n matrix A replaces
the m constraints on the n objective variables, and b = (b1, b2, b3, …, bm)T is a vector
whose components are the inequality bounds.
Nonlinear Programming
3
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Although the linear programming model works fine for many situations, some problems
cannot be modeled accurately without including nonlinear components. One example
would be the isoperimetric problem: determine the shape of the closed plane curve having
a given length and enclosing the maximum area. The solution, but not a proof, was known
by Pappus of Alexandria c. 340 CE:
An important early algorithm for solving nonlinear programs was given by the Nobel
Prize-winning Norwegian economist Ragnar Frisch in the mid-1950s. Curiously, his
approach fell out of favour for some decades, reemerging as a viable and competitive
approach only in the 1990s. Other important algorithmic approaches include sequential
quadratic programming, in which an approximate problem with a quadratic objective and
linear constraints is solved to obtain each search step; and penalty methods, including
the “method of multipliers,” in which points that do not satisfy the constraints incur penalty
terms in the objective to discourage algorithms from visiting them.
•
•
•
•
•
•
•
•
•
•
Convex programming studies the case when the objective function
is convex (minimization) or concave (maximization) and the constraint set
is convex. This can be viewed as a particular case of nonlinear programming or
as generalization of linear or convex quadratic programming.
Linear programming (LP), a type of convex programming, studies the case in
which the objective function f is linear and the constraints are specified using only
linear equalities and inequalities. Such a constraint set is called a polyhedron or
a polytope if it is bounded.
Second order cone programming (SOCP) is a convex program, and includes
certain types of quadratic programs.
Semidefinite programming (SDP) is a subfield of convex optimization where
the underlying variables are semidefinite matrices. It is a generalization of linear
and convex quadratic programming.
Conic programming is a general form of convex programming. LP, SOCP and
SDP can all be viewed as conic programs with the appropriate type of cone.
Geometric programming is a technique whereby objective and inequality
constraints expressed as posynomials and equality constraints
as monomials can be transformed into a convex program.
Integer programming studies linear programs in which some or all variables are
constrained to take on integer values. This is not convex, and in general much
more difficult than regular linear programming.
Quadratic programming allows the objective function to have quadratic terms,
while the feasible set must be specified with linear equalities and inequalities. For
specific forms of the quadratic term, this is a type of convex programming.
Fractional programming studies optimization of ratios of two nonlinear
functions. The special class of concave fractional programs can be transformed
to a convex optimization problem.
Nonlinear programming studies the general case in which the objective
function or the constraints or both contain nonlinear parts. This may or may not
be a convex program. In general, whether the program is convex affects the
difficulty of solving it.
4
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
•
•
•
•
•
•
•
•
•
•
•
•
Stochastic programming studies the case in which some of the constraints or
parameters depend on random variables.
Robust optimization is, like stochastic programming, an attempt to capture
uncertainty in the data underlying the optimization problem. Robust optimization
aims to find solutions that are valid under all possible realizations of the
uncertainties defined by an uncertainty set.
Combinatorial optimization is concerned with problems where the set of
feasible solutions is discrete or can be reduced to a discrete one.
Stochastic optimization is used with random (noisy) function measurements or
random inputs in the search process.
Infinite-dimensional optimization studies the case when the set of feasible
solutions is a subset of an infinite-dimensional space, such as a space of
functions.
Heuristics and metaheuristics make few or no assumptions about the problem
being optimized. Usually, heuristics do not guarantee that any optimal solution
need be found. On the other hand, heuristics are used to find approximate
solutions for many complicated optimization problems.
Constraint satisfaction studies the case in which the objective function f is
constant (this is used in artificial intelligence, particularly in automated
reasoning).
Constraint programming is a programming paradigm wherein relations
between variables are stated in the form of constraints.
Disjunctive programming is used where at least one constraint must be satisfied
but not all. It is of particular use in scheduling.
Space mapping is a concept for modeling and optimization of an engineering
system to high-fidelity (fine) model accuracy exploiting a suitable physically
meaningful coarse or surrogate model.
In a number of subfields, the techniques are designed primarily for optimization in
dynamic contexts (that is, decision making over time):
Calculus of variations seeks to optimize an action integral over some space to
an extremum by varying a function of the coordinates.
Optimal control theory is a generalization of the calculus of variations which
introduces control policies.
Dynamic programming is the approach to solve the stochastic
optimization problem with stochastic, randomness, and unknown model
parameters. It studies the case in which the optimization strategy is based on
splitting the problem into smaller subproblems. The equation that describes the
relationship between these subproblems is called the Bellman equation.
Mathematical programming with equilibrium constraints is where the constraints
include variational inequalities or complementarities.
Feasibility problem
The satisfiability problem, also called the feasibility problem, is just the problem of
finding any feasible solution at all without regard to objective value. This can be regarded
5
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
as the special case of mathematical optimization where the objective value is the same
for every solution, and thus any solution is optimal.
Many optimization algorithms need to start from a feasible point. One way to obtain such
a point is to relax the feasibility conditions using a slack variable; with enough slack, any
starting point is feasible. Then, minimize that slack variable until the slack is null or
negative.
Existence
The extreme value theorem of Karl Weierstrass states that a continuous realvalued function on a compact set attains its maximum and minimum value. More
generally, a lower semi-continuous function on a compact set attains its minimum; an
upper semi-continuous function on a compact set attains its maximum point or view.
Necessary conditions for optimality
One of Fermat's theorems states that optima of unconstrained problems are found
at stationary points, where the first derivative or the gradient of the objective function is
zero (see first derivative test). More generally, they may be found at critical points, where
the first derivative or gradient of the objective function is zero or is undefined, or on the
boundary of the choice set. An equation (or set of equations) stating that the first
derivative(s) equal(s) zero at an interior optimum is called a 'first-order condition' or a set
of first-order conditions.
Optima of equality-constrained problems can be found by the Lagrange
multiplier method. The optima of problems with equality and/or inequality constraints can
be found using the 'Karush–Kuhn–Tucker conditions'.
Sufficient conditions for optimality
While the first derivative test identifies points that might be extrema, this test does
not distinguish a point that is a minimum from one that is a maximum or one that is neither.
When the objective function is twice differentiable, these cases can be distinguished by
checking the second derivative or the matrix of second derivatives (called the Hessian
matrix) in unconstrained problems, or the matrix of second derivatives of the objective
function and the constraints called the bordered Hessian in constrained problems. The
conditions that distinguish maxima, or minima, from other stationary points are called
'second-order conditions' (see 'Second derivative test'). If a candidate solution satisfies
the first-order conditions, then the satisfaction of the second-order conditions as well is
sufficient to establish at least local optimality.
Sensitivity And Continuity Of Optima
The envelope theorem describes how the value of an optimal solution changes
when an underlying parameter changes. The process of computing this change is
called comparative statics. The maximum theorem of Claude Berge (1963) describes the
continuity of an optimal solution as a function of underlying parameters.
Calculus Of Optimization
Finding the points where the gradient of the objective function is zero (that is, the
stationary points). More generally, a zero subgradient certifies that a local minimum has
been found for minimization problems with convex functions and other locally Lipschitz
6
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
functions. Further, critical points can be classified using the definiteness of the Hessian
matrix: If the Hessian is positive definite at a critical point, then the point is a local
minimum; if the Hessian matrix is negative definite, then the point is a local maximum;
finally, if indefinite, then the point is some kind of saddle point.
Constrained problems can often be transformed into unconstrained problems with the
help of Lagrange multipliers. Lagrangian relaxation can also provide approximate
solutions to difficult constrained problems.
When the objective function is a convex function, then any local minimum will also be a
global minimum. There exist efficient numerical techniques for minimizing convex
Functions, Such As Interior-Point Methods.
Literative Methods
The iterative methods used to solve problems of nonlinear programming differ according
to whether they evaluate Hessians, gradients, or only function values. While evaluating
Hessians (H) and gradients (G) improves the rate of convergence, for functions for which
these quantities exist and vary sufficiently smoothly, such evaluations increase
the computational complexity (or computational cost) of each iteration. In some cases,
the computational complexity may be excessively high.
•
•
Methods that evaluate Hessians (or approximate Hessians, using finite differences):
o Newton's method
o Sequential quadratic programming: A Newton-based method for small-medium
scale constrained problems. Some versions can handle large-dimensional
problems.
o Interior point methods: This is a large class of methods for constrained
optimization. Some interior-point methods use only (sub)gradient information and
others of which require the evaluation of Hessians.
Methods that evaluate gradients, or approximate gradients in some way (or even
subgradients):
o Coordinate descent methods: Algorithms which update a single
coordinate in each iteration
o Conjugate gradient methods: Iterative methods for large problems. (In
theory, these methods terminate in a finite number of steps with quadratic
objective functions, but this finite termination is not observed in practice on
finite–precision computers.)
o Gradient descent (alternatively, "steepest descent" or "steepest ascent"):
A (slow) method of historical and theoretical interest, which has had
renewed interest for finding approximate solutions of enormous problems.
o Subgradient methods - An iterative method for large locally Lipschitz
functions using generalized gradients. Following Boris T. Polyak,
subgradient–projection methods are similar to conjugate–gradient
methods.
o Bundle method of descent: An iterative method for small–medium-sized
problems with locally Lipschitz functions, particularly for convex
minimization problems. (Similar to conjugate gradient methods)
7
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
•
•
o Ellipsoid method: An iterative method for small problems
with quasiconvex objective functions and of great theoretical interest,
particularly in establishing the polynomial time complexity of some
combinatorial optimization problems. It has similarities with Quasi-Newton
methods.
o Conditional gradient method (Frank–Wolfe) for approximate
minimization of specially structured problems with linear constraints,
especially with traffic networks. For general unconstrained problems, this
method reduces to the gradient method, which is regarded as obsolete
(for almost all problems).
o Quasi-Newton methods: Iterative methods for medium-large problems
(e.g. N<1000).
o Simultaneous perturbation stochastic approximation (SPSA) method
for stochastic optimization; uses random (efficient) gradient approximation.
Methods that evaluate only function values: If a problem is continuously
differentiable, then gradients can be approximated using finite differences, in
which case a gradient-based method can be used.
o Interpolation methods
o Pattern search methods, which have better convergence properties than
the Nelder–Mead heuristic (with simplices), which is listed below.
Heuristics
Besides (finitely terminating) algorithms and (convergent) iterative methods,
there are heuristics. A heuristic is any algorithm which is not guaranteed
(mathematically) to find the solution, but which is nevertheless useful in certain
practical situations. List of some well-known heuristics:
Mechanics
Problems in rigid body dynamics (in particular articulated rigid body dynamics) often
require mathematical programming techniques, since you can view rigid body dynamics
as attempting to solve an ordinary differential equation on a constraint manifold; the
constraints are various nonlinear geometric constraints such as "these two points must
always coincide", "this surface must not penetrate any other", or "this point must always
lie somewhere on this curve". Also, the problem of computing contact forces can be done
by solving a linear complementarity problem, which can also be viewed as a QP
(quadratic programming) problem.
Economics And Finance
Economics is closely enough linked to optimization of agents that an influential definition
relatedly describes economics qua science as the "study of human behavior as a
relationship between ends and scarce means" with alternative uses
Electrical Engineering
Some
common
applications
of
optimization
techniques
in electrical
engineering include active filter design, stray field reduction in superconducting magnetic
energy storage systems, space mapping design of microwave structures, handset
8
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
antennas, electromagnetics-based design. Electromagnetically validated design
optimization of microwave components and antennas has made extensive use of an
appropriate
physics-based
or
empirical surrogate
model and space
mapping methodologies since the discovery of space mapping in 1993.
Civil Engineering
Optimization
has
been
widely
used
in
civil
engineering. Construction
management and transportation engineering are among the main branches of civil
engineering that heavily rely on optimization. The most common civil engineering
problems that are solved by optimization are cut and fill of roads, life-cycle analysis of
structures
and
infrastructures,
resource
leveling, water
resource
allocation, traffic management and schedule optimization.
Operations Research
Another field that uses optimization techniques extensively is operations research.
Operations research also uses stochastic modeling and simulation to support improved
decision-making. Increasingly, operations research uses stochastic programming to
model dynamic decisions that adapt to events; such problems can be solved with largescale optimization and stochastic optimization methods.
Control Engineering
Mathematical optimization is used in much modern controller design. High-level
controllers such as model predictive control (MPC) or real-time optimization (RTO)
employ mathematical optimization. These algorithms run online and repeatedly determine
values for decision variables, such as choke openings in a process plant, by iteratively
solving a mathematical optimization problem including constraints and a model of the
system to be controlled.
Geophysics
Optimization techniques are regularly used in geophysical parameter estimation
problems. Given a set of geophysical measurements, e.g. seismic recordings, it is
common to solve for the physical properties and geometrical shapes of the underlying
rocks and fluids. The majority of problems in geophysics are nonlinear with both
deterministic and stochastic methods being widely used.
Molecular Modeling
Nonlinear optimization methods are widely used in conformational analysis.
9
ADVANCED ENGINEERING MATHEMATICS FOR ECE MODULE
Computational Systems Biology
Optimization techniques are used in many facets of computational systems biology such
as model building, optimal experimental design, metabolic engineering, and synthetic
biology. Linear programming has been applied to calculate the maximal possible yields
of fermentation products, and to infer gene regulatory networks from multiple microarray
datasets as well as transcriptional regulatory networks from high-throughput data.
Nonlinear programming has been used to analyze energy metabolism and has been
applied to metabolic engineering and parameter estimation in biochemical pathways.
Video Links
Introduction to Optimization
• https://www.youtube.com/watch?v=I1JqGiG_P_w
Introduction to Optimization: What Is Optimization?
• https://www.youtube.com/watch?v=Q2dewZweAtU
Introduction To Optimization: Objective Functions and
Decision Variables
•
https://www.youtube.com/watch?v=AoJQS10Ewn4
References
• https://www.britannica.com/science/optimization
• https://en.wikipedia.org/wiki/Mathematical_optimizati
on
• https://www.macmillanexplorers.com/optimizationand-linear-programming/17836282
10