Separation of Variables
Some slides adapted from Ake
Nordlund and Dennis Healy
Separation of variables …
• Main principles
– Why?
• Techniques
– How?
Main principles
• Why?
– Because many systems in physics are
separable
– In particular systems with symmetries
– Such as spherically symmetric problems
Example:
Example: Spherical
Spherical harmonics
harmonics
iωt
iωt
u(r,Θ,φ,t)
=
Y
(r,Θ,φ)
e
u(r,Θ,φ,t) = Ylm
(r,Θ,φ)
e
lm
Partial Differential Equations:
Separation of Variables …
• Main principles
– Why?
• Common in physics!
• Greatly simplifies things!
• Techniques
– How?
Separation of variables:
the general method
• Suppose we seek a PDE solution u(x,y,z,t)
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
∂u/∂x
∂u/∂x == X’(x)
X’(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
22= X”(x) Y(y) Z(z) T(t)
∂∂22u/∂x
u/∂x = X”(x) Y(y) Z(z) T(t)
Separation of variables:
the general method
• Suppose we seek a PDE solution
u(x,y,z,t):
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
∂u/∂y
∂u/∂y
X(x)
Y’(y)
Z(z)
T(t)
∂u/∂y
X(x)
Y’(y)
Z(z)
T(t)
∂u/∂y====X(x)
X(x)Y’(y)
Y’(y)Z(z)
Z(z)T(t)
T(t)
22= X(x) Y”(y) Z(z) T(t)
∂∂22u/∂y
u/∂y = X(x) Y”(y) Z(z) T(t)
Separation of variables:
the general method
• Suppose we seek a PDE solution
u(x,y,z,t):
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
∂u/∂y
∂u/∂z
X(x)
Y(y)
Z’(z)
T(t)
∂u/∂y
X(x)
Y’(y)
Z(z)
T(t)
∂u/∂z====X(x)
X(x)Y’(y)
Y(y)Z(z)
Z’(z)T(t)
T(t)
22= X(x) Y(y)
u/∂y
Z(z)
∂∂22u/∂z
u/∂y = X(x) Y”(y)
Y”(y)Z”(z)
Z(z) T(t)
u/∂z
Y(y)
Z”(z)
T(t)
Separation of variables:
the general method
• Suppose we seek a PDE solution
u(x,y,z,t):
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
This
This ansatz
ansatz may
may be
be correct
correct or
or incorrect!
incorrect!
IfIf correct
correct we
we have
have
∂u/∂y
∂u/∂t
X(x)
Y(y)
Z(z)
T’(t)
∂u/∂y
X(x)
Y’(y)
T(t)
∂u/∂t====X(x)
X(x)Y’(y)
Y(y)Z(z)
Z(z)T(t)
T’(t)
22=
22
u/∂y
X(x)
Y”(y)
Z(z)
T(t)
∂∂22u/∂t
u/∂y ==X(x)
X(x)Y(y)
Y”(y)Z(z)
Z(z)T”(t)
T(t)
u/∂t
Y(y)
Z(z)
T”(t)
Examples
22sin(bt)
u(x,y,z,t)
=
xyz
u(x,y,z,t) = xyz sin(bt)
u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt
22+y22) z cos(ωt)
u(x,y,z,t)
=
(x
u(x,y,z,t) = (x +y ) z cos(ωt)
Examples
Separable?
9
?
22sin(bt)
u(x,y,z,t)
=
xyz
u(x,y,z,t) = xyz sin(bt)
Yes!
u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt
No!
22+y22) z cos(ωt)
u(x,y,z,t)
=
(x
u(x,y,z,t) = (x +y ) z cos(ωt)
Hm…
Examples
Separable?
9
?
22sin(bt)
u(x,y,z,t)
=
xyz
u(x,y,z,t) = xyz sin(bt)
Yes!
u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt
No!
22+y22) z cos(ωt)
u(x,y,z,t)
=
(x
u(x,y,z,t) = (x +y ) z cos(ωt)
Hm…
22+y22) ⇒ p22
(x
(x +y ) ⇒ p
Examples
Separable?
9
9
22sin(bt)
u(x,y,z,t)
=
xyz
u(x,y,z,t) = xyz sin(bt)
Yes!
u(x,y,z,t)
u(x,y,z,t) == xy
xy ++ zt
zt
No!
22
u(p,z,t)
=
p
u(p,z,t) = p
zz cos(ωt)
cos(ωt)
Yes!
Example PDEs
• The wave equation
22
cc22∇∇22uu== ∂∂22u/∂t
u/∂t
where
22≡ ∂22/∂x22+ ∂22/∂y22+ ∂22/∂z22
∇
∇ ≡ ∂ /∂x + ∂ /∂y + ∂ /∂z
Ansatz:
Ansatz:
u(x,y,z,t)
u(x,y,z,t) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z) T(t)
T(t)
Separation of variables in
the wave equation
• For the ansatz to work we must have (lets
set c=1 for now to get rid of a triviality!)
X”YZT
X”YZT ++ XY”ZT
XY”ZT ++ XYZ”T
XYZ”T == XYZT”
XYZT”
• Divide though with XYZT, for:
X”/X
X”/X ++ Y”/Y
Y”/Y ++ Z”/Z
Z”/Z == T”/T
T”/T
This
Thiscan
canonly
onlywork
workififall
allof
ofthese
theseare
areconstants!
constants!
Separation of variables in
the wave equation
• Let’s take all the constants to be negative
– Negative ⇒ bounded behavior at large x,y,z,t
X”/X
X”/X ++ Y”/Y
Y”/Y ++ Z”/Z
Z”/Z == T”/T
T”/T
These
Theseare
arecalled
called separation
separationconstants
constants
22
-- z
z
22
-- 

22 = - µ22
-- „
„ =-µ
This
Thisisistypical:
typical:IfIf//when
whenaaPDE
PDEallows
allowsseparation
separationof
of
variables,
variables,the
thepartial
partialderivatives
derivativesare
arereplaced
replacedwith
withordinary
ordinary
derivatives,
derivatives,and
andall
allthat
thatremains
remainsof
ofthe
thePDE
PDEisisan
analgebraic
algebraic
equation
equationand
andaaset
setof
ofODEs
ODEs––much
mucheasier
easierto
tosolve!
solve!
Separation of variables in
the wave equation
• Solutions of the ODEs:
Example:
Example: x-direction
x-direction
22
X”/X
=
z
X”/X = - z
General
General solution:
solution:
X(x)
X(x) == AA exp(i
exp(i z
z x)
x) ++ BB exp(-i
exp(-i z
z
x)
x)
or
or
X(x)
X(x) == A’
A’ cos(z
cos(z x)
x) ++ B’
B’ sin(z
sin(z x)
x)
Separation of variables in
the wave equation
• Combining all directions (and time)
Example:
Example:
uu == exp(i
exp(i z
z x)
x) exp(i
exp(i 
 y)
y) exp(i
exp(i „
„ z)
z) exp(-i
exp(-i
cµt)
cµt)
or
or
uu == exp(i
exp(i (z
(z xx ++ 
 yy ++ „
„ zz -- ω
ω t))
t))
This
This is
is aa traveling
traveling wave,
wave, with
with wave
wave vector
vector {z,,„}
{z,,„}
and
ω. AA general
general solution
solution of
of the
the wave
wave
and frequency
frequencyω.
equation
equation is
is aa super-position
super-position of
of such
such waves.
waves.
Example PDEs
• The Laplace equation
κκ∇∇22uu== 00
where
22≡ ∂22/∂x22+ ∂22/∂y22+ ∂22/∂z22
∇
∇ ≡ ∂ /∂x + ∂ /∂y + ∂ /∂z
Ansatz:
Ansatz:
u(x,y,z)
u(x,y,z) == X(x)
X(x) Y(y)
Y(y) Z(z)
Z(z)
Separation of variables in
the Laplace equation
• For the ansatz to work we must have
X”YZ
X”YZ ++ XY”Z
XY”Z ++ XYZ”
XYZ” == 00
• Divide though with XYZ, for:
X”/X
X”/X ++ Y”/Y
Y”/Y ++ Z”/Z
Z”/Z == 00
Again,
Again,this
thiscan
canonly
onlywork
workififall
allof
ofthese
theseare
areconstants!
constants!
Separation of variables in
the Laplace equation
• This time we cannot take all constants to be
negative!
X”/X
X”/X ++ Y”/Y
Y”/Y == 00
22
-- z
z
22 = 0
-- 
 =0
At
At least
least one
one of
of the
the terms
terms must
must be
be positive
positive ⇒
⇒
imaginary
imaginary wave
wave number;
number; i.e.,
i.e., exponential
exponential rather
rather
than
than sinusoidal
sinusoidal behavior!
behavior!
Separation of variables in
the Laplace equation
• Solutions of the ODEs:
Example:
Example: x-direction
x-direction
22
X”/X
=
z
<< 00 assumed
assumed
X”/X = - z
General
General solution:
solution:
X(x)
X(x) == AA exp(i
exp(i z
z x)
x) ++ BB exp(-i
exp(-i z
z
x)
x)
or
or
X(x)
X(x) == A’
A’ cos(z
cos(z x)
x) ++ B’
B’ sin(z
sin(z x)
x)
Separation of variables in
the Laplace equation
• Solutions of the ODEs:
Example:
Example: y-direction
y-direction
22
Y”/Y
=
+
z
>> 00
Y”/Y = + z
follows!
follows!
General
General solution:
solution:
Y(y)
Y(y) == C
C exp(z
exp(z y)
y) ++ D
D exp(-z
exp(-z y)
y)
or
or
Y(y)
Y(y) == C’
C’ cosh(z
cosh(z y)
y) ++ B’
B’ sinh(z
sinh(z y)
y)
Separation of variables in
the Laplace equation
• Combining x & y directions
Example:
Example:
uu ==[A
[Aexp(i
exp(izzx)
x)++BBexp(-i
exp(-izzx)]
x)][C
[Cexp(z
exp(zy)
y)++DDexp(-z
exp(-zy)]
y)]
or
or
uu ==[A’
[A’cos(z
cos(zx)
x)++B’
B’sin(z
sin(zx)]
x)][C’
[C’cosh(z
cosh(zy)
y)++D’
D’sinh(-z
sinh(-zy)]
y)]
Message
• Complex exponentials are a natural basis
for representing the solutions of the wave
equation
• A microphone samples the solution at a
given point
• Variation in time is naturally expressed in
terms of Fourier series.
Fourier Methods
• Fourier analysis (“harmonic analysis”) a
key field of math
• Has many applications and has enabled
many technologies.
• Basic idea: Use Fourier representation to
represent functions
• Has fast algorithms to manipulate them
(the fast Fourier Transform)
Basic idea
• Function spaces can have many different
types of bases
• We have already met monomials and
other polynomial basis functions
• Fourier introduced another set of basis
functions : the Fourier series
• These basis functions are particularly
good for describing things that repeat with
time
Fourier’s Representation
F(t)= A0/2 + A1 Cos( t) + A2 Cos(2 t) + A3 Cos(3 t) + …
+ B1 Sin( t) + B2 Sin(2 t) + B3 Sin(3 t) + …
For coefficients that go to 0 fast enough these sums will converge
at each value of t .
This defines a new function, which must be a periodic function.
(Period 2 π )
Fourier’s claim: ANY periodic function f(t) can be written this way
Music
http://www.phy.ntnu.edu.tw/ntnujava/viewtop
ic.php?t=33
Fourier’s Representation
for | t | < π and 2π periodic
Represent f(t) = t
Sin( t) + …
2
1
-7.5
-5
-2.5
2.5
-1
-2
5
7.5
Fourier’s Representation
1 Sin( t) + …
2
1
-7.5
-5
-2.5
2.5
-1
-2
5
7.5
Fourier’s Representation
Sin( t) - 1/2 Sin( 2 t) + …
2
1
-7.5
-5
-2.5
2.5
-1
-2
5
7.5
Fourier’s Representation
1 Sin( t) - 1/2 Sin( 2 t) + …
2
1
-7.5
-5
-2.5
2.5
-1
-2
5
7.5
Fourier’s Representation
1 Sin( t) - 1/2 Sin( 2 t) +1/3 Sin( 3 t) + …
2
1
-7.5
-5
-2.5
2.5
-1
-2
5
7.5
Fourier’s Representation
1 Sin( t) - 1/2 Sin( 2 t) + 1/3 Sin( 3 t) + …
2
1
-7.5
-5
-2.5
2.5
-1
-2
5
7.5
Fourier’s Representation
20’th degree Fourier expansion
2
1
-7.5
-5
-2.5
2.5
-1
-2
5
7.5
How do you get the Coefficients
for a given f ?
A0/2 + A1 Cos( t) + A2 Cos(2 t) + A3 Cos(3 t) + …
+ B1 Sin( t) + B2 Sin(2 t) + B3 Sin(3 t) + …
Fourier’s claim:
•ANY periodic function f(t) can be written this way (SYNTHESIS)
•The coefficients are uniquely determined by f:
∫0
Bk = 1/ 2 π∫
0
Ak =
1/ 2 π
2π
f (t) cos( k t) d t
2π
f (t) sin ( k t) d t
(ANALYSIS)
Fourier Analysis: match data with sinusoids
1
0.75
0.5
s(t)
0.25
2
4
6
8
10
Time t
12
-0.25
-0.5
-0.75
1
1
1
0.75
0.75
0.75
0.5
0.5
0.5
0.25
0.25
0.25
2
4
6
8
10
12
2
4
6
8
10
12
-0.25
-0.25
-0.25
-0.5
-0.5
-0.5
-0.75
-0.75
-0.75
-1
-1
1
1
1
0.75
0.75
0.75
0.5
0.5
0.5
0.25
0.25
4
6
8
10
12
4
6
8
10
12
2
4
6
8
10
12
-1
0.25
2
2
2
4
6
8
10
12
-0.25
-0.25
-0.25
-0.5
-0.5
-0.5
-0.75
-0.75
-0.75
-1
-1
-1
1
S[k]
=
∫
0.8
0.6
0.4
s (t) cos( k t) d t
0.2
6
10
14
18
22
26 28
Frequency k
Complex Notation
For f, periodic with period p
Fourier transform f(t) -> F[k]
F[k] =
=
1/p
1/p
∫0
∫0 f (t) cos(2 π k t/p) dt
∫0 f (t) sin(2 π k t/p) dt
p
f (t) e- 2 π i k t/p d t
p
− i/p
p
Inverse Fourier transform F[k] -> f(t)
f(t) =
Σ
F[k] e 2 π i k t/p
k in Z
Sampling
Fourier representations work just fine with sampled data
Simple connection to Fourier of the continuous function it came form
Familiar example: Digital Audio
Measuring and Discretizing Input field
Physical Field
(continuum)
PHYSICAL
LAYER
Sample
Physical Field
(continuum)
PHYSICAL
LAYER
Quantize
Physical Field
(continuum)
PHYSICAL discretized waveform
LAYER
Code and output
Digital Representation
Physical Field
(continuum)
PHYSICAL
LAYER
…3, 8, 10, 9, 3, 1, 2…
Sampling
Often must work with a discrete set of measurements of a continuous function
10
10
1
8
0.5
6
-2
4
-2
-3
-1
-1
1
-0.5
2
-3
-1
1
12
3
3
-1
Sampling
Takes a function defined on
10
R
and creates a function defined on
10
Z
1
8
0.5
6
-2
4
Sh
-2
-3
f(t)
-1
-1
1
-0.5
2
-3
-1
1
12
3
φ[n] = f(n h)
3
-1
h
Sampling
In this case, it is a periodic function on Z,
(Assuming p/h = N)
10
10
1
8
0.5
6
-2
4
-2
-3
-1
-1
1
-0.5
2
-3
-1
1
12
0123
φ[n] = f(n h)
3
…..
3
-1
N
φ[n+N] = φ[n]
h
DFT
∫0
p
N-1
f (t)
e- 2 π i k t/p
Σ
dt
φ[n] e -2 π i k nh /p
n =0
10
10
1
8
0.5
6
-2
4
-2
-3
f(t)
-1
-1
1
-0.5
2
-3
-1
1
12
3
3
φ[n] = f(n h)
-1
h
DFT and its inverse for periodic
discrete data
Φ[k]
N-1
Σ
=
φ[n] e -2 π i k n h /p
p =Nh
n =0
N-1
=
Σ
φ[n] e -2 π i k n / N
n=0
This is automatically periodic in k with period N
Inverse is like Fourier series, but with only p terms
DFT: Discrete time periodic
version of Fourier
“time” domain
“frequency” domain
N
1/Ν
Σ γ[k] e
-2 π i k m/N
= Γ [m]
k =0
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
N
1 2 3 4 5 6 7 8 9101112131415161718192021
γ[k]=
γ[k], on PN
i.e. on
Z , Period
Σ
k =0
N
1 2 3 4 5 6 7 8 9101112131415161718192021
Γ [k] e 2 π i m k/N
Γ [m] on PN
i.e. on Z , Period N
o
-0.5
0
0.5
O
∫0
1
-1
10
7.5
C
e e so so
Fourier
p
f (t) e- 2 π i k t/p
d t /p
1
0.8
0.6
5
Σ
2.5
0
-1
-0.5
0
0.5
1
f(t), period p
0.4
0.2
F[k] e 2 π i k t/p
1 2 3 4 5 6 7 8 9 10 1112 1314 1516 1718 1920 21
F[k] , on Z
k in Z
N-1
S p/N
Σ
1/Ν
1
γ[k] e -2 π i k m/N
k =0
0.8
1
0.8
0.6
0.6
0.4
0.4
0.2
0.2
1 2 3 4 5 6 7 8 9101112131415161718192021
γ[k], on PN
i.e. on
Z , Period
“time” domain
N
N-1
Σ
k =0
PN
1 2 3 4 5 6 7 8 9101112131415161718192021
Γ [k] e 2 π i m k/N
Γ [m] on PN
i.e. on Z , Period N
“frequency” domain
Discrete time Numerical Fourier Analysis
DFT is really just a matrix multiplication!
Γ [m] =
N-1
1/Ν
Σ
γ[k]
e -2 π i k m/N
k =0
time index
Γ[0]
Γ[1]
Γ[2]
γ[0]
γ[1]
γ[2]
Freq. index
60
50
40
=
.
.
.
.
.
.
30
20
γ[N-1]
10
Γ[N-1]
0
0
Γ
=
10
20
30
FN
40
50
60
γ
Numerical Harmonic Analysis
FFT: Symmetry Properties permits “Divide and Conquer”
Sparse Factorization
Fmn = ( Fm ⊗ I n ) ⋅ T
mn
n
15000
12500
10000
⋅ ( I m ⊗ Fn ) ⋅ L
mn
m
Naive
60
50
40
30
20
7500
10
0
5000
2500
Fn
0
10
20
20
30
40
40
50
FFT
60
60
80
100
120