Physically Based Sound
COMP259
Nikunj Raghuvanshi
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Motivation
Sounds could in-principle be produced
automatically, just like graphics: Sound
Rendering
Sound Rendering has not received
much research effort
Main Goal: Automatic generation of
non-music, non-dialogue sound
Sound Production Today
Movies: Foley Artists
http://www.marblehead.net/foley/index.html
Games: Anyone noticed the
huge sound directory in
Unreal Tournament?
PBS: Sound Production in Nature
Collisions/Other interactions lead to
surface vibrations
Vibrations create pressure waves in air
Pressure waves sensed by ear
Vibration
Surface Vibration
Propagation
Pressure Wave
Perception
Ear
Main Aims of PBS
Physics simulator gives contact/collision
information
Assign material properties for sound,
Wood, concrete, metal etc.
Sound simulator generates sound using
this data (in real time?)
Challenges
Sound must be produced at a minimum of
~44,000 Hz
Extremely High Temporal Resolution
(timesteps in the range of 10-6-10-8 s)
Stiffness of underlying systems (eg.
Metallic sounds. K/m~=108)
Stability may require even smaller
timesteps
Two Approaches
FEM deformable simulation
O'Brien, J. F. et. al., “Synthesizing Sounds from
Physically Based Motion.” SIGGRAPH 2001.
FoleyAutomatic (Modal Synthesis)
Kees van den Doel et. Al., “FoleyAutomatic: Physicallybased Sound Effects for Interactive Simulation and
Animation.” SIGGRAPH 2001.
Main ideas
 Deformable Simulation (arguably) much more
“physically based”
 Foley Automatic: Additive Synthesis
Component
Sinusoids
Sound Signal
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Simulation Requirements
Temporal Resolution
Simulate Vibration as well as Propagation
Vibration Modeling: Deformable Model for
Objects
Propagation Modeling: Explicit Surface
Representation
Physical/Perceptual Realism
System Structure
Vibration Modelling
 FEM with Tetrahedral Elements
 Linear Basis Functions, green’s strain
 Explicit Time Integration
 Typically #nodes = 500, #elements = 1500,
dt = 10-6-10-7 s
Sound Propagation Modelling
 Fluid Dynamic FEM simulation of
surrounding air? Very expensive. Instead…
 Employ Huygen’s Principle: Pressure Wave
may be seen as sum of pressure wavelets
Receiver
Receiver
Pressure
Wave
Pressure
“Wavelets”
Surface Vibrations and Sound

ˆ
Pressure contribution of a patch, p  z v  n
Unit Normal
Velocity

v
n̂
ds
Density of Air
z  c  415 Pa  s / m
Acoustic Impedance of Air
Sound Propagation Speed in Air
Surface Vibrations and Sound
Approximate differential elements with
surface triangles
Apply band pass filters:
Low pass: windowed sinc filter
High pass: DC blocking filter
Result: Pressure known for all surface
triangles
Putting it all together
Pressure/Signal at Receiver
Filtered Average Pressure
Area of Triangle
Visibility Term
Receiver

r
~
pa x r
s(t ) 
cos( )
x r
n̂

x̂
Approximation of Beam Pattern
Distance Falloff
Vibration
Propagation Delay
Accumulation Buffer
Receiver Distance from Source
1
d1
d
Delay 
c
Source
d2
t2= d2/c
Receiver
t=0
Sound Propagation Speed
t1= d1/c
2
Results: Capabilities
 General models
 Generated sounds are accurate
 Stereo Sound
 Doppler’s Effect
Demo
Results: Accuracy
Results: Speed
Scene
TimeStep(s)
Nodes/Elems
Time/Audio Time
Bowl
10-6
387/1081
91.3/4.01 mins
125/265
240.4/1.26 mins
539/1484
1309.7/5.31 mins
Clamped Bar 10-7
Vibraphone
10-7
(~1 day)
Timings on a 350MHz SGI Origin MIPS R12K processor
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Features
 Modal resonance model of solids
 Location dependent sounds
 Impact, slide, roll excitation models
 Real-time, low latency
 Easy integration with simulation/animation
 Practical
 Do not model propagation of sound from source
to receiver
Synthesis Method
Emission
SoundVibration
Samples
Force
User
Propagation
Listener
Speakers
Vibration
Surface u(x,t) of body responds to external contact force
F(x,t)
u(x,t)
F(x,t)
 i
1 2
[ g ( i , x )  2 2 ]u( x i , t )  F ( x i , t )
x
c t
Strain Functional
Speed of Sound
Under suitable boundary conditions, the solution to
the PDE is a sum of sinusoids
Emission
Sound pressure s(t) linear functional L of surface
vibration u(x,t)
s(t)
L
u(x,t)
s(t )  L[u ( x , t )]
i

pi ~ z vi  nˆ
Note that propagation is not modeled in above
The Modal Synthesis Model
s(t)
L
u(x,t)
F(p,t)
“The response u(x,t) of an arbitrary solid object to an
external force can be described as a weighted sum of
damped sinusoids”
Impulse response/modal model
Since L is linear, it implies at s(t) must be a sum of
damped sinusoids too
Example: A 1D string
a1
a0
1st Mode
Frequency = f0
a0e
 d 0t
2nd Mode
Frequency = f1= 2*f0
sin( 2f 0t ) + a1e
 d1t
ak
…Higher modes
Frequency = fk= k*f0
sin( 2f1t ) +...+ ak e
dkt
sin( 2f k t )
Main Idea: Sum contributions of all the modes
The point of impact decides the proportions in which the modes are
to be mixed: ak. Therefore, ak is a function of p, the point of impact
The frequencies and damping parameters are a property of the
object, and independent of how the object is hit
The Modal Synthesis Model
s(t)
L
u(x,t)
F(p,t)
N
s(t )   ak ( p )e d k t sin( 2f k t )
k 1
Kth mode: Gain Factor Point
Damping
of impact
Term
Impulse response,
modal model
Vibration
Frequency
Parameters measured experimentally
Force Modeling
At runtime: Find gain parameters given the location,
strength and kind of force.
Synthesize sound from previous equation.
Impact
Sliding
Rolling
Wavetable
Stochastic
Impact Forces
•Duration: hardness (T)
•Magnitude: energy transfer (w)
•Multiple micro-collisions
Example: F (t )  w  (1  cos( 2t / T )), 0  t  T
Sliding/Scraping
Micro-collisions lead to noisy audio-force
Sliding/Scraping
Wavetable approach
Store force parameters
Modulate amplitude with energy transfer
Modulate rate with contact speed
Synthesis Approach
Fractal noise represents roughness
Filter through reson filter
Resonance ~ contact speed
Width ~ randomness of surface
Rolling
No relative surface motion
Differences with sliding:
•Smoother: Use low
pass
•More damping
•Harder to create
•Less understood
•Essential coupling?
Rolling: Smooth Surfaces
Polyhedral objects do not lead to smooth rolling forces
Instead use smooth surfaces directly
Rolling: Contact Evolution
c(u,v)
 Evolve the contact in
Reduced coordinates
q = (u,v,s,t,)
..
q
d(s,t)
.
q
q
Rolling: Contact Evolution
 Piecewise parametric surfaces, loop
subdivision surfaces
 Explicit integration, no stabilization
 Multiple contacts and conforming contacts
are not handled
 Used only when multiple contacts in close
spatio-temporal proximity
Demo
Dynamic Forces
Pebble-in-Wok Demo
Contact force
Slipping speed
Rolling speed
Impulses
…and locations
Results






0.1% CPU time per mode
Graceful degradation of quality
The bell demo is interactive
Uses a PHANToM for interaction
Authors do not report any real timings
State that “sound quality” is perceptionbased and has no metric as of now
Overview
Background
FEM Simulation
Modal Synthesis (FoleyAutomatic)
Comparison/Conclusions
Discussion
FEM: Physically Rigorous and General
Too slow for interactive applications
Doesn’t scale well
Inappropriate to apply a 30fps technique to
44000fps?
Maybe too general for the problem
domain?
Discussion
Modal model exploits the vibrational
nature
Higher Efficiency
But, not rigorously physically based
Finding the parameters requires
experimentation and “earballing”
No rigorous correlation between physical
and perceptual parameters
Discussion
For Realtime: Need for a technique to
cover the middle ground
Extracting modal parameters in general
requires solving PDEs
Not possible to do in an automated
manner
Approximate modal parameters and then
use modal synthesis?
Conclusion
PBS involves orders of magnitude smaller
temporal and spatial scales
Research is sparse, problems are dense
Main contributions of the two papers
besides vibration modeling:
FEM: Efficient modeling of sound propagation
FoleyAutomatic: Efficient, Approximate models
to handle surface properties and contact forces
References
 O'Brien, J. F., Cook, P. R., Essl G., "Synthesizing Sounds from
Physically Based Motion." The proceedings of ACM
SIGGRAPH 2001, Los Angeles, California, August 11-17, pp. 529536.
 Kees van den Doel, Paul G. Kry and Dinesh K. Pai,
“FoleyAutomatic: Physically-based Sound Effects for Interactive
Simulation and Animation” Computer Graphics (ACM SIGGRAPH
01 Conference Proceedings), pp. 537-544, 2001.
Acknowledgements
Some images were taken from the referred
papers and the corresponding SIGGRAPH
slides