Synthesis of Sounds & Music
New Approaches to Performance, Composition,
& Improvisation
Donald Byrd
School of Informatics
Indiana University
20 Nov. 2006
Copyright © 2003-06, Donald Byrd
1
Review: Classification: Surgeon General’s
Warning
• Classification (ordinary hierarchic) is dangerous
–
–
–
–
Almost everything in the real world is messy
Absolute correlations between characteristics are rare
Example: some mammals lay eggs; some are “naked”
Example: musical instruments (piano as percussion,
etc.)
• Nearly always, all you can say is “an X has
characteristic A, and usually also B, C, D…”
• Leads to:
– People who know better claiming absolute correlations
– Arguments among experts over which characteristic is
most fundamental
– Don changing his mind
30 Jan. 06
2
Review: Musical Acoustics (1)
• Acoustics involves physics
• Musical (opposed to architectural, etc.) acoustics
–
–
–
–
Frequency (=> pitch)
Amplitude (=> loudness)
Spectrum, envelope, & “other” characteristics => timbre
Partials vs. harmonics
• Psychoacoustics involves psychology/perception
– Perceptual coding (for “lossy” compression)
• Timbre
– Old idea (thru ca. 1960’s?): timbre produced by static relationships
of partials, plus envelope
– …but attack often more distinctive than “steady state”!
– Rich (interesting) sounds are complex; nothing is static
– Time domain (waveform) vs. frequency domain (spectrum,
spectrogram) views
rev. 11 Sept. 06
3
Review: Creating Interesting Sounds (1)
• Periodic, nearly-, & non-periodic signals
• Approaches to creating sounds
–
–
–
–
Additive synthesis: like painting
Subtractive synthesis: like sculpture
Sampling: like collage
Others: modulation, LPC, physical modeling
• Most possible with analog or digital hardware, but
analog is limited
11 Sept. 2006
4
Review: Creating Interesting Sounds (2)
• Cf. “Electronic Music Tutorial” (Ishkur)
• Additive Synthesis
• Fourier's Theorem
– Any periodic signal => sum of harmonically-related sine waves
• Envelopes: continuous & piecewise linear
– Important special case: “ADSR”
– Attack, Decay, Sustain, Release
• Phase, interference, & beats
– Phase by itself rarely important, but relationships are
– Diagrams & demo in CECM Acoustics Primer, Sec. 8
– Interference between channels or very close partials
rev. 13 Sept. 2006
5
Review: Additive Synthesis & Envelopes
• addsynenv does additive synthesis of up to six partials
–
–
–
–
–
Each has arbitrary partial no., starting phase, "ADSR" envelope
Partial no. can be non-integer => not harmonic
ADSR = Attack/Decay/Sustain/Release (3 breakpoints)
…but addsynenv allows much more complex envelopes
Plays one note with waveform specified by partial nos. & their
envelopes (maybe also phases)
– Simultaneously displays “spectrogram” or “sonogram”
– …but not waveform
– Phase in real world normally has little effect, but can be critical in
recording & digital worlds (e.g., cancellation)
• Additive synthesis can’t create aperiodic (non-definite
pitch) sounds
• ...or many realistic attacks
rev. 13 Sept. 06
6
Review: Creating Interesting Sounds (3)
• Early CCRMA (Stanford) studies of acoustic instrument
sounds
– Envelope for each partial with a few segments
– Similar to addsynenv
•
•
•
•
•
•
Subtractive Synthesis
Early (analog: Moog, etc.) synthesizer model
Signal source: sine/square/sawtooth (for additive)
…or noise & filters (for subtractive)
LFO to add vibrato, tremolo, glissando, etc.
ADSR envelope for entire sound
11 Sept. 2006
7
Digital Sound Synthesis: Real-Time (1)
• Real-time = takes 1 sec. to produce 1 sec. of sound
• (Pure) synthesizers vs. sampling synthesizers
• 1st practical digital synthesizers used FM
(frequency modulation)
– Very efficient => could generate interesting sounds w/
slow computers
– Mid-70’s: Dartmouth Digital Synthesizer = Synclavier
– Early 80’s: Yamaha DX7
• Early/mid 80’s: 1st digital samplers
– Used a kind of “simple” wavetable synthesis…
– With many, many cycles sampled from real instruments
– Kurzweil 250 (ca. 1983-1985): 1st truly realistic sounds
of acoustic instruments
15 Nov. 06
8
Digital Sound Synthesis: Real-Time (2)
• Kurzweil 250 (ca. 1983-1985)
– Very expensive: $12-14K
• Issue: should we give it MIDI?
• People around Kurzweil Music Systems
– Chief designers(?): Ray Kurzweil, Bob Chidlaw
– Important/well-known people: Bob Moog, Alan Pearlman (ARP),
Phil Dodds (Close Encounters…), Lyle Mays, etc.
• Features
–
–
–
–
–
–
Very slow (10 MHz 68000)
No high frequencies (max. SR: 25K)
Not much memory for stored sounds (2 MB ROM)
Piano (1 MB), bowed strings, woodwinds, brass, percussion, etc.
Ray Kurzweil: to just record & play back piano sounds, need 4 GB
=> impossible “data compression” problem!
• Comparison: Vienna Symphonic Library has ca. 200 GB
– Mvmt of Debussy’s La Mer on class website
17 Nov. 06
9
Digital Sound Synthesis: Non-Real-Time
• LPC (Linear Predictive Coding)
– Separate excitation source & resonance
– Used to produce modified vocal sounds by Charles
Dodge, Paul Lansky, etc.
– More radical use for talking guitars, waterfalls, etc.
• Paul Lansky: On Being Digital
– http://silvertone.princeton.edu/~paul/lansky_beingdigita
l.htm
– LPC resyntheses of a single spoken phrase. Ex. 1b &
1c from his Six Fantasies on a Poem by Thomas
Campion (1979).
15 Nov. 06
10
Artificial Intelligence & Music
• What’s needed for a computer to have truly
created music (or anything else)? Some ideas:
– Self-awareness (Christy). But how can you be sure?
– A soul. But how do you know people have them?
– Unpredictability. Computers already have this!
• Related question: what’s needed for a computer to
be truly intelligent?
• Questions for philosophy, e.g., philosophy of mind
• AI studied “objectively” by cognitive science
– Term “AI” (somewhat) co-opted & discredited
• Ex: claim by Kurzweil Music Systems that K250 used AI
– IU has Cognitive Science Program (of studies)
18 Nov. 06
11
Complex Phenomena & Levels of Description
• Ideas from Hofstadter, Douglas (1979). Goedel, Escher,
Bach: an Eternal Golden Braid.
• Complex phenomena can be described on different levels
– Ex: Thinking in minds (concepts, etc.) or brains (neurons, etc.)
– Ex: Audio as waveform, spectrogram, MIDI, or notation
– Basic representations of music correspond to levels
• “Rational and irrational can coexist on different levels”
– Ex: thinking that 2+2 = 5
• We're most interested in high (musical) levels
• Level confusion causes serious misunderstandings
– Ex: NY Times stmt “Hit Song Science computes...with
mathematical precision”
– Ex: stmt by student that a computer could probably describe a
certain piece more accurately than a person could
18 Nov. 06
12
Computers & Algorithmic Composition (1)
• Algorithm: a well-defined procedure, like a recipe
• Algorithmic composition: using algorithms to create music
– Usually, formal procedures to make music via chance procedures
and/or computers
– With dice, etc. instead of computers: Cage, Mozart…& Xenakis
– "There is a radical distinction (both in terms of philosophy and in the
heard result) between composers who use indeterminate (e.g. stochastic)
procedures to compose music and those who use routines which produce
deterministic results given a fixed input into the algorithm." —Wikipedia
– Absolutely wrong (for heard result)! A common misconception
among musicians
• Counterexample: WolframTones
• Stochastic: probabilistic; involving chance
• Basic problem: local & global structure
– Local structure can come from tonality, 12-tone technique, etc.
18 Nov. 06
13
Computers & Algorithmic Composition (2)
• Algorithms w/ no immediate musical relevance used as
creative inspiration
– “Algorithms such as fractals, L-systems, statistical models, and even
arbitrary data (e.g. census figures, GIS coordinates, or magnetic field
measurements) are fair game for musical interpretation. The success or
failure of these procedures as sources of "good" music largely depends on
the mapping system employed by the composer to translate the nonmusical information into a musical data stream.” —Wikipedia
• Other approaches: game theory, cellular automata, etc.
• WolframTones based on a cellular automaton
– Every cell has same rule for updating, based on values around it
– Conway’s Game of Life is best-known cellular automaton
• But is Cope’s EMI algorithmic?
• In interactive situation
– Ex: Pachet (Continuator)
20 Nov. 06
14
Markov Chains & Algorithmic Composition
• Markov Chains
–
–
–
–
–
–
Popular method for algorithmic composition
Fine for local control, but not global
nth order “remembers” last n states
Ex: weather, Reveille melody
Controlled by transition probability table, &…
Initial state distribution
• Stochastic Processes
– Stochastic Finite State Automata
– Markov Chains
– Other stochastic methods
• Ex: Xenakis (Diamorphoses, ST/4, ST/10), Byrd (MUSC => Three
Pieces for Three Winds)
18 Nov. 06
15
Markov Model of “Reveille ”
• Must convert count to probability…
– Transition
Count Probability
– G4 => G4
2
.333
– G4 => C5
4
.667
– C5 => E5
– C5 => G4
4
3
.571
.429
– E5 => C4
4
1.00
20 Nov. 06
16