1
>> Ivan Tashev: Happy New Year, everybody. Trust you had an enjoyable holiday season.
Thanks for joining us tonight. It's my pleasure to introduce Nick Wiswell. He's the creative
audio director at Turn 10 Studios and manages the creative audio direction of the Forza
franchise. He was previously at Bizarre Creations in the U.K. And his audio game series,
Project Gotham Racing, pushed the boundaries of sound in racing video games creatively and
technically which got them nominations for some awards like the Develop and the GDC choice
awards.
He joined Microsoft and Turn 10 two years ago, and he has got more nominations for the Forza
franchise, the TEC award and the GANG award for Forza Motorsport 4 and Forza Horizon.
So welcome, Nick.
[applause]
>> Nick Wiswell: Thank you. I'd like to thank the AS for inviting me here today to talk to you.
Thank you all for coming out. Hopefully I'm only supposed to entertain you long enough so you
don't leave, but the door is just there, if I do go on a little too much, which I am known to do.
So a little bit about me. Some of it's just come up there, who do I think I am. Well, I've been
working in games for 14 years now. I got my big break completely by chance by bumping into
my former boss on a train at a trade show. I was working in a video game store and wanted to
make games. He was looking for somebody who had a passion for games who'd just help out
around the office, and within the first year I'd shown myself to be capable of editing sound and
making sound for games. And that was the start of it. They asked me to set up an audio team.
And 13 years on here I am with a reasonably large number of games behind me.
So I spent 12 years at Bizarre Creations in the U.K. I did leave just before they were shut down
unfortunately by Activision a couple of years ago. I've been at Turn 10 now for two years. I've
made quite a lot of games. So there's quite a few up there, of course, various formats. If
anybody would like to buy Geometry Wars and Fur Fighters, they've just come out on iOS. I do
not get any credit for those. So my name's in them, but I make no money. But I know people
who do, and they're very good guys.
So car sounds. Just an engine and four wheels. It's actually a metal box, isn't it, really? No. No,
it is not. So why do cars sound the way they do? This is the interesting bit. You are now going
to get a quick study of fluid dynamics and mathematics. Anybody who didn't think they were
getting science today, I apologize, but it's what you need to know.
So, first off, you're all musical guys, so think of a car as a very large wind instrument. No, not
me after beans, a very large wind instrument. The engine is acting like a mouthpiece creating the
airflow, so the engine sort of sucking air in and blowing air out.
The intake and exhaust systems act as resonant tubes going in and out. The length and diameter
of the tubes, just like in a wind instrument, affect the sound, combine that with the number of
cylinders and the RPM of the engine, and that creates the fundamental note that you'd normally
get from the voom action in your mouth.
That, the size of the tube, then creates the harmonics, and that is the overall sound of the car.
And this is why no car sounds like any other car. Two cars that share the same engine but have a
2
different exhaust system won't sound the same. They will have a very slightly different sound.
Now, that could be just down the to the harmonics. The fundamental will be the same. But the
length and shape of the pipe will create very different harmonics and give you a different sound.
And here's your mathematics. If you want to calculate what the fundamental hertz is, take the
RPM, divide it by 60 because it's revolutions per minute, we're converting it to seconds, multiply
that by the number of cylinders, which is how many times it's actually making the [making
sound] sound as the exhaust pops, divide that by 2, because an engine has to rotate completely
twice to form one revolution ->>: [inaudible]
>> Nick Wiswell: That's for a four-stroke engine -- and you will have the fundamental in hertz.
So this is an equation I worked out 13 years ago, and it's done me proud to this day because it
allows me to listen to a recording, do a quick FFT analysis and tell you exactly what the RPM is.
Or if you've been editing cars as long as I have, you can probably go, hmm, 3100 RPM.
The worst part is when you're trying to sleep at night and you can hear this buzzing in the
background, and in your brain you're thinking that's about 1,200 RPMs. It's not a good place to
be.
So what does that sound like? Well, I broke down that each of these components sounds
different and creates a resonant sound. So here I've brought the recording along. In this case it's
a Ferrari 355 Challenge race car. So this is an engine recording. So this a microphone. In this
case it was probably a DPA 4011 up close to the engine just trying to capture the mechanical
sound of the engine moving through the rev range.
[Car noise]
>> Nick Wiswell: Sorry. That needs to be louder.
>>: So is that your Ferrari, then?
>> Nick Wiswell: No. No, but it was actually recorded just up the road.
>>: Really?
>> Nick Wiswell: Yeah, FAME Automotive on Redmond Way.
Here we have the exhaust. So this is a closed mike on the exhaust. Usually for this we would
use a Sennheiser MKH 40 as our microphone of choice. But you'll hear how it has similar
characteristics but it's still quite different.
[Car noise]
>> Nick Wiswell: Now, the bit that most people don't realize is making so much sound, the
intake system. So the intake is just where the air is being sucked in.
3
>>: This is [inaudible] recording is [inaudible]?
>> Nick Wiswell: Yes.
>>: Okay.
>> Nick Wiswell: So this is all taken from an eight-track session. So in this case the intake is
creating a very unique sound compared to the other two.
[Car noise]
>> Nick Wiswell: So each one of these, while sounding like a Ferrari in its own right, in order to
truly capture the entire sound of the Ferrari, you need to record all of them.
Now, this car was a specific example that I called out because the way the car is configured is
you have the engine in the middle at the back, but then the four cylinders on this side of the V8
and the four cylinders on this side of the V8 are wired up completely independently.
So there's an intake system attached to these four and an intake system attached to these four,
and then an exhaust system attached to these four. So we need to make sure we record both at
the same time, otherwise we're missing half of the sound.
Now, some cars, like a Corvette, have a single point of intake, and they also have a crossover in
the exhaust which means the exhaust note comes together and then they split out again. So in
that case it is possible to record one exhaust pipe and just one intake system, where with this car
we actually had to go out of our way to get both, because when we tried to record just one, we
realized that it just doesn't sound right, what's missing, and then you have to spend time looking
at the way the actual plumbing works in the car.
Yes, question from the back.
>>: Is there a reason the volume envelopes of the bottom two are asymmetrical compared to the
top one?
>> Nick Wiswell: That's just the natural dynamics of the actual source. The engine you
generally find is much noisier, so you do get a little bit of bleed-over from the exhaust manifold
and the intake manifold. So we do notice that that's the way. But it may be down to our
recording. I'm not totally sure. But we do see this quite a lot, especially with the louder race
cars.
>>: If you think about it and then you realize why that second one is positive though, I note,
because you're putting out pressure impulses from -- that are all above atmospheric.
>> Nick Wiswell: Yes.
>>: I'm surprised you don't see more negative going in the other one.
>> Nick Wiswell: Yeah. I never actually thought of it that way actually. That's -- oh, new line
of inquiry. Oh. What I'm doing tomorrow.
4
>>: [inaudible] more gas comes out of the -- more gas volume comes out of the engine, goes in
[inaudible] same temperature.
>> Nick Wiswell: Yeah. So we just take those recordings and play them in the game, right?
No. If only it were so simple. If we were making a linear film, it would be great because we
could just record the car doing what it needs to do and then plug it over the top of the film, and
there you go, we're done. But the audio needs to match what the car is doing in the game.
So that in itself brings up challenges. It's just like the player can do absolutely anything at any
point. So how are we going to break all that down.
So to do this, the game has a physics system that actually you're driving the car on the tracks, so
we're simulating real world, real car physics. So we feed into those and use those to drive the
audio. So any point we know what the RPM is, what the throttle position is, what the engine
load is, what gear you're in, how much boost pressure is being created for a forced-induction car.
We've got hundreds of physics parameters available, and all of these will be doing something to
the audio at any one time. Some of them are driving the engine, some of them are driving other
systems, but without the physics system, we don't know what the car is supposed to be doing,
therefore we don't know how to play it back.
So how's that work? Well, it's complicated. Of course it is. But I will try and simplify, and if
anybody wants me to go into more detail, I can.
We need to record the car doing as many things as possible. We need to record it accelerating,
we need to record it decelerating. We need to record it going up through the gears, down
through the gears. We need to record it at a steady state at various load levels.
So in order for us to do this, we record in dyno. This allows us to drive the car but in a very
controlled way, in very controlled conditions, and it means we can put the microphones where
we need them to be to capture the sound we want to capture.
If we were doing this on a road, which is also a possibility, you have slightly less control, you've
got wind noise to worry about, and you can only put the microphones where you can actually
physically attach them on the car.
So with this it means that we can actually put mics 20, 30 feet behind the car to try and simulate
that view you get in the game of looking back at the rear end of the car. We could never do that
on the road unless we had some sort of really crazy boom pole set up, and I wouldn't like to try
and run that at 160 miles an hour on a racetrack. That would sound like a bad idea.
So we have this recording of the car going up and down through the gears, doing all the things
we want to do. The simple thing then I would say is you take it, you chop it up into little pieces.
Those little pieces allow you to then blend between them all, so it's playing back matching to the
physics, so we'll have lots of little samples covering the entire RPM range of the car in different
load conditions, in different ways, and then we use the parameters we get back from the physics
to stitch it all back together again so it's playing the right sound at the right time. And it all
comes through.
So how can you do this? Well, there are several ways to do this that I've used over the years, and
5
some of these are still being prototyped as we speak.
You could record the car at fixed RPM, so you could record it at, say, every 500 RPMs all the
way up the rev range, from idle through to redline.
And then you can take those loops and using some of the middleware tools available, we use
FMOD on Forza, you can take those samples, overlay them on top of one another, pitch shift
each sample to match the RPM and then cross-fade the entire way up the rev range. So that will
give us a sort of complete sweep of each component through the entire revs.
Granular synthesis. In this case we take a recording similar to the one you've heard and then you
use some specific tools that will actually go through and isolate each individual RPM sequence
across the entire range, extract those as samples, and then as you sweep through them you're
playing back lots and lots and lots of tiny samples all backed to one another.
And as you go through a rev range and you slow down, it will sort of modulate between them
and pick them and just pitch match them slightly. And this is just a different way of making it.
And what's being investigated right now is full synthesis. How can we take a recording and then
synthesize that back. So think of it like an MP3 looks at a music file and determines which
frequencies are important and which ones aren't and then does that. Can we do this with car
recordings, can we actually synthesize it from actual sine waves and first principles and fill in the
gaps with noise.
The great advantage you've got here is with all the others, one of the big systems we have in
graphics in games is a level of detail system. So as a car goes off into the distance, we can
reduce the complexity of the visual model because it's certainly a small thing on the screen; you
can't really see the difference.
But with audio we don't have the ability to do that. We can't just start derezzing samples on the
fly because we have a lot of other things to think about.
So with a full-synthesis system, that would allow us to reduce the number of audible frequencies
and fill the gap in with noise, because as it goes into the distance, you've got a much lower
frequency spectrum anyway because it's going to sound filtered and muted and distant. So you
don't need those high-end frequencies, so it will allow us to scale that in real time.
So this is something that is being investigated now and could well be the future of game audio
for cars.
>>: [inaudible] for this?
>> Nick Wiswell: Sorry?
>>: [inaudible] RPC synthesis, where you predicted [inaudible]?
>> Nick Wiswell: Yes.
>>: That just seems to be the obvious.
6
>> Nick Wiswell: Yeah. There are several different synthesis methods we could use to
reproduce that.
So what else do we need to consider? Lots of things. So making sure the sound of each car
component changes correctly based on the physics, using sample changes and realtime DSP.
So one of the things to go into more detail here would be an exhaust sound changes significantly
as you apply the throttle. A lot of it these days is manufactured with various valves that open
and close in the exhaustion system. But even on the older cars you get character shifts.
So sometimes we could swap out samples. We can have samples at different load levels that we
can swap between. Or it may just be that we could apply an EQ or a filter of some description to
try and change the character of the sound. Or it could be that we're applying a distortion effect or
we could be applying many, many different sort of ways we can approach how we go about
changing the character of a sound based on the actual physics.
The intake system, for example, makes no sound off throttle. Completely silent. It's only when
you put your foot on the gas that it sucks the air in and it starts to resonate and make the sound.
So we need to make sure that we've got the volume in that case mapped to the throttle position
and the engine load so we can modulate the volume based on the physics coming in from the car.
Then we've got to worry about sounds for gear changes, backfires, exhaust back-pressure, turbo,
supercharges, transmission. A car is not just an engine and an exhaust and an intake system.
There's all these other layers of sound.
And it's trying to build all that together to create an overall hole. So we'll have individual gear
change samples so you'll get the actual clunk of the gear engaging, which will become more
prominent depending on how close to the car you are. The closer you are, if you're in the car,
that's a more prominent sound than if you're ten feet behind it. Backfires and exhaust pressure
obviously will be monitoring how the air is flowing through the exhaust. We've got the physics,
we know it's decelerating. Some cars pop more than others. That's where our recording library
comes in handy, because we can then -- we know what it did in real life, so then we've just got to
try and reproduce that.
Turbo and supercharges. They're generally separate systems that we model on top of the normal
game and are driven by their own physics. So we'll have a physics modeling what the turbo is
doing. And transmission whine for straight cut gears. If you've ever heard onboard video of a
race car, you just get this high-pitch scream that just gets higher and higher and higher. And
that's just basically anybody who's driven a car in reverse knows you get that [making noise].
Well, that's a straight cut gear. And instead of the normal gears, which are helical cut so they
mesh together seamlessly and make no noise, you have a straight sawtooth cut and it creates this
whine as the two gears clash together. And it's much, much stronger, so they use them on race
cars.
And the interesting part of that is there are two tones. There's one where the gearbox is attached
to the engine which pitches up and down with RPM, but then there's another one that's attached
to the actual final drive and the transmission itself. So as you go through the gears, as the size of
the cog decreases, the tone rises and it gets higher and higher and higher in pitch.
So all of this needs to be modeled. And then also in our game, to make things even more
7
complicated for us, you can adjust the gear ratios of every car. So we have to sort of map how
pitch reacts to every gear ratio and how that comes together.
Tire noise. Seems crazy, but tire noise is actually the most complex system in our game. Far
more complicated than the car is. Because if you think about what a tire can do, you've got the
contact service between the rubber and the road, which will make the standard rolling sounds
you all hear when you drive around. But then as you start to skid, all the chemical changes kick
in in the rubber compound and you start to hear it skid. And it sort of transitions. If you're going
just sideways, you hear that transition to a sort of scrubbing sound, into a slight squeal, into the
rubber starting to sort of really get upset with what you're doing to it, and it starts to scream a
little bit, and then you'll sort of lose control completely and just get this huge screaming, wailing
sound.
Now, that's just lateral movement. Then you've got what happens when it actually loses traction
because you're spinning the wheels. That's a whole different set of sounds with multiple layers
of sound. And then you can lock the brakes. So that's a whole new set of sounds.
So if you think about our time model, we've got a roll, five layers of lateral skid, two layers of
longitude and wheel spin, and two layers of lock brakes. Plus ABS. Per tire. And there's four
tires.
So -- and that's for driving on the asphalt. Then we do the same for cobbles. Then we do the
same for concrete. Then we do the same for dirt. Then we do the same for gravel. Then we do
the same for sand.
Very, very big system. At any one time, we can have -- the tire system can use up probably half
the voices that were available to us in the game plate side. I'll talk about that a little bit more
later. But at some point half of our entire voice count is used just to play tire audio. So it's a
pretty big system.
Environmental reverb and reflections. Obviously we want the car to sound like it's in the space
it's in. When you drive through a tunnel, the first thing you should always do is put your foot
down and hear what it sounds like. If you're not doing that, why not? If you've driven a really,
really nice sounding car, then we need to simulate that.
So the system we're designing has various reverbs that change as you drive around the track. So
if you're out in an open field, if you're on the Top Gear test track from the U.K., it's just a runway
in a field and there's nothing around it. There's virtually no reverb at all. Very, very little going
on.
On some of our tracks in the Alps, we've got big tunnels, we've got huge mountain walls, we
need the sound to react properly. Even on most racetracks you'll have open sections and then
you'll have big built-up grandstand sections. So there are loads of ways in which the sound can
change.
So we'll be constantly playing with environmental reverbs. And we've also started investigating
modeling early reflections directly. So we're actually taking the sound of the car, mixing it down
to a single point source, position it in the world where the wall is, looking how far away that is,
delaying that sound by how long it would take for the sound to go there and back again, and then
positioning that in 3D space and applying filtering properties based on what surface it's reflecting
8
off. And then doing that multiple times all around the car.
So with the systems that we're building here, and you can hear some of this today, we are
actually really trying to seat the car in the environment around it and constantly have that
change.
Doppler shift. Most sounds in racing games are moving all the time, so obviously you need to
simulate the [making noise] as it goes past. Every sound does that. So we've constantly got a
pitch shift algorithm running on every sound in the game so that we can pitch it in real time.
Would love to investigate doing it properly with delays and actually having the delay shift and
really modeling the speed of sound perfectly. We're not quite there yet, but one day. One day.
Collisions, crowds, ambient world sounds. So at some point you are going to crash. When I
drive shortly for you, I'm going to crash a lot.
We need to simulate crashes between two metal cars, a car that's carbon fiber and the car that's
made of fiberglass. Hitting the
Armco barrier at the side of the world, driving into a tire
wall. All these things are all different surfaces. They all need to be modeled at different impact
levels. You can drive into them at 10 miles an hour, you can drive into them at 200 miles an
hour. So a huge amount of work goes into trying to create all the collision sounds in the game
and all the different surfaces you can collide with.
Crowds. On some tracks you've got 10 people standing by the side of the track making a little
bit of noise. In other tracks you've got 10,000 people in a huge grandstand roaring at you as you
drive by. So we've got to sort of work out how to position all these things and have them react
correctly to what's going on on the track.
And ambient world sounds. This is something that we focus on because sometimes people want
to stop. It's a racing game. You shouldn't stop. You should go fast. You shouldn't hear the
ambient sounds. But if you do want to stop, if you're racing on Forza 4 and you stop by the
waterfall in one of our tracks for [inaudible], you'll hear the water crashing down.
One thing we're trying to do with ambient world sounds now is create a sense of speed by
creating little whooshes as things go past you, sort of creating that sound reflection of little
objects and Hollywooding it up a bit. It's what everyone expects at high speed.
Question from the back?
>>: How do you analyze the sound of collisions to know what to replicate?
>> Nick Wiswell: I have a little video at the end of today that I'll show you how we did some of
our collision recordings. But it's not in the video, but one way we analyze what two cars sound
like when they're cashing into each other at 150 miles an hour is we went to a crash test facility
out in the Mojave Desert where we essentially got two old cars on a big pulley, essentially pulled
the lever and the two got pulled into each other at very high speed. It doesn't sound all that
exciting. It's very, very loud, very short transient, and very little else.
>>: [inaudible] to your replication of collisions, then? You're putting in what it should sound
like?
9
>> Nick Wiswell: Well, putting in what people perceive it to sound like.
>>: [inaudible] Hollywood, all the way?
>> Nick Wiswell: Hollywood [inaudible] ruin everything. Guns do not sound like that, cars do
not sound like that, and collisions do not sound like that. But having unfortunately been in a car
crash, it's a very scary thing. And a big part of that is the feel, but a big part of that is the sound
because it's very loud and there's all this crumpling and noise, and it's most unpleasant. So it's
more about reproducing that unpleasantness while staying within the realm of possibility and
realism. Everything in our game is sourced from a real recording, but we may have just turned it
up a little bit.
Game music. I hate this. Sorry, Lance. Lance is our composer. He arrived late and didn't
introduce himself.
Loads of games have game music while you drive. I think engines are glorious and everybody
should listen to those and turn the music off. I've spent all this time working on them. The last
thing I need is somebody's music getting in the way.
But people expect game music, so it's a big part of working out how we have to piece all that
together.
Interface sounds. You'll hear some of these later. But navigating through the menus, you want
some sort of feedback mechanism, some sort of haptics that let you know that you've pressed A,
that you've moved left and right.
In the game it's little hood -- we have a scoring mechanism whereby as you do a good turn or as
you drift or as you work that back, there's a little hood message on the green, and we just put
little peeps and noises in just to attract your attention to that.
In one of my first games, Project Gotham Racing, we had a mechanic whereby you had to drive
stylishly. So it was all about cool drifts and turns and near misses. And we created a system
whereby there was a counter in the top right-hand corner of the screen and there was just this
little ticking sound that played as the score racked up. And the idea was if the score kept racking
up, you just get this [making noise] the whole time. And then when you -- if you managed to
rack up score and not crash for three seconds, I believe it was, you just got a little banking sound
to say you'd scored. You know, your points have racked up, you've earned those points. And if
you didn't, you got a [making noise] to let you know that you'd missed it.
What I didn't realize is a lot of our better players were actually using that tick, weren't ever
looking at the thing, they just knew as long as that tick kept going, they were scoring. And that's
all they needed. And it became a huge part of the franchise. And I spent absolutely no time on
those sounds at all because I thought they were really unimportant and they were like knocked
together in an hour and suddenly it defined the entire way everybody played the game. So that
was a really happy accident, but I was really happy with that.
So that's it, then. A complete car. No. So cars are very loud. Race cars are very, very, very
loud. We've -- the loudest we recorded was last February. I was down in LA recording a Mazda
787B race car, which is a four-rotor crazy Wankel engine design.
10
And we knew it was going to be loud because the engine is basically just a series of four -- it's
sort of like a rotating device with these chambers and into the chamber goes fuel and air and it
ignites and it burns. And you could look into the intake and see the inside of the pistons of the
rotors, and then you looked in the exhaust, which is this long, and there was the outlet of the -and it was just like, oh. And our SPL meter goes up to 140 dB, and it just pinned it the whole
time. It was the loudest thing I have ever heard.
But that's great for me. I was there. But you're going to take this home and you're going to play
it on your home theater system, or, worse still, you're going to play it on a little flat-screen TV,
these tiny, tiny little speakers that can't make any sound. How am I going to make it sound loud?
Well, one thing we discovered very early on is when things are that loud your ears distort. Your
ears will clamp the sound. And we played around with several distortion effects to try and
reproduce that and discovered that distortion is a great way to simulate loudness. Especially on
cars.
So it's also, as we go back to perception, most people's experience of what a Ferrari sounds like
is by watching one on YouTube. YouTube videos sound really bad.
So everybody thinks it sounds like the big distorted mess they've heard on their camera phone
recording that they posted onto YouTube. So, again, we're going into perception, and we've got
perfectly clean recordings, they're unclipped, everything is beautiful, but when you play it back,
even at full volume, you just don't get that sense of, oh, my God, this is hurting my head.
And the distortion is a really good way of simulating that. So on Forza 4 we actually work with
iZotope and got them to develop a custom version of the Trash distortion algorithm, which many
of you music people will hopefully have used at some point, but it's a multiband distortion that
we could use realtime in game. So it allowed us to play around with not only the amount of
distortion but the amount of distortion we route into all three bands of the thing. We could have
really low-end distortion and not really touch the high end. We could play around with it. And
it gave us a lot of control that allowed us to reproduce how we remember the car sounding on the
day.
The other problem we have is cars move around a lot, especially in a game. So we have to worry
about for your car it's generally in front of you, but all the other cars on the track will be moving
around from left to right, they'll be going behind you. So we have to be able to work out how
we're going to simulate that. Most games console these days are surround sound, so we can do
full 5.1 or if you're making a PlayStation 3 -- sorry, we're in a Microsoft building -- you can do
7.1. So we have algorithms that work out how to pan the sound relative to the camera. So you
can always tell if there's a car coming up behind you, is it alongside you, where is it, if you're
jostling for position, how they move together.
We also use a lot of realtime DSP to simulate distance. So as a car goes off into the distance,
you'll start to hear the sound become more filtered, you'll start to hear the environment come
through a little more in the mix.
And we have independent volume attenuation for wet and dry, so we're playing around with the
environmental stuff.
11
And on one game I've shipped in the past, we even used a phaser effect because we had a lot of
replay long shots. You could hear the car in the distance. And just that subtle swirling you
could get simulates that sort of ground effect you get in the distance. And when you hear planes
fly overhead or you hear things off in the distance [making noise], a phaser was a really
interesting way it was reproducing that. But we only applied the phaser to the wep send
[phonetic]. So it was just a sort of distant verberant swirly noise that you were hearing sort of
generally from over in that direction. And it was a really interesting way it was reproducing that
sound.
So what does that sound like? Well, fortunately I brought my Xbox with me. Just turned itself
off. Sorry.
>>: [inaudible]
>> Nick Wiswell: Yes. For all the audio geeks out there. It's the best thing in the world.
Everyone should have an Xbox that sounds like R2-D2.
So what I'm going to demo for you now is Forza Motorsport 4 which will demonstrate all the
things I've just spoken to you about. Which was all set up, but it appears to have turned itself off
in the last -- while I've been whittling on.
So what I'm going to do is I'm going to load up the Ferrari that we heard before. I've played you
the record enough. On one of our tracks, and I'm just going to drive it around badly mostly.
[demo playing]
>> Nick Wiswell: New Microsoft Studios logo normally has a little piano on it, sounds very
Disney. Screw that. V8 every time.
So music by Mr. Lance, everybody. So we talked about me building a car. Well, this car has
500 game -- cars on disk and another 150 released by downloadable content. So imagine doing
everything I've spoken about today 650 times a year. Oh, no. One more. Not paying attention.
There we go. Challenge. So, yeah, I'm going to drive around. You can hear it in real time. I'll
move the camera around so you can hear as the sound changes based on the camera position. I'll
look back at the car so you can hear the engine view versus the exhaust view and flick around
between the two.
I'm going to do some -- try and do some drifts. I'm going to probably crash a few times, I'm
going to do some skids. There's going to be a lot of different things. There are many people
who can drive better than me at this game, and most of them are probably in this room. You'd
think I'd get good at this after so many years of doing it, but no.
[demo playing]
>> Nick Wiswell: So the first thing you'll hear is I've turned the music off, as it should be.
[demo playing]
>> Nick Wiswell: Whoops. Sorry. Hit the wrong button. See, if we happened to be in 5.1
today, you'd hear that when you're in the cockpit view, we actually try and map the sound to the
12
speakers, because the tire audio is a huge control for us in letting the player know how much
traction they've got and the way the car drives. And if you're playing in 5.1, you'll actually
notice that you'll get the tire from each of the four speakers all working independently mapped
on the physics.
So we always thought this was an interesting mechanism, but then when we actually upgraded
the tire model and made it more detailed, we actually noticed that the best drivers were actually
shaving time off their lap times. So it was a -- it's not just a cool-sounding thing, it's actually a
great way to actually let the player know exactly what's happening on the track at any time.
So if I just quit race, I can also get a little bit of the replay sounds, so the sounds shifting based
on distance, based on angle. A little bit of Doppler shift. And that's how we make a car.
[applause]
>> Nick Wiswell: Thank you. So coming up we have -- I'm going to talk about mixing a game
like this, which obviously a very different thing. But I don't know if now is an opportune point
to break.
Any questions from the room?
>>: Do you queue the four speaker controls off the audio?
>> Nick Wiswell: They're generally controlled from the physics, but we have tweaked both the
force feedback and the audio to work together. So we spend a lot of time working with the game
play team to actually control at what point we know roughly where traction is going to be lost on
our slip scale, and we try and make sure that the audio queues are actually changing at the right
points to actually map to the force feedback and how all that works.
On previous games we have at it so our LFE channel was helping drive the rumble in the
controller. So the actual sort of pulsing we were getting from the LFE was actually pulsing the
controller. So there are certain times where audio does that. Like, for example, the backfire
system that we've got in the game is driven entirely by audio, and then we send a message to the
graphics engine to play the visual effects that go with those backfires. So when you see the
flames coming out of the car, it's audio that's determining when that should play and then telling
them to do a visual effect, which is one of the few times I actually get to tell the graphics team
what to do. Which is really nice.
>>: So now that I'm actually listening, you haven't spoken much about wind noise of the car.
>> Nick Wiswell: Yes. I missed that off my list. Yes, there is wind noise.
>>: Okay. And is that also per car or is there generic wind that you have a spoiler, down a
spoiler [inaudible].
>> Nick Wiswell: We so some variation, but it's mostly speed based. Like obviously there's
very few cars that you really get to hear the 250-mile-an-hour wind sample on. But they are
there.
So another way that we sort of control some of this is like playing around with it. And I'll go
13
through it in the mixing -- wind noise is a very big part of driving very fast, but it's also sort of
washy white noise that can get in the way. So we try and pick our moments to really accentuate
that.
When you're driving in a straight line, it's -- you can sell wind noise and whooshes really, really
well. But as soon as you hit the brakes and you need to steer, you really then start needing to
hear the tire and the engine. So we can talk about this in the mix later because that's a big part of
trying to make sure we focus in on what's important at the time.
>>: [inaudible] a number of man hours involved [inaudible] just for the audio [inaudible]?
>> Nick Wiswell: Forza 4 we had five people working in house and three people working off
site on creating the audio for about 15 months. So each car from recording to end game we've
now got it down to a pretty fine art and we can turn a car around in about three Mondays, from
rolling up at the dyno to actually having the sound up and running in the game.
>>: Is it recommended that you play these games on a screen of this size?
>> Nick Wiswell: Always. And in 5.1 through a very good system.
>>: So you mentioned sampling the actual car. Is that true for all the cars? Because the Forza
Horizon just had a downloadable Aventador J, 2.1 million cars, only one of them, did you get to
record that one?
>> Nick Wiswell: No.
>>: Okay.
>> Nick Wiswell: But it has the same engine as the Aventador. So we record engine types more
than we record cars a lot of the time. But that's saying we will -- I mean, our library now is
800-plus cars we've recorded over the last 12 years. So we've got a fairly extensive library now
of everything. And, you know, locally there are a lot of very wealthy people who most of them
have worked at this company and a lot of them have some very, very nice cars. And fortunately
we've been allowed to record an awful lot of them. Our most expensive was a 40 1/2 million
dollar Ferrari Testarossa 250, which we were very, very careful with.
And then, yeah, we've done -- like we've recorded like a Bugatti Veyron, we've been to -- we've
been recorded at Ferrari, we've been recorded at McLaren. McLaren was a great example
because the former owner of the F1 team, Ron Dennis, it was his personal car and the engineer
pointed that out to us, he was running the car every five minutes, this is Ron's car. Be careful.
Sometimes you do have to be careful, though, because recently we recorded my studio manager's
car and broke it. The supercharger belt came off at about 8,000 RPM. And he was there. And
he was not happy. But I still have a job. We did get it fixed for him.
But, yes, occasionally we have mishaps. Fortunately they've all -- apart from even with that one,
they've all been down to mechanical problems with the cars.
>>: You haven't had a microphone get sucked into the engine?
14
>> Nick Wiswell: Had a microphone little cover get sucked through an engine once. That -- it
was just this [making noise] noise, and then this flaming foam thing came out the back of the
exhaust, and it was just like, oh, that shouldn't have happened. The car was fine, though. The
car was all right.
>>: [inaudible] the game, though, to cause the turbocharger or the supercharger's belt to break?
>> Nick Wiswell: No. But we have a great recording if it ever does.
[laughter]
>> Nick Wiswell: Yes. I do have a recording of what it sounds like to have a Ferrari 430
Challenge race car have its catalytic converter turn into a big white molten piece of metal. That
was unpleasant. That was a very scary moment. But it was a fault with the car and they'd signed
a waiver and it was nothing to do with me.
>>: [inaudible] an engine in the game and blow it?
>> Nick Wiswell: You can't blow the engine. You can damage your engine, but you can't blow
the engine. So we have a few damaged sounds that we can layer in. But not too many.
>>: What kind of a user testing do you do in terms of you were taking about perception of the
user. Do you guys bring users in for testing?
>> Nick Wiswell: Yeah, over in Building 92, Microsoft have a big user research department and
they come in and play the game and give us feedback. But at Turn 10 we employ a team of what
we call SMEs, subject matter experts, who are real car geeks and it's their job to -- we have a
team who verify the audio, we have a team who verify the car models. And their job is to go
through and verify everything for accuracy and to tell us whether or not we got it right. And then
everything has to go back to the manufacturer. So we license all these cars from the
manufacturer, and then we send off the model to them and they tell us if it's right or not and then
make us change it.
The bad news is even audio is now going through that process with some manufacturers. But
they have to sign off on that we've made the car sound correct.
>>: You talked a lot about the engines, but what about like -- can you share anything like how
the suspension noises get modeled? Because I know like as I drive around the track in my real
car, you can hear little pops and groans of everything. So is that modeled some way in the game
as well?
>> Nick Wiswell: A little bit. It's -- with the type of cars we've got in the game, it's not been a
major focus. We have a few little squeaks. We have suspension modeled, so we have all the
parameters. It's not something we've spent a lot of time on audio on yet. That's all I'm going to
say on the subject.
[laughter]
>> Nick Wiswell: Yes, it's on my list.
15
>>: Okay.
>>: Is man hours or is hardware a bigger restriction?
>> Nick Wiswell: Sorry?
>>: Is man hours, like the amount of the hours you spend working on the audio, or the hardware
[inaudible] a bigger restriction?
>> Nick Wiswell: A bit of both. In a game of this size, we've obviously got a lot to do. But I'd
say the hardware restrictions are probably a bigger one for me personally. I've got a topic on that
coming up shortly as part of how we actually mix and control some of the sounds in the game.
So we'll talk into that. But obviously it's got better. When I first started doing this, yes,
performance restrictions were somewhat terrible. Yes. [inaudible] entire game [inaudible], for
example.
Cool. Anybody else? Good.
>>: Are there any clunkers, or is it all like high-performance stuff? I mean, do you have like the
engines that have like lifter noise or some other, you know, failure mode?
>> Nick Wiswell: We've got -- we've got an original '60s VW Bug. We've got an AMC
Gremlin.
>>: Ah. Just part of the engine noise. Whatever the engine noise.
>> Nick Wiswell: Whatever the engine noise is. I think the worst sounding car I've ever
recorded is a DeLorean. Sorry to anybody who owns a DeLorean, but it really is a horrible
sounding car. It just wheezes at you instead of actually making any sort of interesting noise.
>>: So any of like the time [inaudible] timing chain like that would all be modeled as part of the
engine noise?
>> Nick Wiswell: It'll be modeled or captured as part of it. Yeah. Quite often just by using -putting the microphone in the right place you're capturing that as part of the main sound, but
that's something that we do focus on. Like if we're recording a sound, we'll actually get the guy
to drive it on the dyno, and myself or whoever's running the session, their job is to walk around
the car going, oh, that sounds quite interesting, throw a microphone here, and just sort of trying
to work out where the interesting sounds are coming from.
We'll have closed mics in on everything, but then sometimes you want to just sort of capture
what it is you're hearing on the day and where does it sound cool and then why does it sound
cool and can we put a microphone there. So we've got like little live mics, we've got contact
mics, we've got a huge box of microphones, and we're always trying to find the right mic for the
right job depending on where we are at the time.
>>: Do you have your own dyno facility somewhere?
>> Nick Wiswell: No. We just use whatever we can find wherever the car is.
16
>>: Ah.
>> Nick Wiswell: We use FAME Automotive in Redmond quite a bit over the years. We have a
facility in LA we always go to when we're down in the area. We've got another one in the Bay
Area. Two or three I've used in the U.K. over the years. Sometimes the manufacturers will even
provide their own facilities. So when you get the call saying we need you to go to Ferrari to
record their new car, you go, yes, please, where do I sign.
>>: [inaudible]
>> Nick Wiswell: Yeah. Or the best days are the days when you turn up and they say, oh, we
haven't got a driver today, could you take it to the dyno. And again you say, yes, please. So
occasionally you get to drive them, which is really nice.
>>: Do you ever have to pinch yourself?
>> Nick Wiswell: Sometimes. Some days are better than others. When you're out traveling the
world recording the finest cars available, it's the greatest job on earth. When you're sitting in an
office listening to [making noise] for two days, no, it's not. No.
>>: Equipment-wise, when you fly to Ferrari, how much do you bring with you? Is it like
suitcases big, two tractor-trailers?
>> Nick Wiswell: No, we have a flight case full of microphone stands, another flight case full of
microphones. Our gear has got smaller. We used to take sort of a MacBook with running Pro
Tools, but now we're got Sound Devices 788, so we can do everything in a really small, portable
way, which has proved really handy with some of the conditions we've had to record in
sometimes. You don't have a lot of room, you don't have power, close to the dyno. It just gives
us a lot more flexibility.
And quite often if we're travelling to, say, Italy, we'll take the microphones with us but we'll rent
all the stands and the cables and everything when we get there.
>>: Airline luggage restrictions has not been a problem?
>> Nick Wiswell: Excess baggage has been pretty expensive on occasion. But, no, we've never
had a problem getting our stuff there. Although I think it was Lamborghini who asked us were
we bringing our own dyno.
[laughter]
>> Nick Wiswell: And it was like you don't know what a dyno is, do you?
>>: PR person.
>> Nick Wiswell: Yeah.
>>: So what [inaudible] on disk?
>> Nick Wiswell: On disk or at runtime?
17
>>: [inaudible].
>> Nick Wiswell: Runtime all of this fits in 30 meg, the audio sample data. On disk including
the music it's probably 2 gig. So off the top of my head, I would say they feel about right. Car's
about a meg each, 650 of them with everything else. The music's about a gig in itself. Without
music it's about a gig. With music it's closer to 2.
>>: CPU budget for the Xbox for audio, do you get one core, half a core, or get ->> Nick Wiswell: Most of a core.
>>: Most of a core.
>> Nick Wiswell: So the core that nobody else wants.
[laughter]
>> Nick Wiswell: That one. Yes. So, yeah, we get [inaudible] the room, the 360 has three
CPUs each with two threads on it. And we get all of a thread and most of the other thread on
that CPU.
>> [inaudible] a chance to rest his voice a little bit and get some cookies and come back
afterwards. About 15 minutes.
>> Nick Wiswell: So we now know how to make cars, but how do we mix these games? So it's
just like mixing for linear media, isn't it? No. No it is.
>>: Developing a pattern here.
>> Nick Wiswell: Yes. So why is it different? Well, in linear media, everything happens the
same way every time. It's very easy. Spoke about this before. If you got a car driving down the
street and it goes around a corner, you can record the car driving down the street, going around
the corner. It's the same every time you see it. As you saw in our game, you can do anything
you like. So it's different every single time. And what you hear is carefully crafted by the mix
engineer, hopefully. So they can make sure that anything that needs to be here at that moment is
clear and precise. It's been happening that way for many years.
So with games, I've got no idea what's going to happen next, so how am I going to play back my
sounds in real time, but I also need to mix my game in real time, too, and this can cause many
problems. But there are several ways this can be achieved.
Number one, set it and hope. Just tell your sounds this is how loud you're going to be, and then
let the game do everything for you. It is an approach. It's a way you can do it.
In some games, I'll talk about this later, but in some games it's not a bad way to do things.
Number two, mixer snapshots. So in this case I've got different game scenarios defined, I know
what's going to be happening at different times and I can create a snapshot and say, okay, well, at
this time I want you to focus on this, so I want my faders to be here. But now I'm doing this so I
18
want my faders to be here. And you're just constantly adapting between different mix states at
any one time. So the game is tell me I'm doing this and my mix is adapting accordingly. So a
very simple version of sort of adaptive realtime mixing.
Number three, compressors and side chains. This is my personal favorite and how I do a lot of
the work that I do, is I will group all my sounds into a series of sub-mixes on buses and then I
will be monitoring the RMS of that bus and then using that to control the volume of other buses
via side chain.
So, for example, a great way to do this is tires. I know that when I'm starting to slide, I need to
hear my tires probably more than I need to hear my engine at that point. So if I put a side chain
between the tires and the engine, I can use the volume of the tires to turn down the engine as the
tires come up. And then as the tires go away again, my engine returns to its normal volume.
Collisions would probably be tied into both tires and engine. So tires can override engine but
collisions can override tires and engine. So we create this huge sort of hierarchy and this huge
complicated mix setup, but we can use the volume of some sounds to duck out other sounds and
clean up the mix at the appropriate time.
And finally the new way that people are headed, HDR mixing. Now, HDR stands for high
dynamic range. And the concept here, which was pioneered by EA and one of their internal
audio engines, is that every sound is given a relative volume value compared to every other
sound. So, for example, an ambience may be given 55 dB. A footstep given 60 dB. Talking
would be 80 dB. A gunshot is 150. Explosions are 180. Whatever you want to do. You play
around with those levels. And then what the game does is as these sounds play, it works out how
loud that sound is based on its natural volume and how far away it is to give you a final volume
value which it will play at in the game relative to the other sounds. So when a big gunshot goes
off or a big explosion happens, all the quiet sounds just automatically turn off. And then as that
sound stops playing, all the sounds come back up.
So it's a similar approach to compressors and side chains, it's just a different implementation.
But it's all based on real-world numbers. You can actually play around with actually plugging
real-world numbers into your sound.
So this is a new technique that's been developed. A lot more game studios are using this now.
One of the biggest pieces of middleware that's out there that we use to implement sounds in
games is a software called Wise [phonetic], and this is being added to their software very soon.
So this is something we're going to be seeing a lot more of moving forward.
So is there anything else to consider? Yes. So talked about this before, but almost all sounds in
modern games will move around relative to the player, so we need to not just be thinking about
is the sound playing but where it's playing from. Do we want to side chain individual speaker
outputs together so that we're only ducking out the -- if the tires are skidding on the right of the
car, do I want to be ducking out the engine on the left of the car or do I actually just want to be
ducking it out where it's actually coming from. So something we need to consider.
Games need to have an audio system that can deal with attenuating sounds in real times based on
distance, direction, orientation, and is there anything obstructing the sound you're hearing. If
you're playing a game like Halo and the guy who is shooting at you is shooting at you from
behind the wall over there, we need to make sure that it sounds like it's behind the wall over
there. So we talked about before about distance and direction and orientation, but obstruction is
19
something -- it's less important in racing games, because there's very rarely anything between
you and the car you're on, you're on a racetrack, but in a lot of other games, it's a big part of how
we mix the game.
So usually we'll do this with volume, panning, filtering, Doppler shift for moving sounds,
modulation effects for different sounds, all this stuff was spoken about before, but it's all
something we need to consider when mixing the game.
And the environmental audio. We talked about before. So all of these things are things that we
may want to adjust to alter the mix and how sounds interact with one another at any one time.
So which is best? Well, different types work better for different games. The game is very little
audio or it's 2D, if you're just creating a little iPhone game with just a few little interface sounds,
set and hope's fine. There's no need to do any complex mixing in that case. If sound's not a big
part of your game, there's no need to spend a lot of time trying to be too clever about doing it.
If your game is very scripted and you're generally aware of when mix changes will happen,
mixer snapshots is not a bad approach. If you're playing a very sort of linear [inaudible] shooter
where it's a case of you're in this room and things will attack you so we want the mix to be this,
but then when you leave the room nothing's going to attack you in the next section, so we want
the mix to be this. And then as you enter the next room we want the mix to change to this.
So a much more scripted approach. Mixer snapshots is a good way of doing things.
But if your game is very dynamic, like a racing game, you'd need to do side chaining or HDR
really to get the best approach, to make sure that all the sounds are always being heard when you
need to hear them.
So it depends on the type of game you're making and how much performance budget you're
prepared to allocate to it.
So what's a performance budget? I know people who don't work in games will probably want to
know what that is.
So one area where we are very different is we have restrictions placed on us in audio that are the
target capabilities, the capabilities of the target platform.
So as we generate audio in real time, we're mixing in real time, we have restrictions on how
much CPU and memory usage, and this varies greatly from system to system. The modern
consoles have fixed amounts of CPU and memory. PCs have very variable amounts of CPU and
memory and you need to take account of.
When I started out we were making games on the PS1, and that had 512K of memory and a tiny
amount of CPU and a maximum amount of 24 voices that you could play simultaneously. That
was not a fun time to be doing audio for games. But it was a very creative time. Some of the
stuff that I do now I probably don't think about the way I did then because before it was a case of
how the hell am I going to do that. Well, now it's a case of, oh, I can do that. That doesn't even
enter into my head now of how am I going to do it. And some of the most creative stuff I've
probably come up with is at a time when there was no way I could do what I needed to do with
what I had. So you had to be very, very clever about the ways you did it.
20
So in some ways, even though we've got far more available tools, we're probably not as creative
as we used to be or we're trying to be creative in different ways.
>>: [inaudible]
>> Nick Wiswell: I suppose it's much like trying to mix a record in the '60s versus now. If
you're trying to do it on four-track tape, it's a whole different world to what you've got available
to you now, but people still made great stuff.
So it's all here. Voice counts improved, realtime DSPs become available. Up until the Xbox
was released there was very, very little realtime DSP apart from low-pass filtering ever done, and
that sort of opened up a far bigger world to us with realtime DSP as that actually have DSP chip
that you could program in the console, which was just like, oh. That was the best day ever.
So that's it then? A mixed game?
>>: No.
>> Nick Wiswell: No.
[laughter]
>> Nick Wiswell: So this is an area where I bang on a lot and I annoy a lot of people. So if I
annoy anybody today, tough. I don't care.
But games, like a lot of forms of media, can be played on so many different devices. Cell phone
speakers, flat-screen TVs, high-end home theater systems. How are we going to make the game
sound as good as it can everywhere it's going to be played?
So for me it's focusing on, well, how can I make a mix that works on these playback systems as
possible focusing on the most common playback scenario.
So many people, some in this room maybe, will tell you that the games industry will say, oh, 5.1,
everybody's got 5.1 these days. I am not one of them. I still firmly believe that the majority of
the people playing games these days are playing games on TVs. Bad news, folks. And it's TVs
with really small speakers.
>>: [inaudible] or in the music world I deal with laptops and iTunes.
>> Nick Wiswell: Yes.
>>: [inaudible]
>> Nick Wiswell: And here's more science for you. The laws of physics say your 100-hertz
bass thump is not going to work on a flat-screen TV. I'm sorry, folks. It just isn't. But I believe
there is a way to maximize the sonic qualities mixed for every playback system, and especially
for games.
When I make a sound or when my team are making a sound, I always get them to verify it on a
21
low-end system. Roll off the extreme lows, test it out to make sure we've still got the low mid
punch that you need to cut through on TV speakers. And you've got your sound, but it's going to
be lacking low end for those with a high-end system. So then put it back in with a dedicated
LFE sample.
The people who are going to hear it in 5.1 are going to have an LFE. The LFE is there to be
used. So why not have your low end in the low end in the LFE where it belongs and try and
keep all the stuff that's below a hundred hertz out of your main speakers. It doesn't need to be
there for the majority of people who play games.
To do this we have Avantone MixCubes. Some of you may know of these speakers, some may
not. It's a little speaker about this big, the four-inch driver. It sounds like shit. But it's designed
that way. And the best thing is it's a universal shit so it doesn't matter where you put them, they
always sound bad.
[laughter]
>> Nick Wiswell: But it allows me ->>: [inaudible]
>> Nick Wiswell: Yes. But it allows me to have them. We have them in our demo room where
our management review the game, we have them in our mix rooms, and it's a constant
verification. And what we found is that if it sounds good on those, it will sound good on just
about anything. I'm sure the music industry thought of NS-10s the same way.
>>: [inaudible]
>> Nick Wiswell: So there are many speakers out there that do this. But every sound you make,
test it. If you come up with the greatest sound in the world, you plug it into your game, you take
it into the management reviewing room, they put it on the TV and go "What was that," you
haven't done it right. I'm sorry. I'm a firm believer that make your sound sound good for the
most people who are going to play your game. Make it sound good on a TV. Sorry. I'll calm
down now. Deep breaths. Deep breaths.
So what does that sound like? Well, I have a demo. I'm going to play you the last game I made
before I left the U.K., which was Blur. This is a racing game developed for Activision. It's a
racing game with power-ups. So we have real cars. We're still using the same techniques to
make the cars. But this time we've got loads of other worldly weapons shooting shit at you from
every direction.
So if this works. It should have booted up.
[demo playing]
>> Nick Wiswell: Okay. So I made this game. Up until yesterday I didn't own this game, so I
have not unlocked anything yet. I managed to leave before the free copies were given out. So I
had to go out and buy one yesterday because I thought this was the perfect example of demoing
this, and then I realized I didn't actually own it. So apologies for not being able to show you
something [inaudible] in the game. But this is a great example of what I want to do.
22
So here I'm going to race around. I'm boy on Route 66. It's an off-road track. You're going to
hear lots of dirt sounds. There's 20 cars on the track all firing stuff at me, pickups, Mario karts
the whole way. But the big thing you're going to hear is how we dynamically mix this game.
You'll hear that every time we fire a weapon, we're playing around with the mix, we're ducking
things in and out. Tires are affecting the engines. There's a huge amount of [inaudible] work
going in here.
[demo playing]
>> Nick Wiswell: Pretty crazy stuff.
>>: That is crazy. How did you do the sound when the car got airborne and then hit the ground?
>> Nick Wiswell: We took a car, put it on a crane, lifted it up, dropped it, recorded it.
>>: Cool.
[laughter]
>>: Nice. I bet that was fun.
>>: Was that a simulation of the M25?
>> Nick Wiswell: Yeah. So interesting to note there, it's one of the games I'm proudest of from
a mix perspective. There's 20 cars driving around on the track. How many of them were ever
audible? Two.
>>: Four?
>> Nick Wiswell: Yours and one other. And we discovered that's all we ever needed because
the mix was just too full if we put in more and it was all getting in the way. And for me it's all -this game was all about -- with the mix it was all about subtraction. It was all about here's a big
sound, let's make everything else go away.
Now, some of the weapon effects that you heard, as you were getting hit by them, we wanted to
do something cool and interesting. So when you get hit by certain effects, we actually have like
a realtime bit crusher, we have a realtime ring modulator, we have realtime flanges and phasers
and various filters, and they're all being applied either to your car as it hits you or to the other car
that it hits as it hits them. So you get all these really weird, crazy effects. But all that came
together to deliver this.
But one of the coolest things, and I may be able to do it here, there's one coming up -- I'll pause it
because it's too noisy -- is the shield. Because we made the shield. The shield covers your car.
And we thought, mmm, well, if it's this thing that stops things hitting you, it should probably
stop sound getting through it. So when the shield's applied to your car, your car is also muted
and distant inside this bubble. But then if you change the camera view, so you're inside the
bubble, all the outside sounds get filtered. So it was a really crazy way of sort of playing around
with that.
23
So that whole effect of this shield is actually stopping sound, it's not just stopping all these
powers. So I'll see if I can make it happen. There's one just down here.
[demo playing]
>> Nick Wiswell: My car is very broken. If I apply it, it makes my car go away. But when I'm
inside it, I can hear my car. And it's just the little things like that that made it all work. Back to
full health again.
So huge amount of mix work went into this game. We spent -- we were lucky enough to get
about four weeks to mix, which is unheard of in the mixing of games. Usually you get a week if
you're lucky. So it made a big difference. Question?
>>: Just from -- just what I was hearing, am I correct in hearing that you guys added the sound
like the -- like rocks hitting inside of fenders and things?
>> Nick Wiswell: Yeah.
>>: Awesome.
>> Nick Wiswell: That is one of the most complicated systems I've ever built and one of the
most fun things I've ever done. Because we're driving on gravel and we went down to Anaheim
to the Orange County Speedway, and we had a drift driver for the day to record tire sounds. And
the idea was we were going to record it dry and then we were going to get a water truck out and
hose the whole thing down and then record it all wet. And then we were like -- we broke for
lunch and we're sitting down, I'm like, what's that over there? Walked over, it's this huge gravel
parking lot, and it's like get over there now.
So we got this drift guy driving around on this gravel. We broke four microphones, we smashed
up his car. I didn't care. It sounded great.
But when we got back, what we noticed was the recordings were great and we had all this
kick-up and all this noise, but it was missing the definition. It just didn't sound clear enough. So
it's like what we need is the sound of all the stones hitting it. And we thought, well, if we build a
system out of just single ones, just little dinks, and then lots of them all the time and put them on
a sort of randomization where we're playing different samples and different sequence with slight
different variation in them, that should sound really cool.
So one of the guys at work had just been in a big car crash, so we asked if we could have all the
parts of his car that they were replacing, so we got a fender, we got a door, we got a hood, they
were all bent, and we took them out of the back of the building we were working at at the time,
which was pretty quiet, we're in this sort of business park, and we bought a bag of stones from
the local garden center and then spent the day throwing stones at old car parts recording all these
little individual dinks.
And then it was just like all right, we can drive on mud as well, so we should get sort of mud
sounds. So we're in the dirt making these balls, throwing them at the car all day. That didn't
really sound right, though. We wanted something better. So we then went out and got a big sort
of kitchen roll, soaked it in water, and then we're throwing these little sort of wet paper balls at
this door for like two hours. Great fun. Great fun.
24
But what we did then is we cut them up into other individual samples and then we thought, well,
we've got these light sort of gravelly stones and then we've got these heavier sort of sounds, these
dull sounds, and then we've got these wet squishy sounds, and this game has a lot of off-road
surfaces to drive on, so we said, okay, well, gravel is mostly small stones with some big stones.
But then dirt is mostly big stones with a bit of wet and a bit of small. And mud is all the wet
stuff with a little bit of sort of the heavy stuff. And all we did is we varied the amount of each
that played and how often as you went across each of the surfaces.
So it was going from -- you have these underlying sounds underneath, but you're getting all these
little dinks. And that's what created that sound, and that was a lot of fun to do. Used a lot of
perf, though. My programmer was not pleased. But we made it work, and I'm pretty happy with
the result.
>>: Did you limit the amount of rocks that were playing at once? Like how many channels were
you using?
>> Nick Wiswell: Five per tire. Per stone type.
>>: Right [inaudible].
>> Nick Wiswell: So, yeah, it was a maximum of -- if I remember right, you could have -- the
worst-case scenario was dirt, and that could have up to five heavy stones with two light and two
squishy per tire, so that would be nine per tire, so 28 maximum just for that. And then the loops
underneath and then the cars [inaudible] everything else.
Peak voices on that was -- we were using a proprietary engine, which was very good. I think
peak voices on that was 225 per frame. For the whole thing. Which is pretty good going. Wish
I had that now.
So let's go back to laptop.
>>: [inaudible] 225 channels, is that [inaudible] or MP3 or [inaudible].
>> Nick Wiswell: That was XMA on 360 and combination of MP3 and ADPCM on PS3 and
PCM on PC. Because that was a multiformat release.
Questions?
>>: The Doppler for Forza ->> Nick Wiswell: Yeah.
>>: -- was that -- did you use FMOD to do the Doppler, or did you like vary RPM and kind of
like overdrive ->> Nick Wiswell: We feed the Doppler value as a pitch value. The game calculates the Doppler
and then feeds it in as a pitch value into FMOD.
>>: So it's taking the mix and then changing the pitch for that. You're not --
25
>> Nick Wiswell: We're applying it per object. So one thing we did have to do with that,
because it created -- something that's probably real but sounded completely wrong is because the
car's built from the different components and those components are ever so slightly offset from
one another within the car model, you got some weird effects that as the car went by you, the
engine would Doppler shift slightly differently to the exhaust, and that sounded completely
broken. So we did make it so that it's Doppler shift per car. And it's not per emitter, it's one
Doppler shift, but it's applied to all the emitters separately.
>>: Hardware budget for Blur versus Forza for audio? About the same?
>> Nick Wiswell: One core, 22 meg sample memory.
>>: And how well did that compare to when you move to a different platform? PS3 has better
audio, less audio, you need more?
>> Nick Wiswell: Less. Less. We needed more CPU to do the same amount of work. And we
had the same amount of memory, but because MP3 was expensive, we could only do a certain
number of MP3 sounds. So we had to do 80 PCM, which gives a big memory hit. It's less
compression.
>>: And is that from because you have to do two more channels on PS3 [inaudible]?
>> Nick Wiswell: No, that's because there is no -- the Xbox has built-in perceptual codec in
hardware and the PS3 doesn't. So just different formats.
>>: When you're recording on a dyno like inside, do you have to dampen the room at all, or are
the mics close enough ->> Nick Wiswell: We try ->>: -- to get that [making noise], that kind of ->> Nick Wiswell: Yeah. We try and dampen the room where possible, especially if we're using
different distant mics. If you ever get the opportunity to do so, in the U.K. the Motor Industry
Research Association has a dyno in a semi-anechoic chamber, and it's brilliant. And it fits a
double-decker bus in it. It's a huge semi-anechoic chamber with a dyno in the middle.
But don't stand in there to monitor your levels when you're recording a Porsche Boxster because
you start feeling ill and then you say "What's that like," and they say, "Oh, that's the carbon
monoxide warning light; you should probably go outside." That was -- that wasn't a fun day.
>>: Since they probably did not build that facility to make games, why did they have
[inaudible]?
>> Nick Wiswell: Because they use it -- the car manufacturers take new cars there to listen for
unwanted rattles. So they can drive the car in normal conditions and listen for any unwanted
noise and then try and isolate where that noise comes from. An amazing facility, but they -- like
they have these little binaural mics that they can put inside the car to replicate the driver's
position, so it's inside a head model. And then they have all these little mics down the side.
26
They even have rows of them down the room where they can simulate drive-bys by triggering
the mics in sequence.
>>: [inaudible]
>> Nick Wiswell: Sort of, because the mic sort of works the same way. But they have a whole
division who then monitor that information, and then they have a full analysis of how sound
travels through all the materials that are normally used in cars. So they'll say, ah, yeah, you've
got this crazy rattle at 422 hertz. That's probably this plastic that you're using in your dash and if
you replace that plastic with this plastic, that noise goes away.
So that's the type of thing they do. But obviously a lot of standard cars don't go through that.
But people like Aston Martin and Bentley all take their cars there because obviously they want
the very refined experience. It's a high-quality product. And the last thing you want is a crazy
squeaking noise coming from somewhere it shouldn't if you're spending $100,000 on a car.
>>: Is that why they don't vent the exhaust, because it's part of the sound signature?
>> Nick Wiswell: They do have the option to vent the exhaust, but we didn't want the fans on
because they were affecting the microphones. So bad call on my part.
>>: Better than you passed out, right?
>> Nick Wiswell: Yes. But I did mention in my opening that it can be a dangerous world. An
interesting story there is if the dyno operator tells you it's okay to put a four-wheel-drive car in a
two-wheel-drive dyno because he knows how to disable the front wheels, he's lying. As we were
trying to record a Porsche 911 Turbo once and we only had a two-wheel-drive facility available
to us, and we put chocks under the front wheels just in case, what happens is if you're doing 140
miles an hour and then the front wheels start to spin, it sucks the chock under the wheel and out
the other side at a very high speed. And if that happens to hit you, I can tell you it bloody hurts.
And, yes, I have the broken bones to prove it.
>>: [inaudible] you survived.
>> Nick Wiswell: It hit me on the foot.
>>: Okay.
>> Nick Wiswell: And the next thing I knew I was sideways in the air and then into the floor.
And not a fun day.
>>: But you learned from it. What'd the dyno guy say after that? Oops?
>> Nick Wiswell: Oops. Well, they were more concerned because as the wheels spun up it
actually dug a divot into their floor. They're all "Look at my floor"; I'm "Look at my foot."
[laughter]
>> Nick Wiswell: So sorry. As I prefer to call this stage, let's have a heated debate.
27
>>: How about a sample race [inaudible]? You any concerns or any -- I mean, you can do
whatever you want, but ->> Nick Wiswell: We always record at the highest we can. So if we record in a car, up until
about three years ago everything was done at 4824. And now we can record at 9624. When it
comes to actually putting them in the game, it's all about how do you make it fit in that memory
budget. So we try and keep everything as high quality as possible. Most content is 48. Some
content we drop to 36. Some we'll drop to 24. We've just got to make it fit in the memory we've
got.
But like we create a lot of the dedicated LFE sounds that we were talking about before.
Obviously we can have those at really low sample rates because they're only coming out the LFE
anyway, so it's all about playing around with finding the balance between quality and sample
size.
>>: When you have all the different sampling rates, how do you -- what do you do at the mix
point?
>> Nick Wiswell: The game engine deals with it automatically. It will just pitch correct it back
to the native 48.
>>: [inaudible] resampling somewhere in there.
>> Nick Wiswell: Yes. That's where all that CPU budget goes. Well, the thing is in a lot of
games where we're just playing sort of fixed one-shot sounds, you don't need to pitch shift many
of them.
But in a game like [inaudible] we're applying Doppler shift to virtually everything anyway.
There's already a resampler running on that sound to apply that, so we can just use that same
resampler. So put it back to 48 where it needs to be.
>>: On Forza were you mostly feeding between loops for your engine sounds, or are you doing
synthesis or a little of both?
>> Nick Wiswell: That's confidential. We've used many techniques over the years, but a bit of
both is probably the closest I'm going to reveal at this point. Don't want to give too much away.
>>: When you're cutting your loops, do you do it unphased so when you blend or cross fade
between them you have no more phasing?
>> Nick Wiswell: Yes. All the loops that we cut are loop aligned to the same cylinder fire. So
when you're looking at a waveform of a [inaudible] and you can actually count the cylinders
because it always happens in patterns. So you'll be able to see it's a V8 and you'll notice that
there are eight peaks in a pattern, and then the same repeating eight peaks in a pattern. So we'll
always make sure it's on the same cylinder fire and all those sounds will be kicked off at the
same time. So theoretically it should always stay in sync as you go through the rev range. That's
a fun system to build.
>>: Does that also apply to the rotaries?
28
>> Nick Wiswell: Yes. Because they still have a firing order. It's just only two, three, or four.
So it's less prominent, but, yes, they do still have an obvious -- if you look at the waveforms, you
can clearly see how the engine works.
>>: If it's a rotor, I guess three chambers, if you can hear each chamber ->> Nick Wiswell: Yeah.
>>: Okay. So two rotor, it has six peaks you can pick out.
>> Nick Wiswell: No, it has two peaks.
>>: Two peaks.
>> Nick Wiswell: A two-rotor has two peaks.
>>: Okay. So the three chambers within that rotor don't sound different?
>> Nick Wiswell: No.
>>: Okay.
>> Nick Wiswell: No. You can clearly see that it will be a two, four, and eight. You can also
sometimes see, which you can't see with the rotaries, is, like I said, the engine's got to hold -- the
crankshaft has to revolve twice to -- all pistons have to fire twice for one full rotation, and you
can actually see the subtle variation between the two as the pistons are actually ever so slight -the overall sound is very slightly different because of the order of the firing and the way the
crankshaft rotates. So you can actually see that it's not actually a pattern of eight, it's a pattern of
16 with two very similar eight parts in it.
>>: [inaudible] games, racing games [inaudible].
[laughter]
>> Nick Wiswell: Every racing games have something about it, even the ones that aren't
personally my taste. You know, even going back to Pole Position and its hideously synthesized
engine loops, but it was trying to replicate that sound of an engine revving up and down.
So big steps were taken when we finally got to use samples properly as we sort of went into the
32-bit era. Most of the 16-bit games were synthesized. But with 32-bit we actually got to start
using samples for the first time, so that was a big step.
The Need for Speed: Underground I believe was the first game that used granular synthesis for
the first time, so that was probably a big step in evolution.
Don't want to blow my own trumpet, but I will anyway. PGR was the first game to properly mix
racing game in 5.1 and work with that. So there was a lot of things -- there were a lot of
benchmarks over the years. But every game bring its own character. You know, I've worked on
both of these games with a very, very different sounding games. The aesthetic is as important
sometimes as the techniques that you use to build them.
29
Anything else?
>>: [inaudible] question. So for the Xbox setup it knows how many speakers that you have, but
it doesn't know anything about the quality. Can you use something like a Kinect to listen to the
living room and say, hey, these speakers are pretty good, I can do this stuff.
>> Nick Wiswell: Would be lovely if you could, but no. But we do have options in a lot of
these games now. Forza and Blur both have them. Actually, I can do this. This -- this you'll
like. This is audio geekery at its finest. Is it going to work. So if I go -- can I -- so in this game
I have audio options, sound effects, music, menu music, license soundtrack, on and off.
Dynamic range. Home theater or TV. So we can play around with the dynamic range available.
Your output, stereo or surround. And then you go into the interesting bit. Can now play around
with your room size.
Now, this is because of a system we developed whereby when you're mixing in 5.1, as an object
travels through the room, you don't want it to go -- get loud, loud, loud, loud, loud, full volume
in this speaker, then loud in all the speakers and then away again. How long does that bit where
it actually travels through the room actually take.
Don't know why we developed this because it's far too complicated for a racing game, but
anyway. I know why we did this. Scratch that. After we'd made Project Gotham Racing 2, I
was asked to speak at a Dolby conference in London because it was -- we were really pushing
5.1 hard. And it was done at BAFTA in the U.K. in their theater, in the cinema. And we listened
to the game and I'm like, Listen to this. And the car went across around the room, and it sort of
went [making noise] and it just missed this big chunk in the middle because it was mixed for -almost as though all the speakers were together. And it sort of didn't really travel across the big
theater. So next time we demoed it a few years later we'd added auditorium as a room size so it
didn't do that. That's why we did it.
So you can play around. And all this does is it alters how -- the distance. If I remember
correctly, it was only -- it was like 1 1/2 meter room, which was the size of my small mix room;
medium, which was about a sort of 3-meter-square room; large was an 8-meter-square room; and
then auditorium is like 25 meters between the speakers.
>>: But a user using this game doesn't know that. You're narrating for us right now.
>> Nick Wiswell: But I'm just -- but it basically means that if you're in a -- if you live in Europe,
you have a small house. If you live in America, you have a big house. And it allowed us to play
around with it for different people. If you happened to present in a cinema, you have a mode that
means it doesn't sound broken.
Music mode. You can have your music in stereo, just the front two speakers, quad, duplicated
front and back, or actually mixed in 5.1, if you wanted to.
LFE boost. Some people like a lot of LFE. So with this what we were actually doing is we were
increasing -- we weren't sending any of the game sound to the LFE by default. So any low end
that happened to be in there wasn't ever getting sent to the LFE speakers unless the amp was
doing it for you. What this would do, it would override that and it would do it for you anyway.
So you'd get even more sober. And it would turn up the LFE samples a little bit.
30
Now we're get into ultra geekery, where you can alter the angle of your speakers and it adjusts
the panning angle in real time to better match the quality of your room.
>>: That's cool.
>> Nick Wiswell: This is possibly the most pointless thing I've ever done, but it's really, really
cool.
>>: Very cool.
>> Nick Wiswell: So all it would do is the algorithm that worked out were a sound was relative
to use, so I could play around with them, so I could move -- or I can play around with the
speakers, these are just the angles, and let me see if I can get it right now. And then I could
actually do the offset as well. So if your room's not quite square, you can play around with them
individually.
So, yes, full on audio geekery there. Yes. I'm not going to break it completely. Thank you. So
that's as far as I was prepared to go. Yes.
>>: So when you made the adjustments for the different rooms, are you just feeding parameters
into the mix engine?
>> Nick Wiswell: Yes. So it would know at what point it should start adjusting -- as a sound
approached you, at what point it would reach that sort of point where it's at maximum volume in
these speakers and then it starts to bleed across and then away again. So it was just in the mix
engine in real time.
And the same thing with the speaker angles. It would just adjust that and compensate the mix to
make sure -- if you actually spent time and tuned it and got it exactly right, it should mean that
you were getting the exact same experience I was getting in my mix room which had speakers in
perfect angles, because nobody can do that in a house. Yes. Any more?
>>: So I'm curious. You mentioned that the Xbox -- I believe you said has a DSP chip in it, and
I wondering the hardware optimizations for that, are those exposed to you, or is that something
that is just taken care of by the mix engine, as you say, or ->> Nick Wiswell: The hardware DSP?
>>: Yeah. There's like hardware optimizations like SIMD and all that jazz. Is that something
that's exposed that you have to noodle with, or is that kind of just taken care of?
>> Nick Wiswell: They were all custom effects that were written, so our coders had to deal with
all that. And I just told them it didn't sound very good and to do it again until I was happy with
the end result.
>>: Yeah, I was curious.
>> Nick Wiswell: There was quite a bit of that to worry about with that. But with the 360 -- the
old Xbox had the DSP chip. The 360, it's all the new software. So we have a huge suite of
31
software effects running.
Any more? All right. I have one more thing you may enjoy.
So somebody asked before how we do collision recordings and how we do car drops. Well, back
in 2004 I had a much smaller stomach and a lot more hair, but apart from that we also went to a
scrap yard, a sort of junkyard, we had a lot of cars, and we had a big hammer. And one of the
guys who went with us was just one of ours. So it wasn't an audio team member, he just came
along for the day to film it. And he put together a little video that I would like to share with you
now.
>>: This is actually up on YouTube [inaudible]?
>> Nick Wiswell: Yes. Come on, Internet. Come on. It worked before. Boom. There we go.
[video playing]
>>: Nice.
>> Nick Wiswell: So ->>: That's the title [inaudible] is our creations ->> Nick Wiswell: Audio trip 2004. And one of those hits where he went into the light and all
the stuff sprayed, all of the glass, I've still got the scars to prove where that came through from
the busted lip I got that day. So another dangerous point. We all had safety glasses. Health and
safety first, everybody. But it still hurts.
>>: [inaudible] ear protection when you're doing these 140 dB recordings?
>> Nick Wiswell: Oh, yeah. Sorry, what was that?
[laughter]
>> Nick Wiswell: Yes.
>>: [inaudible]
>> Nick Wiswell: Most times we'll have ear defenders, big industrial ones. For cars like 787B
we have ear plugs then ear defenders. And it still hurt. It hurt a lot. The backfire was my
favorite on that one. The first time it backfired we were using a type of dyno called a Dynapac
where it's not a roller in the ground, you actually bolt the car directly -- the hubs directly into
these boxes and the wheels drive. So you've got no roller noises. Great. But the exhaust pointed
straight onto one of these hubs. So we just got this huge blast of hot air going onto this thing that
was controlling how the car drove, and it's like we can't have that.
So we got this huge sheet of metal, it took two of us to lift it, it was like this big and this tall and
this thick, and we put it down, it backfired and the thing jumped four feet in the air.
[laughter]
32
>> Nick Wiswell: Scared us all to death, especially when there's a $2 million car next to it that it
missed by about this much.
>>: One of a kind.
>> Nick Wiswell: Yes. So any more questions? Thank you very much.
[applause]