UC Santa Cruz
Computer Science – Game Design
Sound in games
• Think about truly memorable games
– They almost always have excellent background
music and sound effects
– Legend of Zelda, PacMan, Katamari Damacy, Little
Big Planet, Radiant Silvergun
– Music and artwork style combine to create an
overall tone, or mood, for a game
– Done well, this substantially enhances the overall
gameplay experience
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Finding/Making Sounds
• Where can you find music to use in your game?
– Reminder: there is this legal framework called Copyright Law
– Creative Commons: use licenses that may allow free, non-commercial use
• http://creativecommons.org/
• http://archive.org
• Sites with Creative Commons licensed music
– New Grounds, Jamendo
• http://www.newgrounds.com/audio/
• http://www.jamendo.com/en/creativecommons/
• Look for “Attribution, Non-commercial”
– “No Derivative Works” is OK, so long as you don’t modify
• If you use in your game, make sure you provide attribution
– Put name of artist in your game (About page, splash screen, etc.)
– Is polite to send them an email telling them about the use—will make
them jazzed
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Finding/Making Sounds (cont’d)
• Find someone to create music for you
– Music student at UCSC, for example
– One team has already put up announcements for a music
person
• It has never been cheaper to create high quality music
– Instruments, microphones, mixing technology are all at
historically low prices
– Has led to a proliferation of music
– Biggest problem: finding an audience
– Games provide a good audience
– Sales of many videogames larger than most music album
sales
– For many musicians, might have larger audience for video
game soundtrack than for traditional album
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Finding/Making Sounds (cont’d)
• Use your voice!
– Your voice is wonderfully adaptable and expressive
• Consider:
–
–
–
–
Record a raw voice clip
Bring into an editing software suite
Tweak/filter/alter until it suits your game
Can do much worse…
• Tools
– Audacity
• http://audacity.sourceforge.net/
• Free, open source sound recorder/editor
– FL Studio (grown-up commercial version of Fruity Loops)
• http://flstudio.image-line.com/documents/what.html
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Playing Sounds in XNA
• Two ways
• Hard (but powerful) way
– XACT audio tool
• Cross-platform audio creation tool
– Many neat features
– Edit volume, pitch, looping of sound clips
– Can easily group together sound clips
• Easy (and 95% sufficient) way
– Use Simplified Sound API
– Can start, stop, and pause sound playing
– Much, much easier to use
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Simple Sound API
• Two ways to play music
– As a song
• Good for background music, or other long sounds
– As a sound effect
• Good for short duration sounds
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
XNA Simple Sound API
• Supported music types: wav, wma, mp3
• Add sound into project Contents folder
– Audio files treated like other files in content pipeline
– Copy sound file into project Contents folder
– Right-click on Contents folder inside Visual Studio C#
Express
• Add  Existing Item … select audio file you just copied in
– Will now be visible inside Visual Studio
– Need to double-check the Content Processor
• Sound Effect – XNA Framework – sound effects
• Song – XNA Framework - songs
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
XNA Song API
• Create a variable of type Song
– Used to load songs via the content pipeline
– Song mySong;
• Load sound file
– mySong = Content.Load<Song>(@”{name of song file
without extension}”)
• To play a sound, call Play() method on MediaPlayer object
– MediaPlayer.Play(mySong);
• To pause/resume, call Pause()/Resume() on MediaPlayer
object
– MediaPlayer.Pause(); // no argument
– MediaPlayer.Resume(); // no argument
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
XNA Sound Effect API
• Create a variable of type SoundEffect
– Used to load sounds via the content pipeline
– SoundEffect soundEffect;
• Load sound file
– soundEffect = Content.Load<SoundEffect>(@”{name of
sound file without extension}”)
• To play a sound, call Play() method on
SoundEffect object
– Returns a soundEffectInstance object
– Can use this to stop, pause, and restart sound
– SoundEffectInstance soundEffectInstance =
soundEffect.Play();
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Demo of Song and Sound Effect API
// Demo of use of Songs and Sound Effects
inside XNA
• Caution: Treating a song as a sound effect can
lead to very long compile times
– Solution: keep sound effects short
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
3D in XNA
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
2D to 3D
• Many current games use 3D graphics
– Much more complex than 2D graphics
– This course will provide a basic overview of 3D
graphics
– CMPS 160, 161, 164 provide greater depth
Ratchet and Clank
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
3D Camera
• Analogy
– 2D is like putting stickers on a page
• Place sticker (sprite) at x,y coordinate
• If a sticker is placed at 50,50, you see it
– 3D is like recording a video with a camera
• What is recorded (shown on screen) is what
camera sees
• Can have objects in a scene that aren’t visible
– Can have 3D object at 50,50,50, but if
camera is pointing in the opposite direction, won’t see it!
• Introduces rotation
– Camera can potentially be
rotated around all 3 axes
– Objects can also be rotated
around 3 axes
– Affects what shows up on screen
Adapted from Jim Whitehead’s slides
UC Santa Cruz
•
•
Computer Science – Game Design
3D Coordinate System
3D graphics requires use of x,y,z coordinates
So, which direction is positive z?
– Is it back away from you, or towards you?
– Either choice would work, need to pick one
•
Right handed vs left handed coordinate systems
– XNA uses right handed coordinate system
•
Place hands, palms up
– Point fingers in direction of positive X
– Curl fingers in direction of positive Y
– Thumb is pointing in direction of positive Z
Right-handed coordinate
system
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Camera
• Camera is comprised of two Matrix objects
– View matrix holds information on
• Location of camera in world
• Camera direction
• Camera orientation
– Projection matrix holds
information on
• View angle
• Aspect ratio
• Near and far plane
Direction
Location (x,y,z)
Orientation
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Matrix Structure
• XNA provides a Matrix structure
– A 4x4 matrix, in row vector layout
• Row vector matrices view vectors as a row from left to right
• column vector matrices view vectors as a column from top
to bottom
– Built-in matrix operations
• +, -, *, /, ==
– Also, convenience matrices
• Identity, Up, Down, Left, Right
– Large number of convenience methods
• Rotations, views into 3D world, determinants, invert
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Vector3 Structure
• Represents either:
(x,y,z) coordinate
– An X, Y, Z coordinate, or,
– Distances along X, Y, Z coordinates
(e.g., a vector)
y
(x,y,z) vector
• Often a unit vector
– all values between 0 and 1
– X, Y, Z properties (floats)
• Built-in operators
x
z
– +, -, *, /, ==, !=
• Convenience vectors
– UnitX, UnitY, UnitZ, Up, Down, Left, Right
• Many convenience methods
– Interpolations, rotations, distance, dot product, normalization
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Creating an XNA Camera – View Matrix
• View matrix
– Use CreateLookAt method of Matrix structure
– Parameters (all Vector3)
• cameraPosition – location of camera (x,y,z)
• cameraTarget – coordinate of point where camera is looking
• cameraUpVector – vector indicating up position
cameraUpVector
cameraTarget (x,y,z)
cameraPosition (x,y,z)
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Creating an XNA Camera – Projection Matrix
• Projection Matrix
– Use CreatePerspectiveFieldOfView method
– Parameters (all floats)
• fieldOfView – angle of camera view, in radians
– Typically 45degrees – pi/2 radians
• aspectRatio
– Typically width of screen divided by height of
screen
• nearPlaneDistance
– Distance from camera to near viewing plane
– Objects between camera and near plane are
not shown!
• farPlaneDistance
– Distance from camera to far viewing plane
– Objects beyond far plane are not shown!
cameraPosition (x,y,z)
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Drawing Triangles
• All complex 3D shapes seen in games are composed of a
series of triangles
– A triangle has 3 points, one for each corner
• Points are more typically known as verticies
• Minimum number of points to unambiguously define a plane
• VertexPositionColor object
– Represents the x,y,z location of a vertex
– Also has a color for the vertex
– VertexPositionColor v = new VertexPositionColor(new Vector3(0,1,0), Color.Blue);
– Need 3 verticies to draw a triangle
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Vertex Declaration
• XNA requires you to tell the graphics device what
kind of vertex data you will be using
– Unclear why XNA can’t just figure this out, or handle
multiple types seamlessly
– Probably due to structure of DirectX API, or
capabilities of graphics hardware
– For now, treat as a must-do, black box
– Put following in your main, derived from Game class
– GraphicsDevice.VertextDeclaration = new
VertexDeclaration(GrahpicsDevice,
VertexPositionColor.VertexElements);
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Actually drawing the triangles
• In XNA, all 3D rendering is handled by a shader
– Shaders defined using High Level Shader Language
(HLSL)
– Permits creation of wide range of visual effects
– More on shaders in a few classes
• XNA provides a default shader
– Called BasicEffect
– Will use this for now
• BasicEffect is a type of effect
– Effects contain a series of EffectPass
– Each pass handles some aspect of putting things on
screen
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Using Basic shader
•
•
•
•
Five steps
Create Shader
Copy over camera information
Iterate through EffectPasses
• Examine source code from example in Chapter
9 of XNA 3.0
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Using Basic shader
Five steps:
1. Create Shader
–
BasicEffect effect = new BasicEffect(GraphicsDevice, null);
2. Copy over camera information
–
–
effect.View = camera.view;
effect.Projection = camera.projection;
3. Set world matrix
–
effect.World = …
(more on this in a few slides)
4. Enable vertex capabilities (varies by Vertex type)
–
–
Effect.VertexColorEnabled = true; // for VertexPositionColor
Effect.Texture = myTexture; // for VertexPositionTexture
Effect.TextureEnabled = true;
5. Iterate through EffectPasses
–
Call to DrawUserPrimitives inside EffectPass puts triangles on screen
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Iterating Through Effect Passes
• Each Effect has calls to begin()/end()
• Effects are comprised of passes
– Each pass requires a call to begin()/end()
effect.Begin();
foreach (EffectPass pass in effect.CurrentTechnique.Passes)
{
pass.Begin();
GraphicsDevice.DrawUserPrimitives<VertexPositionColor>
Passes
(PrimitiveType.TriangleStrip, verts, 0, 1);
pass.End();
Draws verticies
}
effect.End();
# of triangles (the “primitive
Index into verts
array
shape” in this context) to
draw
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Triangle Drawing
• Examine this line of code
– GraphicsDevice.DrawUserPrimitives<VertexPositionColor>
(PrimitiveType.TriangleStrip, verts, 0, 1);
– What is a TriangleStrip?
• Three ways to draw triangles
– Triangle List
• Each set of three verticies defines a triangle
• Memory inefficient, since triangles often
share edges in complex 3D meshes
– Triangle Strip
• Builds triangle out of first three verticies
• Each additional vertex creates new triangle
using that vertex, plus previous two verticies
– Triangle Fan
• Each additional vertex creates new triable
using that vertex, the previous vertex, plus
the first vertex
http://escience.anu.edu.au/lecture/cg/surfaceModeling/image/
surfaceModeling015.png
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
World Matrix
• Each triangle has 3 verticies, and each vertex has an x,y,z
position
– This position is given with respect to an origin location
– That is, location is with respect to a local coordinate system
• World matrix
– Translates from the local coordinate
system to the world (i.e., visible
on screen) coordinate system
local
world
Local coordinate system offset, no rotation
(Note: example uses left handed coordinate system,
XNA uses right-handed coordinates)
Source: MSDN DirectX documentation
Local coordinate system offset and rotated
www1.adept.com/main/KE/DATA/V%20Plus/V%20Language%20User/i
mages/World+yaw.gif Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Translation
• A translation shifts the local coordinate system relative
to the world coordinate system
• XNA provides a method for this
– Matrix.CreateTranslation
• 3 parameters are x,y,z movements
• Matrix.CreateTranslation(0.01f, 0, 0); // Shift right (positive) along x axis
• Matrix.CreateTranslation(-0.01f, 0, 0); // Shift left (negative) along x axis
• Multiply world matrix by translation matrix to cause
shift
– All translations and rotations in 3D graphics accomplished
via matrix multiplication
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Rotation
• A rotation shifts the local coordinate system by
an angle relative to the world coordinate system
• XNA helper methods
– Matrix.CreateRotationX, Matrix.CreateRotationY,
Matrix.CreateRotationZ
• Rotations around single axes
• Matrix.CreateRotationY(angle in radians);
– Matrix.CreateFromYawPitchRoll
• Rotate around multiple axes
• Matrix.CreateFromYawPitchRoll(yaw rad., pitch rad., roll rad.)
• Demonstration of example triangle drawing code
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Triangles are fine, but models are better
• Today’s 3D games have a large number of objects
– Theoretically possible to create these objects by
manually writing C# code to create each individual
triangle
– In practice, this is rarely done.
– Far better to use a 3D modeling tool
• Maya, XSI, 3DS Max, Blender
– Allows artists to create objects in world
– Allows programmers to focus on behavior of objects
– Modeling tools permit much faster creation and
editing of models
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
3D Modeling
• Several ways to model a 3D object
– Polygonal modeling
• Objects are subdivided into a series of polygons (triangles)
• Can only approximate curved surfaces
• Dominant modeling form in computer games and computer
graphics due to speed of rendering
– NURBS
• Surfaces are defined by spline curves
• Curves defined and controlled by control points
– Splines and patches
• Curved lines define surfaces. Between polygons and NURBS
– Primitives modeling
• Objects built up from primitive shapes (balls, cubes,
cylinders, etc.)
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
3D Model formats
• There is a huge number of 3D model formats
– No dominant standard
– Interchange among models is often lossy
• XNA supports two 3D model formats
– .X (DirectX)
– .FBX
• Originally for FilmBox by Kaydara, then Alias, now
Autodesk
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Models in XNA
• Model
– Represents some entity
Model
• A person, a car, or potentially a complex scene
with many parts
– Typically used to represent one logical object
(e.g., a person) that has multiple parts (head,
arms, legs, etc.)
– A model contains multiple meshes and bones
1
N
1
N
ModelMesh
Bone
• Mesh
– Represents a single physical object
– Triangles, textures, shaders, etc.
– XNA ModelMesh class
• Bone
– Represents placement of each mesh relative to
other meshes
– A transformation matrix
Car
Body
Wheel
Door
+bone
+bone
+bone
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Models in XNA (cont’d)
Model
• ModelMesh contains:
1
– List of ModelMeshPart
– List of Effects
– Verticies for triangles that
comprise this mesh
N
1
1
– Also has a bounding sphere
– Represents a set of triangles
that share the same materials
(e.g., shader, or Effect)
– Has indexes into the ModelMesh
– Starting index, number of triangles,
number of primitives to use from
parent ModelMesh’s VertexBuffer
N
ModelMesh
• VertexBuffer
• ModelMeshPart
1
1
N
N
1
1
StartIndex
NumVerticies
PrimitiveCount
1
Effect
ModelMeshPart
1
1
1
Effect
Bone
1
VertexBuffer
1
N
Verticies
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Working with Models in XNA
• Bring model into XNA project inside Visual
Studio
• Load model into XNA via Content manager
• Draw model by iterating through all contained
meshes
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Importing Models into Visual Studio
• Import model into Content directory
– Copy .x or .fbx file, along with associated texture
images, into Content directory in XNA project
– Inside Visual C#, right-click on Content directory
• Add… Add Existing Item
• Select .x or .fbx file
– Similar process to adding bitmap textures, etc.
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Loading Model via Content manager
• Models are loaded via the content manager
– Model model = Content.Load<Model>(@”{name of
my model without .x or .fbx extension}”)
• XNA parses the model file, and fills in verticies,
textures, and effects in Model, and ModelMeshes
– In XNA, this is a robust operation, big time savings
– In many open source 3D game engines, model import
can be a big problem
– At present, typically is not safe to assume model
import will work smoothly in a given 3D game engine
– Need to test your tool chain
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
Drawing a Model
• Iterate through all of the meshes
– Iterate through each effect for each mesh
• Set lighting, camera, and world for each effect
– Draw each mesh using Draw() method on ModelMesh class
foreach (ModelMesh mesh in model.Meshes) {
foreach (BasicEffect be in mesh.Effects) {
be.EnableDefaultLighting();
be.Projection = camera.projection;
be.View = camera.view;
be.World = world * mesh.ParentBone.Transform;
}
mesh.Draw();
}
• Examine example code from Chapter 10 of
Learning XNA 3.0
Adapted from Jim Whitehead’s slides
UC Santa Cruz
Computer Science – Game Design
More on Models
• Explanation of parts of XNA models
– http://blogs.msdn.com/shawnhar/archive/2006/1
1/20/models-meshes-parts-and-bones.aspx
Adapted from Jim Whitehead’s slides