1 Utilizing Next-Gen Technology to Create a Dynamic Game Environment Mat Toellner, Nick Heitzman, Elizabeth Stringer Abstract— With the release of the eighth generation of video game consoles, the visual capabilities of games and their engines have improved to allow for more options in the creation of dynamic environments. The primary characteristics of dynamic environments are that they change in real-time. This paper documents the technology and development techniques in the creation of a set of materials that allow the gameplay space to change while players are experiencing it for a next-generation dynamic environment. I. INTRODUCTION F or a game environment to be successful it needs to restrict and guide players through the game while providing a context to the world they are experiencing, how they can interact with it, and how it affects them [1]. Specifically, developers are able to apply the concept of a dynamic environment in which the passage of time shows changes in the setting and how the world communicates to the player the results of their interactions. Technically speaking, to make an environment dynamic, its assets need the capability for quick modification in real time [2]. The primary goal is to create assets for a dynamic environment using next-generation game technology and discover techniques applicable to the new console generation (PlayStation 4, Xbox One, and Wii U) while analyzing what techniques from the previous generation can be carried forward. The project utilizes the Unreal Engine 4 (UE4) to focus on developing assets for a horror/survival game that takes place in an abandoned arctic base. UE4 has advanced technology for making game worlds more dynamic: real-time lighting, physically based rendering (PBR), material instancing and layering, and mesh distortion via materials [3]. These capabilities allow for added visual depth due to materials’ interactions in the environment, specifically with bounced light and reflectance resulting from light scattering and world-space specific reflections due to PBR[4]. For the project artifact, the majority of the real-time changes to the environment utilize parameters in material setups and the Blueprint scripting system. This allows the environment to change while players navigate through, instead of remaining static for the duration of the user experience. II. RESEARCH REVIEW Dynamic environments in videogames often include changes in lighting, different weather particles, and animated assets [5]. Examples of popular games from the previous generation of consoles with dynamic environments include The Elder Scrolls V: Skyrim [6], Journey [7], Dead Space 3 [8], Uncharted 2: Among Thieves [9], and The Legend of Zelda: Twilight Princess [10]. Analyzing these titles provides insight into ways that lastgen games successfully create their locations. This analysis also provides ideas to expand upon environment interaction with the player to convey aspects of gameplay to them through the changes in the environment. Observable traits include how games use of textures, colors, lighting, particles, player visibility, and character interaction with the environment. Skyrim’s northern regions feature an abundance of snow and ice [6]. Assets have lower color saturation in this area than other regions of the game. However, large blocks of ice have a blue coloring rather than being simply white or grey. There is snow on the top of structural models in the texture, but smaller models do not have snow on them. There are also snow and ice models used to make certain areas appear to have more build up than others. The day and night cycle features different lighting effects. The day lighting is primarily neutral (unsaturated), and can be rather dark depending on the active weather. At night, there is mostly green and blue lighting, as well as added sky illumination with an aurora borealis. Interior space lighting differs dramatically depending on the space, but man made interiors tend to have warm lighting (from flames) while caves have primarily blue tinted (cold) lighting. Use of fog and low saturation color create the atmosphere for this mostly frozen region. Similarly, ice-covered interiors have a swirling mist effect and certain areas have dripping water particles to suggest that ice is melting; these effects make the space less static. Exteriors also have frequent snowfall particles that further enhance the atmosphere. Characters in the game do not leave footprints or kick up any snow as they move around, creating a minor break in player immersion. Overall, the snow does not dynamically build up anywhere, but instead the illusion of that is a result of different areas having varying amounts of static snow and different frequencies of snow particles. This makes it seem as though the snow level is changing while the player travels through different locations. 2 Fig. 1: The Elder Scrolls V: Skyrim [6]. Journey introduces its wintery environment rather abruptly, with introductory areas of snow piles and light snowfall [7]. The player then enters a very monochromatic landscape of rock and snow, with excessive wind that blows flurries of snow along with the player. Throughout majority of this area, there is a constant light snowfall. There is snow built up on tops of surfaces and sides of models facing the wind, and areas bare of snow to convey safe paths to the player. In some sections, there are cloudy particles to further develop atmosphere and obstruct the players view. The snow terrain acts similar to sand shown earlier the game, with players permanently displacing it as they move through it. The player’s character slowly has snow accumulate on them. This causes them to lose saturation in their red clothes as the character frosts over from the edges of their clothes inward. Similarly, the corners of the player’s screen begin to white out and on-screen visibility lowers as the player makes their final ascent to symbolize the character’s exhaustion in this frozen locale. Overall, a majority of this final area lacks saturation and relies mostly on bluish greys, contrasting the warm areas earlier in the game. Fig. 2: Journey [7]. Dead Space 3’s arctic environment varies a lot depending on the different gameplay that takes place [8]. Overall colors on environment assets are low in saturation, but there are vibrant colors on electronics. The exterior structures and assets have frost and snow on them, which shows the most build up near the ground. There are sections in the levels with very low visibility and others with a lot, depending on the atmosphere needed for gameplay. The use of either heavy or light snow particles is what determines this the most. In addition to heavy snow particles, there are tarps attached to various parts of the outposts flapping in wind to emphasize the high wind speed. Exterior lighting relies on the dynamic skylight, but there are sometimes red flares on the ground to guide the player. Interiors are mostly dark with various colored lighting, which depends mostly on the colors of different electronics. The interiors also have icicles and soft snow particles where there are holes in ceilings or broken windows. In the corners of rooms and along the walls there is snow built up to show that these interior environments have had weather exposure for a while. The player’s character has frost built up on their armor, which changes based on if they are in an exterior or interior. The frost dynamically grows/shrinks based on the players exposure to the weather. The character leaves persistent boot prints in the snow as they walk around. The character also has visible breath particles to further show the environment’s low temperature. Fig. 3: Dead Space 3 [8]. Uncharted 2’s environment assets have more color saturation than other games’ snowy environments, but are darker in contrast to the snow [9]. Environmental objects have snow and frost on them, which is heaviest near the ground. The various forms of ice are mostly white, but have a blue/green sheen. There is snow built up along edges and in corners of the floor, as well as in cracks between stones on various materials. Large surfaces, such as rooftops, also have snow spread unevenly across them for variation, rather than being an even blanket. Generally, there is a light blue atmospheric fog over majority of the environment, which adds depth to objects in the distance. Lighting is mostly neutral sky illumination, occasionally tinted purple or blue. The lighting is significantly bluer in darker areas and enclosed spaces, such as caves. There is falling snow and soft cloud particles that convey weather without obstructing the players view, except when gameplay intends for it to. The player character has dynamic snow and frost build up on them that fades over time, and is specific to areas that encounter snow (mostly the legs). The character kicks up snow piles behind them as they walk, but the piles disappear quickly over time. However, characters do leave persistent footprints in the snow. 3 III. METHODOLOGY Fig. 4: Uncharted 2: Among Thieves [9]. Twilight Princess’ Snow Peak exterior is mostly snow and rocks, with its interior textures being mostly desaturated earth tones [10]. The interiors have open windows with soft snowflakes drifting in. Snow piles are in most corners, if there is a prominent amount of snow in the room. Where the walls meet ice floors there is frost creeping up the sides. Lighting in the exterior is primarily neutral with a slight blue/green tint. Indoor lighting is the same, but there is orange lighting around flame light sources to bring attention to important or safe areas. In certain areas, there are constantly blowing fog-like particles to represent the wind direction, with the occasional use of snow flurries to further obscure players’ view. There are large ice blocks with a vibrant green/blue color and mist particles. The color and particles serve to warn players they take damage on contact. The player does not leave footprints nor raised snow piles, but there are particles of snow kicked up when moving. This artifact-based thesis primarily focuses on the creation of art assets for a team-developed horror/survival game that is set in an arctic base. This game, called Snow depends on a changing environment to communicate to the player that the frigid arctic is starting to reclaim the facility around them. Using the arctic as the setting for this thesis allows for quick changes in the environment from snow build up while also communicating to the player about temperature differences in their environment. Assets needed to develop this dynamic environment include accumulating snow, melting snow, and frosting glass. Snow, developed in UE4, focuses primarily on using multi-parameter materials to simulate the location’s conditions changing around the player. These materials are also set up using a vertex color expression, specifically pulling from the green channel, to allow vertex painting to better denote where snow builds up rather than relying solely on the masks. These assets allow the application of next-gen procedural techniques to create successful environments while reducing the amount of development time needed [11]. Some of the key features that make a game’s environment look next-gen revolve around materials, polygon counts, and lighting; however, this artifact focuses primarily on materials [4]. A. Accumulating snow Snow accumulation on a variety of surfaces is a result of fluctuating masks linked to a scalar parameter. The setup for the snow accumulation effect differs depending on if the snow is on flat surfaces or models. Fig. 6: Accumulating Snow Materials Fig. 5: The Legend of Zelda: Twilight Princess [10]. All of these last-gen games have similarities in how they build the features of their locations. The common takeaways from these games are textures with low saturation and snow build up only on areas snow would be able to collect or stick. This tends to be on objects’ tops and sides closer the ground. Some independent snow and ice assets are also helpful to have in order to show an uneven buildup, with snow piling higher along walls and in corners. Lighting either would be tinted blue/green or be neutral, unless there are important areas that need emphasis. Having fog and additional particles to help build atmosphere can help to add depth to the space and make it feel less static. In addition, any sort of interaction the character has with the environment, such as displacing snow or leaving footprints, makes the environment seem more dynamic. For flat surface or BSP textures, the material setup includes texture parameters, which allows the swapping out of base textures. To help the snow build up in an uneven manner, the albedo map is desaturated and applied as the base mask. What is darker on the albedo is where the snow appears first. To help this work better, having some shading in the albedo makes the snow build up in areas of recession before completely covering the base textures. An ambient occlusion or cavity map would work well for this purpose too, however to simplify this material the mask only uses the albedo. In the albedo segment of the material, the mask plugs into the alpha of a linear interpolate (lerp) expression to determine between either the base or snow textures. This lerp then goes through a power expression to make the snow more apparent. Afterwards, it goes into another lerp, which is set to add a 5% overlay of the original base material, which keeps the snow from completely overriding the base texture. The normal segment is also set up so that the snow does not completely cover up the base normal map. To achieve this, the 4 result of the first lerp plugs into another lerp along with the base normal texture. The mask, plugged through a clamp limiting it to 70% strength, then determines between the base normal and the first lerp. The result is that the base normal is always at approximately 30% strength at a minimum. The metallic, specular, and roughness segments are all standard simple lerp combinations, with the exception being an added scalar parameter in the metallic segment. This parameter is multiplied over the base texture to override it (by making it black) in case the base texture set does not include a metallic map. Conversely, a plain black texture used in the metallic texture parameter would work as well, in which case the multiply and scalar parameter expressions would not be needed. Fig. 8: Material Setup for Accumulating Snow on Models B. Melting Snow Snow tracks shrinking and water puddles growing simulates the effect of melting snow. This material uses set values for the puddle’s base textures rather than relying on maps except for the normal texture. The snow texture maps are the same as the ones used in the previous materials, with exclusion of the mask being unique for the boot-print pattern. Fig. 7: Material Setup for Accumulating Snow on Flat Surfaces For models that have snow building up on top of them, a similar material setup to the prior one is used. The main difference is that this material features mesh distortion to make it appear as though snow is piling up only on the tops of the models. For this, a custom mask makes the snow build up unevenly, rather than relying on the base albedo texture for a mask. This is because the material has a world-space vertex expression that calculates the model’s positive z-axis as part of the mask, so snow only builds up on the tops of meshes regardless of the base textures or orientation. This material also features mesh tessellation and displacement so that the geometry of the mesh moves upward with the snow, to create the illusion that there is actual snow collecting on top of the model. To achieve this the mask is multiplied with its clamped world-space vertex normal and that result is multiplied with the initial scalar parameter (now clamped to a maximum in 20 instead of 1) too allow for more height of the snow buildup. Afterwards, it multiplies with another world-space vertex normal to make sure it follows the same axis restriction. In addition to this, a scalar parameter plugged into the tessellation multiplier node on the material output allows for varying vertex tessellation in the buildup, which may need adjustment depending on the model. Fig. 9: Melting Snow Tracks Material The shrinking snow tracks use a scalar parameter that, after being multiplied with the mask, goes through an additional one-minus expression (to invert it), before plugging into the final clamp. The expanding water uses the same scalar parameter as the snow tracks and is set up similar to the previous materials. Both of these masks then use a lerp to determine the textures to display in the corresponding areas. The effect of this double mask is that the snow diminishes as the puddle grows. This material is set to be translucent so that it does not cover up the floor where there is not snow or the puddle. This material also has a scalar parameter tied to the overall opacity, which makes it look like the puddle slowly fades overtime. The albedo, metallic, roughness are set up similar to the previous materials, only with two lerp nodes instead of one. However, the specular is one map individually and does not use a lerp. 5 IV. RESULTS Fig. 10: Material Setup for Melting Snow Tracks C. Frosting glass Glass frosting over due to the cold is a result of using a Linear Interpolate with a Scalar Parameter multiplied through the mask. The mask has a radial gradient so that the achieved affect makes it appear as though the glass is frosting from the corners inward. This material is also set to be translucent, as the player needs to see through the glass. Fig. 11: Frosting Glass Material This frosting glass material uses the mask to directly lerp between the plain window and the iced window textures. Much of the material is comprised of simple lerp combinations, but the opacity and refraction segments are slightly more complex. Both of these segments feature an additional set of expressions used to make the inside of the glass obstruct the player’s vision less. To achieve this, the normal map data lerp goes into a Fresnel expression. The Fresnel, which calculates the edges of an object against the camera, then plugs into two lerp expressions, one in opacity and one in refraction. These two lerp expressions then determine between the prior glass/ice data and another set value to make the insides of the glass less opaque/refractive. Fig. 12: Material Setup for Frosting Glass The artifact is successful in creating a base set of materials for a dynamic environment. The material shaders allow for the real-time alteration of both meshes and materials by adjusting them in Blueprint. The active buildup of snow and ice also conveys to the player that the environment they are in is changing and they will need to move to escape the cold. Furthermore, the artifact also successfully applies next-gen techniques. It applies PBR to distinguish the appearance of materials by making surfaces reflect differently depending on the light scattering resulting from the roughness and metalness maps. The material setups also utilizes parameters in the material to adjust the procedural snow accumulation to accommodate different base textures. The procedural capabilities are particularly successful with the tessellated snow material, which allows snow to build upwards on any model, despite its orientation. However, while the artifact does create a good base for a dynamic environment, the game world would benefit from more dynamism. The implementation of particles, changing lighting, and animated assets would reinforce the amount of real-time changes that communicate with the player about their surroundings. The use of flickering lights, blowing snow flurries, and moving assets would make the world much more engaging than relying on shaders alone. Fig. 13: Artifact Materials Applied to a Scene Despite the successes of the artifact, there were issues found during its development. One such issue is that the snow accumulation material, which uses the base albedo as the mask, has a lot of variation on when snow will begin building up (based on the parameter’s value). Specifically, darker albedo maps will begin accumulating snow faster than lighter ones. To correct this, using ambient occlusion or cavity maps for the different material textures would serve as a consistent base to calculate the snow accumulation on. Another problem found during development of the artifact is that mesh distortion on models with UV seams that are in an area with tessellation have a chance to make holes in the geometry of the mesh. However, optimizing models so that there are not UV seams where snow builds up fixes the issue. Similarly, this material works better for static objects because if an object rotates the snow will auto adjust to the positive zaxis for the new orientation. This could easily break a player’s immersion, as the snow would seemingly slide across the 6 surface while the object rotated. An additional issue discovered was that UE4, as of version 4.6.1, does not support the use of parameters in material layers. Part of this artifact was to have multiple electronics that would easily toggle between different light states. The electronics were intended to be tied together using a material layer for the light level so that they could all easily switch through on/off/low power states, however since the parameter could not be used in a material layer each material has to be called independently in Blueprint. The ability to apply a material layer that has a parameter to multiple other materials would give developers control over everything that used that material layer in a single instance. V. CONCLUSION Overall, the components used to make dynamic environments in last-gen games carry over to the new generation with some improvements. Rendering lighting in real-time instead of pre-rendering it helps make spaces more dynamic because the lighting is less static and reflections and highlights are much more responsive to the camera and world space. The major additions come from new technological allowances to create real-time adjustments to meshes and materials, which further help convey to the player about changes in the game space. While the development of next-gen games focuses more on procedural techniques to reduce development time and disk space, they also use more processing power than the last-gen method of development [11]. The majority of lessons learned during development of this artifact was about procedural generation. Using a parameter tied to the mask for these materials allows the shaders to work with most base textures. It also reduced the amount of time needed to create the visual effect of snow/ice accumulating and melting; once the parameters and the material were set up correctly they were applicable to many different surfaces. Being able to adapt this to more assets in the future will allow faster iteration along with more flexibility in the look of materials. However, procedural generation also has a chance to work improperly, so it may require optimization or tweaking depending on its use. Furthermore, using UE4’s PBR system in conjunction with real-time lighting allowed for easier distinction between different material texture types. This is because the addition of roughness and metalness expressions to the material editor allows light and reflections to be scattered off the surface differently depending on the input value. However, there are potential risks with using PBR. The primary risk is potential for increased development time upfront; it is a newly adopted technique with a learning curve, but once artists learn it future development should not take any longer. PBR is also likely to cost more GPU than memory, so in terms of performance it just needs consideration to the project’s constraints. Similarly, real-time lighting may be more taxing than precomputed lighting. Performance will vary depending on assets in the level in addition to the amount of dynamic, static, and shadow casting lights. To optimize performance, keeping a low number of shadow casting dynamic lights would be the most beneficial, as well as using spotlights over point lights, since spotlights only affect objects within a cone. Additionally, with minimal adjustments the materials created for this thesis artifact could have a variety of procedural applications, despite their intended use in an arctic setting. An example would be changing the snow textures and masks to dirt/grime, which would make these materials usable for different levels of weathering. Further study of this subject would include research into the Snowdrop Engine from Ubisoft Massive, as tech demos of this engine have shown dynamic snow accumulation in an urban environment. Additionally, this engine is in development in conjunction with the next-gen game Tom Clancy’s The Division, which primarily takes place in New York in the winter. Analyzing this game would also help provide insight into more next-gen techniques for creating a dynamic environment. Additionally, any further development done in combination with programming assistance could provide additional support to optimize the procedural materials further by adding more features to the tools. One such example would be the ability to use parameters in UE4’s material layers. This function could add more possibilities and reduce the amount of set-up time for new materials, since a material layer functions independently from the normal materials that use it. VI. REFERENCES H. Smith and M. Worch. Game Developer’s Conference 2010, Topic: “What Happened Here? Environmental Storytelling.” San Francisco, California, March 2010. [2] M. Walsh, Game Developer’s Conference 2010, Topic: “Dynamic Navmesh: AI in the Dynamic Environment of Splinter Cell: Conviction.” San Francisco, California, March 2010. [3] Z. Parrish, 'Unreal Engine 4 - Tools Demonstration GDC 2014', YouTube, 2014. [Online]. Available: https://www.youtube.com/watch?v=9hwhH7upYFE. [Accessed: 07Apr- 2015]. [4] S. McAuley, Game Developer’s Conference 2015, Topic: “Rendering the World of Far Cry 4.” San Francisco, California, March 2015. [5] Capcom, 'Dragon's Dogma dynamic environment', 2011. [Online]. Available: https://www.youtube.com/watch?v=ybJD9jAg_Cg. [Accessed: 08- Apr- 2015]. [6] The Elder Scrolls V: Skyrim, Bethesda Softworks, 2011. [7] Journey, Thatgamecompany, 2012. [8] Dead Space 3, Visceral Games, 2013. [9] Uncharted 2: Among Thieves, Naughty Dog, 2009. [10] The Legend of Zelda: Twilight Princess, Nintendo, 2006. [11] M. Olsen, Game Developer’s Conference 2015, Topic: “The Ultimate trim: Texturing Techniques of Sunset Overdrive.” San Francisco, California, March 2015. [1] M. Toellner has been creating art for video games for two years while attending The Guildhall at Southern Methodist University in the Master of Interactive Technology program, with expected graduation in May 2015. Prior to this, he earned his Bachelors of Fine Art in Drawing and Painting from University of North Texas in 2013. G. Heitzman has been creating art for video games since SEGA Enterprises Arcade Division in Tokyo back in 1992. With contributions to games on multiple platforms from the Nintendo 64 and PlayStation 1 days through to today’s consoles and computers, Mr. Heitzman has been fortunate to see firsthand the evolution of technology as it relates directly to game development for the last two decades, and now is sharing that knowledge with 7 new generations of game creators as part of the Art Faculty for The Guildhall at Southern Methodist University. E. Storz-Stringer, M. Ed. Southern Methodist University. estringer@smu.edu