Matt Mitchell
David Pennenga
John Formica

 Topics to cover
 Levels of threat
 Straying from the team
 Use of caution
 Team context
 Speed of movement



The more targets, the greater the possibility of obstructing line of sight (LOS)
 How to fix this?
If the player is looking in the direction of NPC
 Move to the angle of LOS that has less targets

 If the player is looking away from
NPC
 NPC should move in opposite direction that the player is looking



If above heuristics fail, prioritize obstruction
 Threat level of enemy
A player is more likely to target:
 enemies with greater threat levels
▪ Obstruct view of enemies with lesser threat levels
 Groups of enemies over individuals
▪ Opt for obstructing view of smaller groups or individuals
▪ Threat level also should be considered



NPCs should stay close to the team
 Set maximum radius from team
Why?
 Straying from the team could:
▪ Alert enemy combatants
▪ Distract player from game
▪ Trying to find lost teammates
▪ Cause failure to assist player



Teammates should avoid risking the welfare of the team
NPCs should not be the first to engage an enemy
 Explore a room
 Move over a hill or around a corner
 This could undermine the player’s play style



Take different play styles of the player into consideration
 Some games allow player to set play styles
▪ Aggressive vs passive
▪ E.G. Republic Commando
▪ “search and destroy”
▪ “form up”
▪ “hold position”
Move at the same pace as player
 But not obstructing LOS
 Consider tactically sound behavior
▪ We will cover more about this shortly

 Types of behavior
 Use of cover
 Selective firing
 reloading

 Path-finding algorithm
 Prefer cover and the shadows
▪ Causes player to believe in tactical prowess of NPCs
▪ Adds to realism of game play
▪ Prevents accidentally alerting NPCs
 Take player cues If NPC behavior does not match player behavior
▪ E.G. Speed of player movement could cancel priority for cover in path-finding algorithm if player appears to be in a rush

 Only fire when within range and in view
 Controlled bursts
▪ Maintain accuracy
▪ Conserve ammunition
 Reaction times
▪ Should be manipulated if individual combat-related AI is too powerful
▪ Balance high accuracy with low reaction times
▪ Certain manipulations also make contextual sense: Rainbow Six style squad games
▪ Team reaction time should not be lower than enemy team’s reaction time



Cycling aggressive NPCs
 Reloading NPCs fall back or take cover
▪ Return when finished
Stagger fire of NPCs
 So NPCs do not run out of ammunition at the same time
 “taking turns” at attacking ensures that player will most likely always have supporting fire

 Close support of the player
 Protecting the rear
 Reporting to the player
 Selecting a target
 Responding to orders

 Protect player’s sides
 Report threats via audio or visual cues
▪ Messages are a staple of squad game play
▪ Do not overwhelm player
▪ Reporting threats while in combat is not usually necessary
 Pick appropriate engagement times
▪ Not in a fire fight
▪ Do not aggravate more enemies than already fighting

 Chose targets other than the player’s
 Chose targets farthest from the player’s LOS

 NPCs are there to complement game play not hinder it
Also should take threat level into account
 Player might need assistance
▪ Health of player’s target versus player’s own health
 Consider higher concentration of targets




Weapon selection
 NPC weapon should not be more powerful than player’s unless dictated by player
Weapon and item selection
 Allow player first choice
▪ Implemented by tracking player proximity to items
▪ If the player approaches items and walks away then NPCs may pick up items
Reaction time
 Allow player to attack enemies first
▪ Unless team is under attack




Line of sight avoidance hierarchy
 Each child avoids the line of sight of the parent
 Allows for easy restructuring in the event of casualties
▪ Node replacement algorithms
Threat awareness
 Use LOS algorithm to find and report targets
▪ Maintain group awareness of threats by “looking around” so that NPC LOS updates, player awareness of environment
▪ Used in some groups’ Robocode this semester
Environment awareness
 Identify cover/restricted areas/straying from player



NPCs reference a team knowledge base that governs
NPC behavior
Finding available NPCs
 Availability = (1+N)(1+O)(1+P)+(Q*[infinity])
▪ N= # enemies in covering area (integer value)
▪ O= # enemies within range (integer value)
▪ P= # enemies threatening team (integer value)
▪ Q= supporting another teammate (boolean value)
 The above formula
▪ Provides a return value of greatest value in the event that the NPC in query is supporting a teammate (optimal behavior)
▪ If the NPC is not supporting teammate, it provides a finite return value which can easily be compared with the availability of other
NPCs




Rainbow Six: Raven Shield
 Team of AI finish a mission http://www.youtube.com/watch?v=-9f3cXBB-Nc
Visible in this video
 Maintaining constant awareness of environment
 Reaction time alterations for context
▪ Very fast for mimicking elite soldier behavior
 “Stick to the shadows” path-finding
 Controlled fire
 NPCs stay close to the team
 Appears to use LOS dominance hierarchy



Decentralized approach
 Squad members exchange info equally
▪ Messages!
 No commanding force
▪ Instead, equally shared knowledge bases for reference of each member
 Squad behavior is an extension of individual AI
Centralized approach
 Leader receives and interprets info from soldiers
 Orders soldier behavior
▪ Differing strength of orders
▪ This will be covered shortly



Pros
 Extension of individual AI (easier to implement)
▪ All AI simply must communicate with each other either directly (messages) or indirectly (communicating with an abstract knowledge base)
 Robustly handles situations
▪ Emergent behavior facilitates problem solving without having to specifically code for given tasks
 Deals well with variations in capabilities of teammates
▪ Individual AI is permitted to interpret or act upon supplied info differently
▪ hence this approach being an extension of individual AI
 Easily combined with scripted AI
▪ Same reason as above
Cons
 Weak at autonomy or tight coordination
▪ Lack of specific plans for problems hinders team cohesion
 Cannot deal with individual strengths/weaknesses



Tightly coordinated
 Members rely on detailed, repeatedly rehearsed drills, and continuous flow of info
▪ In CSE terms, events are anticipated and coded specifically to be solved
Loosely coordinated
 Does not rely on planned maneuvers
 Relies more on verbal communications



Fire and maneuver behavior
 individual behavior of either firing or moving
▪ Communicate this within the squad
▪ Emergent behavior of position cycling
Goal behavior achieved via communication
 Stay close to cover
▪ Communicate info of cover whereabouts
 Prevent blocking LOS of others
▪ Communicate direction of LOS
 Take weapon capabilities into account
▪ Damage, area of effect, etc.
 Maintain group cohesion
▪ NPC location and intentions
 Spread out
▪ Communicate distance from other NPCs
 Keep LOS on targets
▪ Target locations, if the target is moving in the direction of another NPC




For each member
 Current position and activity
 Claimed destination position
 LOS
For each opponent
 Last known position/state
 Estimated current position
 Members engaging opponent
 Members able to observe opponent
 LOS
For other hazards/threats
 Known/estimated position
 Damage radius

 Why so preferred?
 Model communication latency by queuing messages
 Present the message in game play
 Can filter messages for player benefit
▪ Prioritize info of messages
 Sends to dead members assumed alive for realism
▪ Only serves a purpose when player witnesses it
 Can use scripted entities to direct members
▪ The content of messages serves a purpose whether the source of the info is adaptable AI or not
 Accommodates human members

 Goals
 Wait for enemy to enter kill zone
 Pull back to predefined rally point after engagement

 Fall back when discovered
How does a decentralized approach handle ambushing?

 Decentralized approach…
 Returns fire well
 Can prefer being near rally point
 BUT
▪ Cannot reach unanimous behavior on attacking because each agent will carry out its own behavior
▪ Cannot orchestrate movement well because there is no central executive force to create a consensus of behavior

 Centralized approach styles
 Authoritarian command style
▪ Focuses team performance by forcing agents to perform commands
▪ Cannot be ignored by individual AI
 Coaching command style
▪ Advises, rather than forces, agents
▪ Allows for agents to override commands
▪ Problems could arise from agents wanting to always perform their own behavior

 Things to consider
 Force ratio
▪ ratio of allies to enemies

 Line of fire ratio
▪ ratio of # of enemies in allies’ line of fire to the opposite
Consider Boolean states to make judgment calls
 Knowledge bases
 First order logic conjunctions

 Chosen path is a good combination of
 Short travel time
 Concealment from pursuing enemies
 Sufficient space to maneuver

 Progress should be monitored
 Arriving at position informs the squad

 Provides individuals with a new command
Pros of this approach
 Allows for changes in squad behavior midmaneuver
▪ E.G. a member is trapped in a spot which forces a reevaluation of the path finding algorithm and sends it far away from the squad



Use maps or an alteration of the A* function
New cost function considers
 Nodes that can be fired at from preceding nodes
 Nodes that provide insufficient space to bypass

 Nodes that are a bad position from which to fight
The time and spatial complexity of this new function force it to be applied to the squad aggregate for practicality

 To properly pull back
 Mark each position just before a bend in the path that blocks the LOS of many preceding positions on the path
 Features sufficient room for members to bypass it
▪ If first choice of bypass is obstructed some how, a second choice should be available
 Is a good position from which to fire
▪ Pulling back across an open field versus a rocky hillside



Chain of command:
 Commander -> captain -> sergeant -> soldier
 More subdivisions are allowed if necessary
Creates an organized and efficient way of commanding soldiers



Strategic Decision Process
 Check if maneuver is possible/appropriate
▪ Doors are accessible (referencing door IDs so there are no duplicate actions)
 Compare
▪ Distance between opponent and door with distance between soldier and door
Maneuver organization
 Consider distance to closest door so maneuver is as fast as possible

 Strategic decision process
 Try to determine whether the maneuver is possible/advantageous
▪ Check if ally must cross a specific room in order to get from their room to a target

 Maneuver organization
 Find how many soldiers are needed to cover exits
 Order soldiers to cover designated positions
 Decide on a room to search
 Order soldiers to move to reachable doors of the first room
 When all of the above have been completed, a search command is sent to squad

 Types of formations
 Line
 Left flank
 Right flank
 Column
 Box
 Wedge
 Vee

 Different facing applies to different tactics
 Defense vs offense
 LOS
 Friendly fire


 Project more firepower
Staggered line
Outward facing vs inward facing



Mixed unit type ordering
 Weaker/longer-ranged units in the back of the line or center of a box
 Faster units placed at flanks
Closest position
 Calculating the best match of unites to positions
▪ Sort units based on their minimum distance to the closest position
▪ Iterate through the sorted list
▪ Assign each unit the closest unused position



Direction may play a role if turning is difficult
Create formation positions that avoid crossing paths of units

 Use movement vector and parallel paths
Arriving time
 Fix variations in arrival time by making all units’ paths lead to the center of the formation and order is determined by arrival time (illustration on next slide)




Spacing distance
 Must be scaled so that the size difference in units is considered
Ranks
 Each formation has a maximum occupant limit
▪ Start new formation once reached
Playbook
 If certain places are reserved for specific units restrictions must be coded
▪ If no match found for a position, slot may be skipped (hole is made)



When do the units fall into formation?
Upon initial movement
 Might delay movement

 Might cause unit backtracking
Just before destination
 Might give player impression that nothing is happening
 Might be best choice for coordinated attack

 do formation at destination
 Faster movement/no delays
 Formations made as distance to destination decreases
 Not as effective while en route
 Not as organized upon arrival

 Group path finding and movement
 Calculate path for lead, others apply formation offsets to path
 Flocking style where group follows slightly behind lead in rough formation
 When mixed units move as a group, differences in locomotion can break formation integrity
▪ Solution: all units move like the slowest unit
▪ Problem -> faster units might appear to move in slow motion



React to ambush
 Spread out to avoid area of effect or converge to chase down an enemy
After passing obstacle or attack regroup in the same fashion that the group was originally formed
 Reassign positions to make up for lost units if necessary



Algorithms capable of tactical assessments in a dynamic environment
Modification of influence mapping
 Cells of influence are calculated from unit positions outward
 Total influence: a factor which assesses bot tactical situation in the engagement area
 Each “cell” superimposed on the influence map has a calculated value associated with agent and environmental qualities




Learning algorithms that give an NPC the ability to adapt to opponents’ playing styles in real time
Pros
 Provide more challenge as opponents because they are unpredictable
 Facilitate the discovery of new strategies
Cons
 Developers cannot test what behaviors the NPCs will exhibit



Entire team of NPCs view as one chromosome
 Each agent derived from a different part of the chromosome
▪ Creates heterogeneous teams
Five node sets, total of fifty nodes used in evolution



Fitness calculation
 Takes into consideration the game’s duration and remaining health of team and enemy agents
Selection
 Elitism: m copies of the best n chromosomes from each generation are copied directly into the next generation
 Roulette wheel selection
▪ Any chromosomes selected are subject to randomness

 http://www.youtube.com/watch?feature=pla yer_embedded&v=4ErEBkj_3PY