Current Methods to Create Human-Level Artificial Intelligence in Computer Simulations and Wargames. D. Ezra Sidran Ph.D. Student, University of Iowa Computer Science Department dsidran@cs.uiowa.edu When we talk about Artificial Intelligence in the context of strategy games or military simulations we can all picture what we’re talking about even if we can’t agree on the methods used to create it or even a precise name to call it. John Laird has used the phrase “Human-Level Intelligence”1 by which he means an AI that is able to compete with a human on an equal level. George and Cardullo2 use the term “humanlike expertise in the military domain.” However, we know what we want the AI to do: confronted with the situation that faced Grant before Vicksburg in 1863 we want the AI to ignore conventional military wisdom and send its troops below the objective and conduct a lightening campaign that ends with the envelopment and surrender of the enemy forces. Likewise, in a simulation of the Korean peninsula in September, 1950, we want the AI to conceive of a daring amphibious invasion 180 miles behind enemy lines as MacArthur did. We want an AI that understands the rules and the physics of its environment and yet can think outside of the box when necessary. Last summer I compiled an overview of current techniques to accomplish this task (available here:http://www.gilgameshcontrite.com/Research/ReadingsForResearch.pdf). Research into developing human-level AI can be divided into the following areas: Intelligent Agents A recurring theme in many military and academic papers are Intelligent Agents (IAs) that implement ‘smart behaviors’ or attempt to mimic personality traits of commanders or cultures. Laird’s work with ‘bots can also be put into the category of Intelligent Agents as they are both relatively small, self-contained applets. IAs are also becoming more common in commercial Role Playing Games (RPGs) to implement Non Player Characters (NPCs). There are now commercial third-party (middleware) packages for creating IAs3. IAs really do not introduce any new ideas in AI coding. An IA may be created using scripting, fuzzy logic, finite state machines, fuzzy state machines or even, theoretically, genetic algorithms. Their strengths include the ability to perform certain specific tasks (such as a QuakeBot’s4 ability to hide, ambush and fight) and their “set it and forget it” autonomy. Rene’ G. Burgess has just completed a very interesting thesis5, Realistic Evaluation of Terrain by Intelligent Natural Agents (Retina), which describes a method of “using a fluid flow model to develop likely avenues of approach through the terrain.” Finite State Machines Finite State Machines (FSMs) were described in Turing’s Computing Machinery and Intelligence6 (though he called them Discrete State Machines) over fifty years ago. To Turing an FSM was a physical device. Since then the term FSM, while it still can refer to hardware, is more likely to mean a series of AI routines that are linked together with specific entry and exit points. Obviously, FSMs are basic building blocks of AI and will continue to be so far into the future. However, it is their very nature of chopping real life actions into discrete compartments that makes me suspicious of FSMs playing more than an incidental or supporting role in the quest for Human-Level AI. Indeed, as Turing wrote, “Strictly speaking there are no such machines. Everything really moves continuously.” Fuzzy Logic Fuzzy Logic (FL) – a rule-based methodology as opposed to a strict mathematical system7 - shows some promise for the creation of Human-Level AI. Humans rarely think in absolute terms. For example: the driver’s education book states definitively that you are to signal a turn precisely 100 ft. before the intersection but people never get out and measure the exact distance. The same is true with military or strategic axioms. It is “good” to get behind an enemy. It is “good” to cut an enemy’s lines of supply and communication. However, there are no precise mathematical definitions for the terms. In 1985 I unknowingly used fuzzy logic in my commercial wargame, UMS: The Universal Military Simulator (screen shots below). Figure 1: Screen shot from “UMS: The Universal Military Simulator.” (1986). The program understood such concepts as ‘flanks’ and ‘center of mass’. Published by Firebird/Rainbird. Figure 2: Screen shot from “UMS: The Universal Military Simulator.” (1986). The computer AI could be ordered to conduct “left flank attack” “double envelopment”, etc.. Published by Firebird/Rainbird. The program “understood” concepts such as “line” and “flanks”. This was accomplished by first drawing an imaginary box around all the units in an army, then determining the army’s facing by comparison with the enemy army. It was then easy to determine left and right flanks as well as the ‘center’ of the army’s lines (a double envelopment is simply a simultaneous left and right flank attack). It is reasonable to assume that fuzzy logic will play a role in the development of Human-Level AI. Neural Nets Surprisingly it is the commercial gaming industry that is showing the most interest in Neural Nets. André LaMothe, a much respected writer about computer game programming, has produced an in-depth piece on the subject8. Neural Nets have gone in and out of style since they were first proposed over fifty years ago and are bound to become popular again. Neural Nets are reminiscent of the “meat machine” view of the human brain popularized in the ‘60s. Indeed, part of the allure of Neural Nets is the almost homeopathic belief that mimicry of the wiring of the human brain will, eventually, produce a sentient, human-level intelligence. Fuzzy State Machines “A fuzzy state machine (FuSM) brings together fuzzy logic and finite state machines (FSMs). Instead of determining that a state has or has not been met, a FuSM assigns different degrees of membership to each state (Russel & Norvig, 20029). Therefore, instead of the states on/off or black/white, a FuSM can be in the states ‘slightly on’ or ‘almost off’. Furthermore, a FuSM can be in both the ‘on’ and ‘off’ states simultaneously to various degrees.” - Sweetser10. Fuzzy State Machines will almost certainly be a part of any attempt to create Human-Level AI. A Life A Life (Artificial Life) is the current “hot topic” in computer game AI. This umbrella term has been used to describe any program that emulates or simulates living creatures even though the techniques employed can be quite different. The Sims, frequently cited as an example of A Life, actually uses a system of Intelligent Agents (see “Some Notes On Programming Objects In The Sims™”; Forbus; Wright)11. Other A Life programs employ genetic algorithms. At this time the term ‘A Life’ is too amorphous to have any value in this discussion. Scripting Scripting refers to any system in which a previously created file of behaviors (often a text file) is loaded at runtime to control the actions of Non Player Characters (NPCs) or units. AI scripts, in a way, are not unlike “book openings” in chess. They are a series of “canned” responses. For “The War College” (1996) I created three scripts for each army that contained detailed instructions for the coordinated movements of each unit. At runtime one script picked at random was loaded. The computer AI continued executing the script until any of a number of triggers (hostile units approaching within so many meters, key strategic positions threatened, etc.) were encountered at which time a series of heuristic tactical combat routines took over. Figure 3: Screen shot from “The War College.” (1996) published by GameTek. The battle of Arbela. The computer AI is following one of three pre-programmed ‘scripts’; in this case a feint on the right flank while the main attack from the left rolls up the enemy line. Scripts can be a very powerful tool but they presuppose some knowledge of the simulation. This was possible because “The War College” contained simulations of the battles of Antietam, Pharsalus, Tannenburg and Austerlitz. Being familiar with the real events and knowing the terrain I could create reasonably intelligent scripts for the armies to follow. Generic scripts could also be created that are not tied to a specific terrain or simulation such as football passing route scripts or guard patrol scripts. In the right circumstances, scripting can be a very powerful AI tool. Genetic Algorithms Genetic Algorithms (GAs) are an interesting concept; especially since they seem to promise to deliver quite a bit of AI for very little human coding. While a GA can, theoretically, deliver an ever more intelligent AI the downside is that GAs are notorious for consuming CPU memory and clock cycles. Because of this the commercial gaming world has not yet explored these methods. Belief-based Bayesian Network. I encountered only one example of this method (“A Bayesian Network Representation of a Submarine Commander”. Yu). Bayesian Networks (see http://www.cs.ualberta.ca/~greiner/bn.html) certainly seem to be very useful for representing uncertainty and making deductions, but they do not seem directly applicable to the creation of human-level AI. Hierarchal AI. Pottinger12 described a two-tier hierarchal AI as one of the methods that he used in his “Ages of Mythology” (published by Microsoft). The highest level AI, which Pottinger termed “AI Goals” are “super-high-level constructs such as ‘Attack Player 4’ or ‘Build a forward base in Player 3’s direction.’” Beneath this the lower level AI, dubbed “Plans” would then fetch the appropriate script and execute the behavior. In “UMS II: Nations at War” (1992) I designed a four-tiered hierarchal AI that corresponded to a nation’s Order of Battle Table (screen shot below). For example, the highest level (here represented as Montgomery the 21st Army Group Commander) would pass down the order to attack the German player. The next level down, Bradley, would then assign objectives such as attack Berlin. The next level down, the VII Corps in this case, would then assign a series of way points utilizing the road net to Berlin (passing through Paris for example) and give orders to the divisions under its control. The lowest level units would then execute pathfinding algorithms to get on to the road net and advance toward its first waypoint (Paris). Figure 4 Screen shot from “UMS 2: Nations at War” (1992) published by MicroProse. This screen shot shows an Order of Battle Table that displays the four-level hierarchy. A multi-tiered hierarchal AI system is crucial for emulating the behavior of complex military formations. Each tier of the hierarchy corresponds to an equivalent level of the Order of Battle Table. Indeed, any Human-Level AI that is operating in a strategic simulation, be it a military campaign or a football game, will need to implement a hierarchal AI. Final Thoughts In my research I did not encounter any program – commercial game or DoD funded - that had made substantial progress towards Human-Level AI as defined by Laird, George and Cardullo. That was not surprising. What was surprising was that I could not find any research group even attempting to create AI of that caliber. Research in developing Human-Level AI can be divided into two distinct areas: the commercial game industry and military sponsored research (which includes military funded academia). The two groups approach the problem differently because they have different motivations and interests. Commercial computer game developers are primarily interested in creating a challenging opponent that can play the game that they designed. A great deal of the commercial gaming AI is ‘purpose built’ and frequently hardwired for a specific game. Furthermore, there is a long history of cheating in commercial computer game AI design. Even when it is not overtly cheating many of the AI routines that are now being created have no real value outside the original application. For example, one recent technique is for the level designer or artist to pre-plot waypoints to be used in lieu of heuristic pathfinding. Indeed, this is even a built in feature of AI.implant13; a commercial middleware product. At the same time, the wargames created for the military (WARSIM, JWARS, TACOPS) do not have any strategic AI whatsoever. While creating Human-Level strategic AI is a daunting task, the failure to do so can be extraordinarily dangerous (recall V Corps commander Lt. Gen. William S. Wallace’s recent comments from Iraq, “The enemy we're fighting is a bit different from the one we wargamed against."14). The enemy they had wargamed against was controlled by U. S. officers following U. S. military doctrine. What they needed was to wargame against an enemy controlled by a Human-Level AI that could think outside the box so that problems were encountered during simulations and not on the battlefield. John Laird of the University of Michigan admits that there is not much of a technology transfer from academia to the commercial gaming industry, either (“When game developers look at AI research, they find little work on the problems that interest them, such as nontrivial pathfinding, simple resource management and strategic decision-making…” – “Bridging the Gap between Developers & Researchers”15). Yet it is inevitable that Human-Level strategic AI will be created and it will probably be created by the end of this decade. Let us consider the straightforward wisdom of U. S. Grant, "The art of war is simple enough. Find out where your enemy is. Get at him as soon as you can. Strike him as hard as you can and as often as you can, and keep moving on" 16 Now, let us add one more dictum: “position your forces so that they can inflict the greatest harm upon the enemy while the enemy is unable to respond in kind.” Now, with this one addition, we have created the basic algorithms of any successful strategic AI: 1. Locate the enemy. 2. Attack the enemy in such a way that the results of combat favor your forces. 3. Continue attacking for as long as the enemy is able to resist. Steps 1 and 3 are easy to implement. The second step is the subject of my doctoral research. References 1 Human-level AI's Killer Application:Interactive Computer Games John E. Laird and Michael van Lent. http://ai.eecs.umich.edu/people/laird/papers/AAAI-00.pdf “Intelligent systems such as CGF must possess humanlike expertise in the military domain. Like a human or group of humans in a military organization they must be able to adapt and learn” - G. R. George and F. Cardullo. “Application of Neuro-Fuzzy Systems to Behavioral Representation in Computer Generated Forces”, Proceedings of the Eighth Conference on Computer Generated Forces and Behavioral Representation, Orlando FL, May 11-13 1999, pp. 575-585. 2 3 These include DirectIA here: http://www.directia.com/ 4 See http://www-mddsp.enel.ucalgary.ca/People/dreveny/itchy/bot.html. “Realistic Evaluation of Terrain by Intelligent Natural Agents (Retina)” http://fp2k.redshift.com/rayburge3/BurgessSep03a.pdf. A synopsis of this fascinating research appears in this issue of the M&S Online Journal. 5 An online copy of Turing’s 1950 classic, “On Computing Machinery and Intelligence” is available here: http://www.abelard.org/turpap/turpap.htm 6 See “A Fuzzy Logic Tutorial” (http://www.seattlerobotics.org/encoder/mar98/fuz/flindex.html) and the Fuzzy Logic Archive (http://www.austinlinks.com/Fuzzy/). 7 “A Neural-Net Primer” by André LaMothe; Tricks of the Windows Game Programming Gurus. André LaMothe New York, Sams Publishing 1999. 8 9 Artificial Intelligence: A Modern Approach; Russell & Norvig, Prentice Hall, 1995. 10 Penny Sweetser (is an Australian computer science Ph.D. student also working on AI and Computer Games. She has written a number of excellent papers on the subject available from her web site http://www.itee.uq.edu.au/~penny/index.htm). 11 http://www.cs.northwestern.edu/~forbus/c95gd/2001/Programming%20Objects%20in%20The%20Sims.pdf “Computer Player AI: The New Age.” Pottinger (2003) in July 2003 Game Developer magazine. Not yet available in electronic form. 12 13 http://www.gamasutra.com/features/20030725/dybsand_01.shtml 14 http://slate.msn.com/id/2080814/ 15 http://www.gamasutra.com/features/20001108/laird_pfv.htm “Grant's Secret Service: The Intelligence War from Belmont to Appomattox.” William B. Feis; Lincoln: University of Nebraska Press, 2002. Pp. 330 16