Adaptive Intelligent Agent in RTS Games
Using Case-Based Planning and Reinforcement Learning
26/06/2010
Faculty of Computer and Information Science – Ain Shams University
Abdelrahman Al-Ogail
Ahmed Atta
Amr Saqr
Omar Enayet
Supervision
Prof. Dr. Mostafa Aref
Dr. Ibrahim Fathy
Adaptive Intelligent Agent in RTS Games
ABOUT PROJECT MEMBERS
Abdelrahman Al-Ogail is graduated from computer science department. He’s interested in
Artificial Intelligence especially in machine learning and planning, Game Development and
Cryptography. For further contact: abdelrahman.ogail@hotmail.com.
Ahmed Atta is graduated from computer science department. He’s interested in software
engineering and cryptography. For further contact: ahmad-atta@live.com.
Amr Saqr is graduated from scientific computing department. He’s interested in algorithms,
problem solving and ACM Competitions. For further contact: amrsaqr@hotmail.com
Omar Enayet is graduated from computer science department. He is passionate about Artificial
Intelligence as well as applying different AI Techniques to Games. He enjoys doing research &
tackling new problems. He is also trying to be a good lecturer. He's looking forward to placing
another brick in the world of artificial general intelligence. For further contact:
omar.enayet@hotmail.com
2|Page
Adaptive Intelligent Agent in RTS Games
ACKNOWLEDGMENTS
We'd like to thank our supervisors, Prof. Dr. Mostafa Aref and Dr. Ibrahim Fathy, for their skill,
insight, and patience in leading this project. The meetings, discussions and encouragements
were unlimited and directly affected our performance and motivation duo to the project.
We'd also like to express our appreciation to Eric Kok for providing us with his Masters source
code which helped us a lot in implementation and Santi Ontañón for answering a lot of our
questions.
Further, thanks to the faculty stuff specially Dr. Haythem El-Messiry for his encouragement
through Cairo ICT 2010 Conference where we were provided a booth for viewing the project and
Prof. Dr. Mostafa Gad for his technical advices.
Finally, we'd like to thank our colleagues Hassan Ibrahim and Atef Hashem for their effective
help in the project, also Karim Mohammed, Omar Aymen, Mohammed Sharaf, Amr Mahdi,
Muhamad Hesham, Rehab Reda, Yassmine Taha and Omnia Nour for their revision to the
documentation and precious help.
On that note, remember that the most important goal of good AI in games is to make games
more fun. Play on!
3|Page
Adaptive Intelligent Agent in RTS Games
PROBLEM DESCRIPTION
Computer opponent in RTS games behaves in static boring, and inefficient manner. It has no
ability to think, learn, plan and adapt itself to what’s happening around. Unfortunately, several
computer opponents are not able to share and exchange their knowledge and experience each
other.
In this document the problem of automated learning and planning during the game shall be
studied.
4|Page
Adaptive Intelligent Agent in RTS Games
ABSTRACT
Some domains, such as real-time strategy (RTS) games, pose several challenges to traditional
planning and machine learning techniques. In this document, we present a novel on-line casebased planning architecture that addresses some of these problems. Our architecture addresses
issues of on-line plan execution, interleaved planning and execution and on-line plan adaptation.
We also introduce IStrategizer system, which implements this architecture in order to play
Wargus (an open source clone of the well-known RTS game Warcraft II).We present empirical
evaluation of the performance of IStrategizer and show that it successfully learns to play the
Wargus game.
5|Page
Adaptive Intelligent Agent in RTS Games
CONTENTS
Contents
List of Figures
List of Tables
1. Introduction
1.1. Motivation.
1.2. Goals.
1.3. Structure of document.
2. Background
2.1. Real-time strategy games.
2.2. Machine learning techniques.
2.3. Requirements for game environment.
2.4. Choice of game environment.
2.5. Development of game environment.
2.6. Learning in RTS games.
3. I-Strategizer design
3.1. Architecture.
3.2. Case representation.
3.3. Case acquisition: Learning from Human Demonstration [OFFLINE-PHASE].
3.4. Case retrieval: Situation Assessment and Selection Policy.
3.5. Online plan expansion & execution.
3.6. Case adaptation.
3.7. Case revision: SARSA(𝜆) Learning with Eligibility Traces [ONLINE-PHASE].
3.8. Case abstraction/concretization.
3.9. Game preceptors.
3.10. Case-base editor.
4. Testing and results
4.1. Abstraction/Concretization results.
4.2. Situation Assessment results.
5. Conclusion and further work
5.1. Further work.
5.1.1. Cooperative AI Agents.
5.1.2. Finding solutions based on local and global past experience.
5.1.3. Strategy visualization tool.
6|Page
Adaptive Intelligent Agent in RTS Games
5.1.4. Generic situation assessment.
5.1.5. Weighted game state similarity.
5.1.6. Online I-Strategizers.
5.1.7. Generic Abstraction/Concretization.
Appendices
A. Project requirements.
References
7|Page
Adaptive Intelligent Agent in RTS Games
WEB RESOURCES
To get introduced for the whole project journey, evolution, technical summaries, presentations,
discussions, meeting minutes and others visit project blog:
http://rtsairesearch.wordpress.com/
For full materials of papers, technical summaries, documentations, articles, external links and
running version of WARGUS (our test best) use the repository link:
svn checkout http://rtsairesearch.googlecode.com/svn/trunk/ rtsairesearch-read-only
For downloading the latest source code for the I-Strategizer Project, please use the following:
svn checkout http://istrategizer.googlecode.com/svn/trunk/ istrategizer-read-only
8|Page