Evolving interpretable strategies for zero-sum games
Section snippets
Code metadata
Permanent link to reproducible Capsule: https://doi.org/10.24433/CO.1303516.v1.
Related work
We review related works focused on synthesizing scripts for games. The so-called Dynamic Scripting (DS) is a reinforcement-learning-based technique that synthesizes scripts for zero-sum role-playing games. DS extracts rules according to a learned policy [11], which separates rules following the type of agents related to them. Each rule has an associated weight stating its chance to be inserted in a script. The weights are modified after a match based on their contribution to improving the
RTS games and RTS
Real-time strategy (RTS) games usually aim to fight for resources and eliminate enemy units and buildings within a warfare scenario. The scenarios provided by RTS games can be seen as a testbed for real-time planning, and decision making under uncertainty [21]. The domains in RTS games usually request search algorithms able to find a satisfactory action from a large number of options, with the planning of actions being built within milliseconds [40].
The moves executed in RTS games have features
Script synthesis in zero-sum games
Let be a zero-sum game, where is the set of players, is the set of states, where denotes the set of non-terminal states and the set of terminal states, is the start state of the game, and is the set of actions a player can perform in state .
A script is a strategy represented as a function that returns a legal action for player at state . Player has a utility value of the game rooted at state denoted by . This
Genetic programming to synthesize scripts
In this section, we introduce Gesy, a genetic programming approach to script synthesis. Given a DSL , Gesy searches for the script that approximates a solution to the script synthesis problem defined by Eq. (1). Algorithm 1 describes Gesy. The algorithm receives as input a zero-sum game , a fitness function and a grammar that defines a DSL . The parameters and state the number of generations and population length. The parameter establishes the number of individuals
Empirical methodology
This section presents the DSL designed to define a space of programs for RTS and describe the experiments using it, where we compare and with state-of-the-art approaches in terms of strength of play and computation time. We also analyze the interpretability of the scripts synthesized by our methods.
Conclusions and future works
This paper introduced Gesy, a genetic programming approach to script synthesis for zero-sum games. Gesy approximates a solution to the script synthesis problem by evolving an initial population of scripts through genetic operators. Results on RTS showed that our approach synthesizes competitive scripts in terms of strength of play within a reduced response time than search-based approaches from the literature. Moreover, our approach provides scripts that can be understood and used to fix
CRediT authorship contribution statement
Julian R.H. Mariño: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Drafting the manuscript. Claudio F.M. Toledo: Conception and design of study, Analysis and/or interpretation of data, Revising the manuscript critically for important intellectual content.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This research was partially supported by CNPq and CAPES. The research was carried out using the computational resources of the Center for Mathematical Sciences Applied to Industry (CeMEAI) funded by FAPESP (grant 2013/07375-0).
References (44)
- et al.
Evolutionary learning based simulation optimization for stochastic job shop scheduling problems
Appl. Soft Comput.
(2021) - et al.
Procedural generation of dungeons’ maps and locked-door missions through an evolutionary algorithm validated with players
Expert Syst. Appl.
(2021) - et al.
A pattern mining approach to study strategy balance in RTS games
IEEE Trans. Comput. Intell. AI Games
(2015) - et al.
Portfolio greedy search and simulation for large-scale combat in StarCraft
- L.H.S. Lelis, Stratified Strategy Selection for Unit Control in Real-Time Strategy Games, in: International Joint...
- et al.
Game tree search based on nondeterministic action scripts in real-time strategy games
IEEE Trans. Games
(2017) - N.A. Barriga, M. Stanescu, M. Buro, Combining Strategic Learning and Tactical Search in Real-Time Strategy Games, in:...
- S. Ontañón, M. Buro, Adversarial Hierarchical-Task Network Planning for Complex Real-Time Games, in: Proceedings of the...
Combinatorial multi-armed bandits for real-time strategy games
J. Artificial Intelligence Res.
(2017)- et al.
Action abstractions for combinatorial multi-armed bandit tree search
Online adaptation of game opponent AI with dynamic scripting
Int. J. Intell. Games Simul.
Improving Adaptive Game AI with Evolutionary Learning
Evolving agents for the hanabi 2018 cig competition
Evolving controllers for mario AI using grammar-based genetic programming
Peeking inside the black-box: A survey on explainable artificial intelligence (XAI)
IEEE Access
Domain-specific languages: An annotated bibliography
ACM Sigplan Not.
Cited by (0)
The code (and data) in this article has been certified as Reproducible by Code Ocean: (https://codeocean.com/). More information on the Reproducibility Badge Initiative is available at https://www.elsevier.com/physical-sciences-and-engineering/computer-science/journals.