Evolving interpretable strategies for zero-sum games

https://doi.org/10.1016/j.asoc.2022.108860Get rights and content

Highlights

  • Synthesizing scripts for zero-sum games.

  • The proposition of a genetic programming method.

  • Evolve a population of scripts through generations.

  • Searching for scripts that maximize victories.

  • Computational comparison against state-of-the-art search-based methods.

Abstract

The present paper introduces Gesy, a genetic programming approach to script synthesis for zero-sum games. We will explore the sum-zero game context in Real-Time Strategy (RTS) games, where players must look for strategies (planning of actions) to maximize their gains or minimize their losses. The goal is to solve the script synthesis problem, which demands the synthesis of a computer program from a space of programs defined by a Domain-Specific Language (DSL). The synthesized program must encode a practical strategy for zero-sum games. Empirical results validate Gesy using the μRTS platform, an academic test bed game that presents the main features found in RTS commercial games. The results show that our method provides interpretable strategies that are competitive with state-of-the-art search-based approaches in terms of play strength. Moreover, once synthesized, scripts require only a tiny fraction of the time needed by search-based methods to decide on the agent’s next action.

Section snippets

Code metadata

Permanent link to reproducible Capsule: https://doi.org/10.24433/CO.1303516.v1.

Related work

We review related works focused on synthesizing scripts for games. The so-called Dynamic Scripting (DS) is a reinforcement-learning-based technique that synthesizes scripts for zero-sum role-playing games. DS extracts rules according to a learned policy [11], which separates rules following the type of agents related to them. Each rule has an associated weight stating its chance to be inserted in a script. The weights are modified after a match based on their contribution to improving the

RTS games and μRTS

Real-time strategy (RTS) games usually aim to fight for resources and eliminate enemy units and buildings within a warfare scenario. The scenarios provided by RTS games can be seen as a testbed for real-time planning, and decision making under uncertainty [21]. The domains in RTS games usually request search algorithms able to find a satisfactory action from a large number of options, with the planning of actions being built within milliseconds [40].

The moves executed in RTS games have features

Script synthesis in zero-sum games

Let G=(N,S,sinit,A,R,T) be a zero-sum game, where N={i,i} is the set of players, S=DF is the set of states, where D denotes the set of non-terminal states and F the set of terminal states, sinitD is the start state of the game, and A(s) is the set of actions a player can perform in state sD.

A script is a strategy represented as a function p(s) that returns a legal action aAi for player i at state s. Player i has a utility value of the game rooted at state s denoted by V(s,pi,pi). This

Genetic programming to synthesize scripts

In this section, we introduce Gesy, a genetic programming approach to script synthesis. Given a DSL D, Gesy searches for the script piD that approximates a solution to the script synthesis problem defined by Eq. (1). Algorithm 1 describes Gesy.

The algorithm receives as input a zero-sum game G, a fitness function Ψ and a grammar G that defines a DSL D. The parameters g and n state the number of generations and population length. The parameter f establishes the number of individuals

Empirical methodology

This section presents the DSL designed to define a space of programs for μRTS and describe the experiments using it, where we compare Gesy and Gesys with state-of-the-art approaches in terms of strength of play and computation time. We also analyze the interpretability of the scripts synthesized by our methods.

Conclusions and future works

This paper introduced Gesy, a genetic programming approach to script synthesis for zero-sum games. Gesy approximates a solution to the script synthesis problem by evolving an initial population of scripts through genetic operators. Results on μRTS showed that our approach synthesizes competitive scripts in terms of strength of play within a reduced response time than search-based approaches from the literature. Moreover, our approach provides scripts that can be understood and used to fix

CRediT authorship contribution statement

Julian R.H. Mariño: Conception and design of study, Acquisition of data, Analysis and/or interpretation of data, Drafting the manuscript. Claudio F.M. Toledo: Conception and design of study, Analysis and/or interpretation of data, Revising the manuscript critically for important intellectual content.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

This research was partially supported by CNPq and CAPES. The research was carried out using the computational resources of the Center for Mathematical Sciences Applied to Industry (CeMEAI) funded by FAPESP (grant 2013/07375-0).

References (44)

  • GhasemiA. et al.

    Evolutionary learning based simulation optimization for stochastic job shop scheduling problems

    Appl. Soft Comput.

    (2021)
  • PereiraL.T. et al.

    Procedural generation of dungeons’ maps and locked-door missions through an evolutionary algorithm validated with players

    Expert Syst. Appl.

    (2021)
  • BoscG. et al.

    A pattern mining approach to study strategy balance in RTS games

    IEEE Trans. Comput. Intell. AI Games

    (2015)
  • ChurchillD. et al.

    Portfolio greedy search and simulation for large-scale combat in StarCraft

  • L.H.S. Lelis, Stratified Strategy Selection for Unit Control in Real-Time Strategy Games, in: International Joint...
  • BarrigaN.A. et al.

    Game tree search based on nondeterministic action scripts in real-time strategy games

    IEEE Trans. Games

    (2017)
  • N.A. Barriga, M. Stanescu, M. Buro, Combining Strategic Learning and Tactical Search in Real-Time Strategy Games, in:...
  • S. Ontañón, M. Buro, Adversarial Hierarchical-Task Network Planning for Complex Real-Time Games, in: Proceedings of the...
  • OntañónS.

    Combinatorial multi-armed bandits for real-time strategy games

    J. Artificial Intelligence Res.

    (2017)
  • MoraesR.O. et al.

    Action abstractions for combinatorial multi-armed bandit tree search

  • Z. Yang, S. Ontanón, Guiding Monte Carlo Tree Search by Scripts in Real-Time Strategy Games, in: Proceedings of the...
  • T. Chakraborti, A. Kulkarni, S. Sreedharan, D.E. Smith, S. Kambhampati, Explicability? Legibility? Predictability?...
  • SpronckP. et al.

    Online adaptation of game opponent AI with dynamic scripting

    Int. J. Intell. Games Simul.

    (2004)
  • PonsenM. et al.

    Improving Adaptive Game AI with Evolutionary Learning

    (2004)
  • A. Dahlbom, L. Niklasson, Goal-Directed Hierarchical Dynamic Scripting for RTS Games, in: Proceedings of the AAAI...
  • A. Benbassat, M. Sipper, Evolving board-game players with genetic programming, in: Proceedings of the 13th Annual...
  • CanaanR. et al.

    Evolving agents for the hanabi 2018 cig competition

  • De FreitasJ.M. et al.

    Evolving controllers for mario AI using grammar-based genetic programming

  • E. Butler, E. Torlak, Z. Popović, Synthesizing interpretable strategies for solving puzzle games, in: Proceedings of...
  • AdadiA. et al.

    Peeking inside the black-box: A survey on explainable artificial intelligence (XAI)

    IEEE Access

    (2018)
  • M. Jacob, S. Devlin, K. Hofmann, “It’s Unwieldy and It Takes a Lot of Time” — Challenges and Opportunities for Creating...
  • Van DeursenA. et al.

    Domain-specific languages: An annotated bibliography

    ACM Sigplan Not.

    (2000)
  • Cited by (0)

    The code (and data) in this article has been certified as Reproducible by Code Ocean: (https://codeocean.com/). More information on the Reproducibility Badge Initiative is available at https://www.elsevier.com/physical-sciences-and-engineering/computer-science/journals.

    View full text