Strategies for Evolving Diverse and Effective Behaviours in Pursuit Domains

Abstract

Gamer engagement with computer opponents is an important aspect of computer games. Players will be bored if computer opponents are predictable, and the game will be monotonous. Computer opponents that are both challenging and exhibit interesting and novel behaviours are ideal. This research explores different strategies that encourage diverse emergent behaviours for evolved intelligent agents, while maintaining good performance with the task at hand. We consider the pursuit domain, which consists of a single predator agent and twenty prey agents. The predator’s controller is evolved through genetic programming, while the preys’ controllers are hand-crafted. The fitness of a solution is calculated as the number of prey captured. Inspired by Lehman and Stanley’s novelty search strategy, the fitness is combined with a diversity score, determined by combining four rudimentary behaviour measurements. We combine these basic scores using the many objective optimization strategy known as “sum of ranks”, which is proven to effectively balance a high number of conflicting objectives in optimization problems. We also examine different population diversity strategies, as well as different weighting schemes for combining fitness and diversity scores. After producing sets of solutions for the above experiments, we manually tabulate higher-level emergent behaviour observed in the evolved predators. The use of K-nearest neighbours (K=32) with population archive, combined with a fitness:diversity weighting of 50:50, gave the best results, as it effectively balanced good fitness performance and diverse emergent behaviour.