Using Evolution and Deep Learning to Generate Diverse Intelligent Agents

Abstract

Emergent behaviour arises from the interactions between individual components of a system, rather than being explicitly programmed or designed. The evolution of interesting emergent behaviour in intelligent agents is important when evolving non-playable characters in video games. Here, we use genetic programming (GP) to evolve intelligent agents in a predator-prey simulation. A main goal is to evolve predator agents that exhibit interesting and diverse behaviours. First, we train a convolutional neural network (CNN) to recognize “generic” prey behaviour, as recorded by an image trace of a predator’s movement. A training set for 6 generic behaviours was used to train the CNN. A training accuracy of 98% was obtained, and a validation performance of 90%. Experiments were then performed that merge the CNN with GP fitness. In one experiment, the CNN’s classification values are used as a “diversity score” which, when weighted with the fitness score, allow both agent quality and diversity to be considered. In another experiment, we use the CNN classification score to encourage the evolution of one of the known classes of behaviours. Results were that this trained behaviour was indeed more frequently evolved, compared to GP runs using fitness alone. One conclusion is that machine learning techniques are a powerful tool for the automated generation of diverse, high-quality intelligent agents.