Skip to main content

Advertisement

SpringerLink
Log in

Fitness approximation for bot evolution in genetic programming

Lessons learned from the UT2004™ computer game

  • Methodologies and Application
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Estimating the fitness value of individuals in an evolutionary algorithm in order to reduce the computational expense of actually calculating the fitness has been a classical pursuit of practitioners. One area which could benefit from progress in this endeavour is bot evolution, i.e. the evolution of non-playing characters in computer games. Because assigning a fitness value to a bot (or rather, the decision tree that controls its behaviour) requires playing the game, the process is very costly. In this work, we introduce two major contributions to speed up this process in the computer game Unreal Tournament 2004™. Firstly, a method for fitness value approximation in genetic programming which is based on the idea that individuals that behave in a similar fashion will have a similar fitness. Thus, similarity of individuals is taken at the performance level, in contrast to commonly employed approaches which are either based on similarity of genotypes or, less frequently, phenotypes. The approximation performs a weighted average of the fitness values of a number of individuals, attaching a confidence level which is based on similarity estimation. The latter is the second contribution of this work, namely a method for estimating the similarity between individuals. This involves carrying out a number of tests consisting of playing a ‘static’ version of the game (with fixed inputs) with the individuals whose similarity is under evaluation and comparing the results. Because the tests involve a limited version of the game, the computational expense of the similarity estimation plus that of the fitness approximation is much lower than that of directly calculating the fitness. The success of the fitness approximation by similarity estimation method for bot evolution in UT2K4 allows us to expect similar results in environments that share the same characteristics.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. For more information about the game, the reader is referred to the official website: http://www.unrealtournament.com.

  2. Another possible way of doing this would be to prepare in advance a set of representative inputs for this purpose instead of generating them at random. However, for the environment in the UT2004 game, this would be too complex a challenge.

  3. See http://pogamut.cuni.cz for details.

  4. Note that the similarity estimation part involves testing each individual 100 times against each one of the 100 individuals in the Y set.

References

  • Akrour R, Schoenauer M, Sebag M (2011) Preference-based policy learning. In: Gunopulos D, Hofmann T, Malerba D, Vazirgiannis M (eds) ECML/PKDD—part I. LNCS, vol 6911. Springer, Berlin, pp 12–27

  • Alfaro-Cid E, Merelo JJ, de Vega FF, Esparcia-Alcázar AI, Sharman K (2010) Bloat control operators and diversity in genetic programming: a comparative study. Evol Comput 18(2):305–332

    Article  Google Scholar 

  • Booker AJ, Dennis JE Jr, Frank PD, Serafini DB, Torczon V, Trosset MW (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Optim 17(1):1–13

    Article  Google Scholar 

  • Bouzarkouna Z, Auger A, Ding D-Y (2011) Local-meta-model CMA-ES for partially separable functions. In: Krasnogor N et al (ed) Genetic and evolutionary computation conference (GECCO). ACM Press, New York, pp 869–876

  • Bouzarkouna Z, Auger A, Ding DY (2010) Investigating the local meta-model CMA-ES for large population sizes. In: Di Chio C et al (ed) Proceedings on the applications of evolutionary computing, part I. EvoApplications 2010, Istanbul, Turkey, 7–9 April 2010. LNCS, vol 6024. Springer, Berlin, pp 402–411

  • Branke J, Schmidt C (2005) Faster convergence by means of fitness estimation. Soft Comput 9(1):13–20

    Article  Google Scholar 

  • Büche D, Schraudolph NN, Koumoutsakos P (2005) Accelerating evolutionary algorithms with Gaussian process fitness function models. IEEE Trans Syst Man Cybern 35(2):183–194

    Article  Google Scholar 

  • Burke E, Gustafson S, Kendall G, Krasnogor N (2002) Advanced population diversity measures in genetic programing. In: Merelo JJ et al (ed) Parallel problem solving from nature. Proceedings of the 7th conference. LNCS, vol 2439. Springer, Berlin, pp 341–350

  • Burke EK, Gustafson S, Kendall G (2004) Diversity in genetic programming: an analysis of measures and correlation with fitness. IEEE Trans Evol Comput 8(1):47–62

    Article  Google Scholar 

  • Carpenter W, Barthelemy J (1993) A comparison of polynomial approximations and artificial neural nets as response surfaces. Struct Multidiscip Optim 5:166–174

    Article  Google Scholar 

  • Custódio FL, Barbosa HJC, Dardenne LE (2010) Full-atom ab initio protein structure prediction with a genetic algorithm using a similarity-based surrogate model. In: IEEE World congress on computational intelligence, WCCI 2010, pp 2554–2561

  • Emmerich M, Giotis A, Özdemir M, Bäck T, Giannakoglou K (2002) Metamodel-assisted evolution strategies. In: Merelo JJ et al (ed) Parallel problem solving from nature. Proceedings of the 7th conference. LNCS, vol 2439. Springer, Berlin, pp 361–370

  • Esparcia-Alcázar AI, Martínez-García AI, Mora A, Merelo JJ, García-Sánchez P (2010) Controlling bots in a first person shooter game using genetic algorithms. In: Proceedings of the 2010 IEEE World congress on computational intelligence, Barcelona, Spain, 18–23 July 2010. IEEE Press, New York

  • Fonseca LG, Barbosa HJC, Lemonge ACC (2009) A similarity-based surrogate model for enhanced performance in genetic algorithms. Opsearch 46(1):89–107

    Article  MathSciNet  MATH  Google Scholar 

  • Forrester AIJ, Keane AJ, Bressloff NW (2006) Design and analysis of “noisy” computer experiments. AIAA J 44(10):2331–2339

    Article  Google Scholar 

  • Giacobini M, Tomassini M, Vanneschi L (2002) Limiting the number of fitness cases in genetic programming using statistics. In: Merelo JJ et al (ed) Parallel problem solving from nature. Proceedings of the 7th conference. LNCS, vol 2439. Springer, Berlin, pp 371-380

  • Gomez FJ (2009) Sustaining diversity using behavioral information distance. In: Proceedings of the 11th annual conference on genetic and evolutionary computation, GECCO ’09, pp 113–120, New York, NY, USA. ACM, New York

  • Gustafson S, Vanneschi L (2008) Crossover-based tree distance in genetic programming. IEEE Trans Evol Comput 12(4):506 –524

    Article  Google Scholar 

  • Hien NT, Hoai NX (2006) A brief overview of population diversity measures in genetic programming. In: Proceedings of the third Asian-Pacific workshop on genetic programming, pp 128–139

  • Jackson D (2010) Phenotypic diversity in initial genetic programming populations. In: Esparcia-Alcazar AI et al (ed) Proceedings of the 13th European conference on genetic programming, EuroGP 2010. LNCS, vol 6021, Istanbul, 7–9 April 2010. Springer, Berlin, pp 98–109

  • Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. Soft Comput 9(1):3–12

    Article  Google Scholar 

  • Jones DR (2001) A taxonomy of global optimization methods based on response surfaces. J Global Optim 21(4):345–383. ISSN 0925-5001

    Google Scholar 

  • Kadlec R (2008) Evolution of intelligent agent behaviour in computer games. Master’s thesis, MFF UK, Czech Republic

  • Langdon WB, Soule T, Poli R, Foster JA (1999) The evolution of size and shape. In: Spector L, Langdon WB, O’Reilly U-M, Angeline PJ (eds) Advances in genetic programming 3, Chap 8. MIT Press, Cambridge, pp 163–190

  • Llorà X, Sastry K, Goldberg DE, Gupta A, Lakshmi L (2005) Combating user fatigue in iGAs: partial ordering, support vector machines, and synthetic fitness. In: Beyer H-G, O’Reilly U-M (eds) Proceedings of the genetic and evolutionary computation conference, GECCO, pp 25–29

  • Loshchilov I, Schoenauer M, Sebag M (2012) Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy. In: Proceedings of the 14th international conference on genetic and evolutionary computation conference, GECCO ’12, New York, NY, USA, 2012. ACM, New York, pp 321–328

  • Mattiussi C, Waibel M, Floreano D (2004) Measures of diversity for populations and distances between individuals with highly reorganizable genomes. Evol Comput 12(4):495–515

    Google Scholar 

  • Mora AM, Montoya R, Merelo JJ, García-Sánchez P, Castillo PA, Jiménez- Laredo JL, Martínez-García AI, Esparcia-Alcázar AI (2010) Evolving bot’s AI in unreal. In: Di Chio C et al (ed) Proceedings on the applications of evolutionary computing, part I, EvoApplications 2010, Istanbul, Turkey, 7–9 April 2010. LNCS, vol 6024. Springer, Berlin, pp 170–179

  • Moravec J (2008) Fitness landscape in genetic algorithms. In: PhD conference ’08, ICS Prague. MatfyzPress, Prague, pp 69–75

  • Ong YS, Nair PB, Keane AJ (2003) Evolutionary optimization of computationally expensive problems via surrogate modelling. Am Inst Aeronaut Astron 41(4):687–696

    Article  Google Scholar 

  • Patrick M (2010) Online evolution in unreal tournament 2004. In: Proceedings of the 2010 IEEE conference on computational intelligence and games, CIG’10, Copenhagen, Denmark, 18–21 August 2010. IEEE Press, New York

  • Rasheed K, Ni X, Vattam S (2005) Comparison of methods for developing dynamic reduced models for design optimization. Soft Comput 9(1):29–37

    Article  Google Scholar 

  • Ratle A (1998) Accelerating the convergence of evolutionary algorithms by fitness landscape approximation. In: Eiben AE et al (eds) Proceedings of the 5th international conference on parallel problem solving from nature, PPSN V. LNCS, vol 1498, Amsterdam, The Netherlands.Springer, Berlin, pp 87–96

  • Rosca JP (1995) Entropy-driven adaptive representation. In: Rosca JP (ed) Proceedings of the workshop on genetic programming: from theory to real-world applications, pages 23–32, Tahoe City, California, USA

  • Sareni B, Krähenbühl L (1998) Fitness sharing and niching methods revisited. IEEE Trans Evol Comput 2(3):97–106

    Article  Google Scholar 

  • Sastry K, Goldberg DE, Pelikan M (2001) Don’t evaluate, inherit. IlliGAL Report No. 2001013

  • Schmidt MD, Lipson H (2008) Coevolution of fitness predictors. IEEE Trans Evol Comput 12(6):736–749

    Article  Google Scholar 

  • Small R, Congdon CB (2009) Agent Smith: towards an evolutionary rule-based agent for interactive dynamic games. In: IEEE Congress on evolutionary computation, 2009, CEC ’09, pp 660 –666

  • Tenne Y, Goh C-K (eds) (2010) Computational intelligence in expensive optimization problems. Adaptation, learning, and optimization, vol 2. Springer, Berlin

  • van Hoorn N, Togelius J, Schmidhuber J (2009) Hierarchical controller learning in a first-person shooter. In: IEEE symposium on computational intelligence and games, 2009, CIG 2009, pp 294 –301

  • Wyns B, De Bruyne P, Boullart L (2006) Characterizing diversity in genetic programming. In: Collet P et al (ed) Proceedings of the 9th European conference on genetic programming. LNCS, vol 3905, Budapest, Hungary, 10–12 April 2006. Springer, Berlin

  • Yu T, Davis L, Baydar C, Roy R (eds) (2008) Evolutionary computation in practice. Springer, Berlin

  • Zhou Z, Ong YS, Lim MH, Lee BS (2006) Memetic algorithm using multi-surrogates for computationally expensive optimization problems. IEEE Trans Evol Comput 11(10):957–971

    Google Scholar 

Download references

Acknowledgments

The research reported in this paper was supported by the Grant Agency of the Czech Republic, Grants 201/09/H057, ICC/08/E018 (Jaroslav Moravec) and by the Spanish Ministry of Science and Innovation, Project NoHNES—Non-Hierarchical Network Evolutionary System, ref. TIN2007-68083-C02. (Anna I. Esparcia-Alcázar). The authors wish to thank the annonymous reviewers for their helpful comments.

Author information

Authors and Affiliations

  1. S2 Grupo, Valencia, Spain

    Anna I. Esparcia-Alcázar

  2. Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic

    Jaroslav Moravec

Authors
  1. Anna I. Esparcia-Alcázar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jaroslav Moravec
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anna I. Esparcia-Alcázar.

Additional information

Communicated by F. Herrera.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Esparcia-Alcázar, A.I., Moravec, J. Fitness approximation for bot evolution in genetic programming. Soft Comput 17, 1479–1487 (2013). https://doi.org/10.1007/s00500-012-0965-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-012-0965-7

Keywords

Search

Navigation