Skip to main content
SpringerLink
Log in
Book cover

European Conference on Genetic Programming (Part of EvoStar)

EuroGP 2022: Genetic Programming pp 212–227Cite as

Creating Diverse Ensembles for Classification with Genetic Programming and Neuro-MAP-Elites

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13223))

Abstract

Model diversity is essential for ensemble classifiers, which make predictions by combining predictions from multiple simpler models. While ensemble classifiers often outperform single-model classifiers, their success crucially depends on the ensemble’s construction. Genetic programming (GP) is a powerful evolutionary algorithm that can evolve populations of simple classifiers; however, standard GP algorithms produce populations of models with correlated predictions. Recent work in the broader evolutionary computing community has begun focusing on methods for evolving diverse populations, such as MAP-Elites [24], which can evolve populations that are diverse in a low dimensional behavior space. In this work, we demonstrate a novel technique for using MAP-Elites to create diverse GP populations, which can be used as ensemble classifiers. We demonstrate the utility of our framework, which we call Neuro-MAP-Elites, by comparing it with other classification algorithms across a diverse set of classification datasets.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    See github.com/BigTuna08/nme for the code to tune parameters of all models.

References

  1. Beadle, L., Johnson, C.: Semantically driven crossover in genetic programming, pp. 111–116, July 2008

    Google Scholar 

  2. Beadle, L., Johnson, C.: Semantically driven mutation in genetic programming, pp. 1336–1342, May 2009

    Google Scholar 

  3. Berkson, J.: Application of the logistic function to bio-assay. J. Am. Statist. Assoc. 39(227), 357–365 (1944)

    Google Scholar 

  4. Boisvert, S., Sheppard, J.W.: Quality diversity genetic programming for learning decision tree ensembles. In: Hu, T., Lourenço, N., Medvet, E. (eds.) EuroGP 2021. LNCS, vol. 12691, pp. 3–18. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72812-0_1

    Chapter  Google Scholar 

  5. Brameier, M.F., Banzhaf, W.: Linear Genetic Programming, 1st edn. Springer Publishing Company Inc., Cham (2007)

    MATH  Google Scholar 

  6. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Article  Google Scholar 

  7. Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: a survey and categorisation. Inf. Fusion 6(1), 5–20 (2005)

    Article  Google Scholar 

  8. Canaan, R., Togelius, J., Nealen, A., Menzel, S.: Diverse agents for ad-hoc cooperation in Hanabi. In: 2019 IEEE Conference on Games (CoG), pp. 1–8 (2019)

    Google Scholar 

  9. Cardoso, R.P., Hart, E., Kurka, D.B., Pitt, J.V.: Using novelty search to explicitly create diversity in ensembles of classifiers. In: GECCO 2021, pp. 849–857. ACM, New York, NY, USA (2021)

    Google Scholar 

  10. Cully, A., Clune, J., Tarapore, D., Mouret, J.B.: Robots that can adapt like animals. Nature 521, 503–507 (2015)

    Article  Google Scholar 

  11. Dolson, E., Lalejini, A., Ofria, C.: Exploring genetic programming systems with map-elites, August 2018

    Google Scholar 

  12. Fontaine, M.C., Lee, S., Soros, L.B., De Mesentier Silva, F., Togelius, J., Hoover, A.K.: Mapping hearthstone deck spaces through map-elites with sliding boundaries. In: GECCO 2019, pp. 161–169. ACM, New York, NY, USA (2019)

    Google Scholar 

  13. Gagné, C., Sebag, M., Schoenauer, M., Tomassini, M.: Ensemble learning for free with evolutionary algorithms? In: GECCO 2007, pp. 1782–1789. ACM, New York, NY, USA (2007)

    Google Scholar 

  14. Gravina, D., Khalifa, A., Liapis, A., Togelius, J., Yannakakis, G.N.: Procedural content generation through quality diversity (2019)

    Google Scholar 

  15. Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, Cham (2009). https://doi.org/10.1007/978-0-387-84858-7

    Book  MATH  Google Scholar 

  16. Higgins, I., t al.: beta-VAE: learning basic visual concepts with a constrained variational framework. In: ICLR (2017)

    Google Scholar 

  17. Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)

    Article  MathSciNet  Google Scholar 

  18. Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2014)

    Google Scholar 

  19. Koza, J.R.: Genetic programming: automatic programming of computers. EvoNews 1(3), 4–7 (1997)

    Google Scholar 

  20. Krawiec, K., Lichocki, P.: Approximating geometric crossover in semantic space. In: GECCO 2009, pp. 987–994. ACM, New York, NY, USA (2009)

    Google Scholar 

  21. Lehman, J., Stanley, K.O.: Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–223 (2011)

    Article  Google Scholar 

  22. Miller, J.: Cartesian Genetic Programming, vol. 43, June 2003

    Google Scholar 

  23. Moraglio, A., Krawiec, K., Johnson, C.G.: Geometric semantic genetic programming. In: Coello, C.A.C., Cutello, V., Deb, K., Forrest, S., Nicosia, G., Pavone, M. (eds.) PPSN 2012. LNCS, vol. 7491, pp. 21–31. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32937-1_3

    Chapter  Google Scholar 

  24. Mouret, J., Clune, J.: Illuminating search spaces by mapping elites (2015)

    Google Scholar 

  25. Nguyen, Q.U., Hoai, N., O’Neill, M., McKay, R., Galván-López, E.: Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genetic Program. Evol. Mach. 12, 91–119 (2011)

    Article  Google Scholar 

  26. Olson, R.S., La Cava, W., Orzechowski, P., Urbanowicz, R.J., Moore, J.H.: PMLB: a large benchmark suite for machine learning evaluation and comparison. BioData Mining 10(1), 36 (2017)

    Article  Google Scholar 

  27. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  28. Platt, J.C.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in Large Margin Classifiers, pp. 61–74. MIT Press (1999)

    Google Scholar 

  29. Rodrigues, N.M., Batista, J.E., Silva, S.: Ensemble genetic programming. In: Hu, T., Lourenço, N., Medvet, E., Divina, F. (eds.) EuroGP 2020. LNCS, vol. 12101, pp. 151–166. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-44094-7_10

    Chapter  Google Scholar 

  30. Sha, C., Cuperlovic-Culf, M., Hu, T.: Smile: systems metabolomics using interpretable learning and evolution. BMC Bioinform. 22(1), 284 (2021)

    Article  Google Scholar 

  31. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)

    Article  Google Scholar 

  32. Vassiliades, V., Chatzilygeroudis, K., Mouret, J.: Using centroidal voronoi tessellations to scale up the multidimensional archive of phenotypic elites algorithm. IEEE Trans. Evol. Comput. 22(4), 623–630 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Memorial University of Newfoundland, St. John’s, NL, Canada

    Kyle Nickerson & Antonina Kolokolova

  2. School of Computing, Queen’s University, Kingston, ON, Canada

    Ting Hu

Authors
  1. Kyle Nickerson
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Antonina Kolokolova
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ting Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyle Nickerson .

Editor information

Editors and Affiliations

  1. University of Trieste, Trieste, Italy

    Eric Medvet

  2. Universidade Federal de Minas Gerais, Belo Horizonte, Minas Gerais, Brazil

    Gisele Pappa

  3. Victoria University of Wellington, Wellington, New Zealand

    Bing Xue

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nickerson, K., Kolokolova, A., Hu, T. (2022). Creating Diverse Ensembles for Classification with Genetic Programming and Neuro-MAP-Elites. In: Medvet, E., Pappa, G., Xue, B. (eds) Genetic Programming. EuroGP 2022. Lecture Notes in Computer Science, vol 13223. Springer, Cham. https://doi.org/10.1007/978-3-031-02056-8_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-02056-8_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-02055-1

  • Online ISBN: 978-3-031-02056-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation