Skip to main content
SpringerLink
Log in

On the importance of specialists for lexicase selection

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Lexicase parent selection filters the population by considering one random training case at a time, eliminating any individual with an error for the current case that is worse than the best error of any individual in the selection pool, until a single individual remains. This process often stops before considering all training cases, meaning that it will ignore the error values on any cases that were not yet considered. Lexicase selection can therefore select specialist individuals that have high errors on some training cases, if they have low errors on others and those errors come near the start of the random list of cases used for the parent selection event in question. We hypothesize here that selecting such specialists, which may have high total error, plays an important role in lexicase selection’s observed performance advantages over error-aggregating parent selection methods such as tournament selection, which select specialists less frequently. We conduct experiments examining this hypothesis, and find that lexicase selection’s performance and diversity maintenance degrade when we deprive it of the ability to select specialists. We also conduct experiments with a form of tournament selection that has been modified to allow for the selection of specialists, and find that it performs better than ordinary tournament selection, but not as well as lexicase selection. These findings, and other data that we present here, help explain the improved performance of lexicase selection compared to tournament selection, and suggest that specialists help drive evolution with lexicase selection toward global solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. We assume lower errors are better, meaning the best error on a training case is the lowest one.

  2. This article is an expanded version of a paper presented at the 2019 Genetic and Evolutionary Computation Conference, which was awarded Best Paper in the Genetic Programming track [15].

  3. https://github.com/lspector/Clojush.

  4. https://git.io/Je8BR.

  5. Figure 1, which plots the probability distribution defined by Eq. 1, is useful to visualize these cumulative probabilities.

References

  1. S. Aenugu, L. Spector, Lexicase selection in learning classifier systems. in GECCO ’19: Proceedings of the Genetic and Evolutionary Computation Conference, (ACM, Prague, Czech Republic, 2019) pp. 356–364. https://doi.org/10.1145/3321707.3321828

  2. T. Bäck, Selective pressure in evolutionary algorithms: a characterization of selection mechanisms. in Proceedings of the First IEEE Conference on Evolutionary Computation, 1994. IEEE World Congress on Computational Intelligence, vol. 1, pp. 57–62 (1994). https://doi.org/10.1109/ICEC.1994.350042

  3. T. Blickle, L. Thiele, A mathematical analysis of tournament selection. in Proceedings of the 6th International Conference on Genetic Algorithms (Morgan Kaufmann Publishers Inc., San Francisco, CA, USA 1995), pp. 9–16. http://dl.acm.org/citation.cfm?id=645514.658088

  4. V.V. de Melo, D.V. Vargas, W. Banzhaf, Batch tournament selection for genetic programming: the quality of lexicase, the speed of tournament. in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’19 (ACM, New York, NY, USA, 2019), pp. 994–1002. https://doi.org/10.1145/3321707.3321793. http://doi.acm.org/10.1145/3321707.3321793

  5. A.J. Ferguson, J.G. Hernandez, D. Junghans, A. Lalejini, E. Dolson, C. Ofria, Characterizing the effects of random subsampling and dilution on lexicase selection, in Genetic Programming Theory and Practice XVII, ed. by W. Banzhaf, E. Goodman, L. Sheneman, L. Trujillo, B. Worzel (East Lansing, Michigan, 2019)

    Google Scholar 

  6. S. Forstenlechner, D. Fagan, M. Nicolau, M. O’Neill, A grammar design pattern for arbitrary program synthesis problems in genetic programming. in EuroGP 2017: Proceedings of the 20th European Conference on Genetic Programming, eds. by M. Castelli, J. McDermott, L. Sekanina, LNCS, vol. 10196. (Springer, Amsterdam, 2017), pp. 262–277. https://doi.org/10.1007/978-3-319-55696-3_17

  7. S. Forstenlechner, D. Fagan, M. Nicolau, M. O’Neill, Extending program synthesis grammars for grammar-guided genetic programming. in 15th International Conference on Parallel Problem Solving from Nature. eds. by A. Auger, C.M. Fonseca, N. Lourenco, P. Machado, L. Paquete, D. Whitley, LNCS, vol. 11101, (Springer, Coimbra, Portugal, 2018), pp. 197–208. https://doi.org/10.1007/978-3-319-99253-2_16. https://www.springer.com/gp/book/9783319992587

  8. S. Forstenlechner, D. Fagan, M. Nicolau, M. O’Neill, Towards effective semantic operators for program synthesis in genetic programming. in GECCO ’18: Proceedings of the Genetic and Evolutionary Computation Conference (ACM, Kyoto, Japan, 2018), pp. 1119–1126. https://doi.org/10.1145/3205455.3205592

  9. S. Forstenlechner, D. Fagan, M. Nicolau, M. O’Neill, Towards understanding and refining the general program synthesis benchmark suite with genetic programming, in 2018 IEEE Congress on Evolutionary Computation (CEC), ed. by M. Vellasco (IEEE, Rio de Janeiro, 2018)

    Google Scholar 

  10. T. Helmuth, N.F. McPhee, E. Pantridge, L. Spector, Improving generalization of evolved programs through automatic simplification. in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’17 (ACM, Berlin, Germany, 2017), pp. 937–944. https://doi.org/10.1145/3071178.3071330. http://doi.acm.org/10.1145/3071178.3071330

  11. T. Helmuth, N.F. McPhee, L. Spector, Lexicase selection for program synthesis: a diversity analysis. in Genetic Programming Theory and Practice XIII, Genetic and Evolutionary Computation eds. by R. Riolo, W.P. Worzel, M. Kotanchek, A. Kordon (Springer, Ann Arbor, USA, 2015). https://doi.org/10.1007/978-3-319-34223-8. http://www.springer.com/us/book/9783319342214

  12. T. Helmuth, N.F. McPhee, L. Spector, Effects of lexicase and tournament selection on diversity recovery and maintenance. in GECCO ’16 Companion: Proceedings of the Companion Publication of the 2016 Annual Conference on Genetic and Evolutionary Computation (ACM, Denver, Colorado, USA, 2016), pp. 983–990. https://doi.org/10.1145/2908961.2931657

  13. T. Helmuth, N.F. McPhee, L. Spector, The impact of hyperselection on lexicase selection. in GECCO ’16: Proceedings of the 2016 Annual Conference on Genetic and Evolutionary Computation, ed. by T. Friedrich (ACM, Denver, USA, 2016), pp 717–724. https://doi.org/10.1145/2908812.2908851

  14. T. Helmuth, N.F. McPhee, L. Spector, Program synthesis using uniform mutation by addition and deletion. in Proceedings of the Genetic and Evolutionary Computation Conference, GECCO ’18 (ACM, Kyoto, Japan, 2018), pp 1127–1134. https://doi.org/10.1145/3205455.3205603. http://doi.acm.org/10.1145/3205455.3205603

  15. T. Helmuth, E. Pantridge, L. Spector, Lexicase selection of specialists. in GECCO ’19: Proceedings of the Genetic and Evolutionary Computation Conference (ACM, Prague, Czech Republic, 2019), pp 1030–1038. https://doi.org/10.1145/3321707.3321875. https://dl.acm.org/citation.cfm?doid=3321707.3321875

  16. T. Helmuth, L. Spector, General program synthesis benchmark suite. in GECCO ’15: Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation (ACM, Madrid, Spain, 2015), pp. 1039–1046. https://doi.org/10.1145/2739480.2754769. http://doi.acm.org/10.1145/2739480.2754769

  17. T. Helmuth, L. Spector, J. Matheson, Solving uncompromising problems with lexicase selection. IEEE Trans. Evol. Comput. 19(5), 630–643 (2015). https://doi.org/10.1109/TEVC.2014.2362729

    Article  Google Scholar 

  18. T. Helmuth, L. Spector, N.F. McPhee, S. Shanabrook, Linear genomes for structured programs. in Genetic Programming Theory and Practice XIV, Genetic and Evolutionary Computation (Springer, Ann Arbor, USA, 2016)

  19. E. Hemberg, J. Kelly, U.M. O’Reilly, On domain knowledge and novelty to improve program synthesis performance with grammatical evolution. in GECCO ’19: Proceedings of the Genetic and Evolutionary Computation Conference (ACM, Prague, Czech Republic, 2019), pp. 1039–1046. https://doi.org/10.1145/3321707.3321865

  20. J.G. Hernandez, A. Lalejini, E. Dolson, C. Ofria, Random subsampling improves performance in lexicase selection. in GECCO ’19: Proceedings of the Genetic and Evolutionary Computation Conference Companion (ACM, Prague, Czech Republic, 2019), pp. 2028–2031. https://doi.org/10.1145/3319619.3326900

  21. D. Jackson, Promoting phenotypic diversity in genetic programming. in PPSN 2010 11th International Conference on Parallel Problem Solving From Nature, eds. by R. Schaefer, C. Cotta, J. Kolodziej, G. Rudolph. Lecture Notes in Computer Science, vol. 6239 (Springer, Krakow, Poland, 2010), pp. 472–481

  22. L. Jundt, T. Helmuth, Comparing and combining lexicase selection and novelty search. in GECCO ’19: Proceedings of the Genetic and Evolutionary Computation Conference (ACM, Prague, Czech Republic, 2019), pp. 1047–1055. https://doi.org/10.1145/3321707.3321787. https://dl.acm.org/citation.cfm?doid=3321707.3321787

  23. J. Kelly, E. Hemberg, U.M. O’Reilly, Improving genetic programming with novel exploration—exploitation control. in EuroGP 2019: Proceedings of the 22nd European Conference on Genetic Programming, eds. by L. Sekanina, T. Hu, N. Lourenço, H. Richter, P. García-Sánchez (Springer International Publishing, 2019), pp. 64–80

  24. W. La Cava, T. Helmuth, L. Spector, J.H. Moore, A probabilistic and multi-objective analysis of lexicase selection and ε-lexicase selection. Evol Comput. 27(3), 377–402 (2019).

    Article  Google Scholar 

  25. W. La Cava, L. Spector, K. Danai, Epsilon-lexicase selection for regression. in GECCO ’16: Proceedings of the 2016 Annual Conference on Genetic and Evolutionary Computation ed. by T. Friedrich (ACM, Denver, USA, 2016), pp. 741–748. https://doi.org/10.1145/2908812.2908898

  26. P. Liskowski, K. Krawiec, T. Helmuth, L. Spector, Comparison of semantic-aware selection methods in genetic programming. in GECCO 2015 Semantic Methods in Genetic Programming (SMGP’15) Workshop (ACM, Madrid, Spain, 2015), pp. 1301–1307. https://doi.org/10.1145/2739482.2768505. http://doi.acm.org/10.1145/2739482.2768505

  27. N.F. McPhee, D. Donatucci, T. Helmuth, Using graph databases to explore genetic programming run dynamics. in Genetic Programming Theory and Practice XIII, Genetic and Evolutionary Computation (Springer, Ann Arbor, USA, 2015). http://www.springer.com/us/book/9783319342214

  28. B. Metevier, A.K. Saini, L. Spector, Lexicase selection beyond genetic programming. in Genetic Programming Theory and Practice XVI (Springer, Cham, 2019), pp. 123–136. https://doi.org/10.1007/978-3-030-04735-1_7

  29. J.M. Moore, A. Stanton, Lexicase selection outperforms previous strategies for incremental evolution of virtual creature controllers. in Proceedings of the European Conference on Artificial Life (2017), pp 290–297. https://doi.org/10.1162/ecal_a_0050_14. https://www.mitpressjournals.org/doi/abs/10.1162/ecal_a_0050_14

  30. J.M. Moore, A. Stanton, Tiebreaks and diversity: isolating effects in lexicase selection. The 2018 Conference on Artificial Life (2018), pp. 590–597. https://doi.org/10.1162/isal_a_00109. https://www.mitpressjournals.org/doi/abs/10.1162/isal_a_00109

  31. A. Moraglio, K. Krawiec, C.G. Johnson, Geometric semantic genetic programming. in Parallel Problem Solving from Nature, PPSN XII (part 1) eds. by C.A. Coello Coello, V. Cutello, K. Deb, S. Forrest, G. Nicosia, M. Pavone. Lecture Notes in Computer Science, vol. 7491 (Springer, Taormina, Italy, 2012), pp. 21–31. https://doi.org/10.1007/978-3-642-32937-1_3

  32. E. Pantridge, T. Helmuth, N.F. McPhee, L. Spector, Specialization and elitism in lexicase and tournament selection. in Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’18 (ACM, New York, NY, USA, 2018), pp. 1914–1917. https://doi.org/10.1145/3205651.3208220. http://doi.acm.org/10.1145/3205651.3208220

  33. C.D. Rosin, Stepping stones to inductive synthesis of low-level looping programs. in Proceedings of the Thirty-Third AAAI Conference on Artificial IntelligenceAAAI ’19, vol. 33 (AAAI Press, Palo Alto, California USA, 2019)

  34. L. Spector, Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report. In 1st Workshop on Understanding Problems (GECCO-UP), eds. by K. McClymont, E. Keedwell (ACM, Philadelphia, Pennsylvania, USA, 2012), pp. 401–408.. https://doi.org/10.1145/2330784.2330846. http://hampshire.edu/lspector/pubs/wk09p4-spector.pdf

  35. L. Spector, J. Klein, M. Keijzer, The push3 execution stack and the evolution of control. in GECCO 2005: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, vol. 2 (ACM Press, Washington DC, USA, 2005), pp. 1689–1696. https://doi.org/10.1145/1068009.1068292. http://www.cs.bham.ac.uk/~wbl/biblio/gecco2005/docs/p1689.pdf

  36. L. Spector, W. La Cava, S. Shanabrook, T. Helmuth, E. Pantridge, Relaxations of lexicase parent selection. in Genetic Programming Theory and Practice XV, eds. by W. Banzhaf, R.S. Olson, W. Tozier, R. Riolo (Springer, Cham, 2018), pp. 105–120

  37. L. Spector, A. Robinson, Genetic programming and autoconstructive evolution with the push programming language. Genet. Program. Evol. Mach. 3(1), 7–40 (2002). https://doi.org/10.1023/A:1014538503543

    Article  MATH  Google Scholar 

Download references

Acknowledgements

We thank Nicholas McPhee and members of the Hampshire College Institute for Computational Intelligence for discussions that advanced this work. This material is based upon work supported by the National Science Foundation under Grant No. 1617087. Any opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. Hamilton College, 198 College Hill Rd., Clinton, NY, 13323, USA

    Thomas Helmuth

  2. Swoop, 125 Cambridgepark Drive, Cambridge, MA, 02140, USA

    Edward Pantridge

  3. Amherst College, P.O. Box 5000, Amherst, MA, 01002-5000, USA

    Lee Spector

  4. Cognitive Science, Hampshire College, Amherst, MA, 01002, USA

    Lee Spector

  5. University of Massachusetts Amherst, 140 Governors Drive, Amherst, MA, 01003-9264, USA

    Lee Spector

Authors
  1. Thomas Helmuth
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edward Pantridge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lee Spector
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Helmuth.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Helmuth, T., Pantridge, E. & Spector, L. On the importance of specialists for lexicase selection. Genet Program Evolvable Mach 21, 349–373 (2020). https://doi.org/10.1007/s10710-020-09377-2

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-020-09377-2

Keywords

Search

Navigation