Abstract
The publication of Grammatical Evolution (GE) led to the development of numerous variants of this Grammar-Based approach to Genetic Programming (GP). In order for these variants to be compared, the community requires a rigorous means for benchmarking the algorithms. However, GP as a field is not known for the quality of its benchmarking, with many identified problems, making direct comparisons either difficult or impossible in many cases. Aside from there not being a single, agreed-upon, benchmarking test, the tests currently utilised have shown a lack of standardisation. We motivate the following research by identifying some key issues with current benchmarking approaches. We then propose a standardised set of metrics for future benchmarking and demonstrate the use of these metrics by running a comparison of three Grammar-Based Genetic Programming methods. We conclude the work by discussing the results and proposing directions for future benchmarking.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Behera, R., Pati, B.B., Panigrahi, B.P., Misra, S.: An application of genetic programming for power system planning and operation. Int. J. Control Syst. Instrum. 3(2) (2012)
Georgiou, L: Constituent grammatical evolution. PhD thesis, University of Wales, Bangor (2004)
Georgiou, L., Teahan, W.J.: Constituent grammatical evolution. In: Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence-Volume, vol. 2, pp. 1261–1268. AAAI Press (2011)
Headleand, C., Teahan, W.J.: Grammatical herding. J. Comput. Sci. Syst. Biol. 6, 043–047 (2013)
Headleand, C., Teahan, W.J.: Swarm based population seeding of grammatical evolution. J. Comput. Sci. Syst. Biol. 6, 132–135 (2013)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT press, Cambridge (1992)
Langdon, W.B, Poli, R.: Better trained ants for genetic programming (1998)
McDermott, J., White, D.R., Luke, S., Manzoni, L., Castelli, M., Vanneschi, L., Jaskowski, W., Krawiec, K., Harper, R., De Jong, K., et al.: Genetic programming needs better benchmarks. In: Proceedings of the Fourteenth International Conference on Genetic and Evolutionary Computation Conference, pp. 791–798. ACM (2012)
O’Neil, M., Ryan, C.: Grammatical evolution. In: Grammatical Evolution, pp. 33–47. Springer, Berlin (2003)
O’Neill, M., Brabazon, A.: Grammatical swarm: the generation of programs by social programming. Nat. Comput. 5(4), 443–462 (2006)
O’Neill, M., Vanneschi, L., Gustafson, S., Banzhaf, W.: Open issues in genetic programming. Genet. Program Evolvable Mach. 11(3–4), 339–363 (2010)
Robilliard, D., Mahler, S., Verhaghe, D., Fonlupt, C.: Santa fe trail hazards. In: Artificial Evolution, pp. 1–12. Springer, Berlin (2006)
Si, T., De, A., Bhattacharjee A.K.: Grammatical bee colony. In: Swarm, Evolutionary, and Memetic Computing, pp. 436–445. Springer, Berlin (2013)
Teahan, W.J.: Artificial Intelligence-Agent Behaviour. BookBoon, London (2010)
White, D.R., McDermott, J., Castelli, M., Manzoni, L., Goldman, B.W., Kronberger, G., Jaśkowski, W., OReilly, U., Luke, S.: Free text responses. Online, Sept 2012. http://gpbenchmarks.org/wp-content/uploads/2012/09/SurveyFreeTextResponses.txt
White, D.R., McDermott, J., Castelli, M., Manzoni, L., Goldman, B.W., Kronberger, G., Jaśkowski, W., OReilly, U., Luke, S.: Better gp benchmarks: community survey results and proposals. Genet. Program. Evolvable Mach. 14(1), 3–29 (2013)
Acknowledgments
The authors would like to thank HPC Wales for providing their facilities and technical support during the running of the experiments described in this research. Chris Headleand would also like to thank Fujitsu for their ongoing financial support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Headleand, C.J., Cenydd, L.A., Teahan, W.J. (2014). Benchmarking Grammar-Based Genetic Programming Algorithms. In: Bramer, M., Petridis, M. (eds) Research and Development in Intelligent Systems XXXI. SGAI 2014. Springer, Cham. https://doi.org/10.1007/978-3-319-12069-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-12069-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-12068-3
Online ISBN: 978-3-319-12069-0
eBook Packages: Computer ScienceComputer Science (R0)