Abstract
The success of Artificial Neural Networks (ANNs) highly depends on their architecture and on how they are trained. However, making decisions regarding such domain specific issues is not an easy task, and is usually performed by hand, through an exhaustive trial-and-error process. Over the years, researches have developed and proposed methods to automatically train ANNs. One example is the HyperNEAT algorithm, which relies on NeuroEvolution of Augmenting Topologies (NEAT) to create Compositional Pattern Production Networks (CPPNs). CPPNs are networks that encode the mapping between neuron positions and the synaptic weight of the ANN connection between those neurons. Although this approach has obtained some success, it requires meticulous parameterisation to work properly. In this article we present a comparison of different Evolutionary Computation methods to evolve Compositional Pattern Production Functions: structures that have the same goal as CPPNs, but that are encoded as functions instead of networks. In addition to NEAT three methods are used to evolve such functions: Genetic Programming (GP), Grammatical Evolution, and Dynamic Structured Grammatical Evolution. The results show that GP is able to obtain competitive performance, often surpassing the other methods, without requiring the fine tuning of the parameters.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
NEAT – https://github.com/noio/peas [6].
GP – https://github.com/DEAP/deap.
References
Secretan, J., et al.: Picbreeder: evolving pictures collaboratively online. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1759–1768. ACM (2008)
Ahmadizar, F., Soltanian, K., AkhlaghianTab, F., Tsoulos, I.: Artificial neural network development by means of a novel combination of grammatical evolution and genetic algorithm. Eng. Appl. Artif. Intell. 39, 1–13 (2015)
Assunção, F., Lourenço, N., Machado, P., Ribeiro, B.: Automatic generation of neural networks with structured grammatical evolution. In: 2017 IEEE Congress on Evolutionary Computation (CEC), pp. 1557–1564, June 2017
Assunção, F., Lourenço, N., Machado, P., Ribeiro, B.: Towards the evolution of multi-layered neural networks: a dynamic structured grammatical evolution approach. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2017, pp. 393–400. ACM, New York (2017). http://doi.acm.org/10.1145/3071178.3071286
Bengio, S., Bengio, Y., Cloutier, J.: Use of genetic programming for the search of a new learning rule for neural networks. In: 1994 Proceedings of the First IEEE Conference on Evolutionary Computation, IEEE World Congress on Computational Intelligence, pp. 324–327. IEEE (1994)
van den Berg, T.G., Whiteson, S.: Critical factors in the performance of HyperNEAT. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 759–766. ACM (2013)
Buk, Z., Koutník, J., Šnorek, M.: NEAT in HyperNEAT substituted with genetic programming. In: Kolehmainen, M., Toivanen, P., Beliczynski, B. (eds.) ICANNGA 2009. LNCS, vol. 5495, pp. 243–252. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-04921-7_25
David, O.E., Greental, I.: Genetic algorithms for evolving deep neural networks. In: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation, pp. 1451–1452. ACM (2014)
Drchal, J., Koutník, J., Snorek, M.: HyperNEAT controlled robots learn how to drive on roads in simulated environment. In: 2009 IEEE Congress on Evolutionary Computation, CEC 2009, pp. 1087–1092. IEEE (2009)
Gomez, F., Schmidhuber, J., Miikkulainen, R.: Accelerated neural evolution through cooperatively coevolved synapses. J. Mach. Learn. Res. 9(May), 937–965 (2008)
Khan, M.M., Khan, G.M., Miller, J.F.: Evolution of neural networks using Cartesian genetic programming. In: 2010 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8. IEEE (2010)
Kim, H.B., Jung, S.H., Kim, T.G., Park, K.H.: Fast learning method for back-propagation neural network by evolutionary adaptation of learning rates. Neurocomputing 11(1), 101–106 (1996)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection, vol. 1. MIT Press, Cambridge (1992)
Lourenço, N., Pereira, F.B., Costa, E.: Unveiling the properties of structured grammatical evolution. Genet. Program Evolvable Mach. 17(3), 251–289 (2016)
Machado, P., Cardoso, A.: All the truth about NEvAr. Appl. Intell. 16(2), 101–118 (2002)
O’Neil, M., Ryan, C.: Grammatical evolution. In: O’Neil, M., Ryan, C. (eds.) Grammatical Evolution, pp. 33–47. Springer, New York (2003). https://doi.org/10.1007/978-1-4615-0447-4
Parra, J., Trujillo, L., Melin, P.: Hybrid back-propagation training with evolutionary strategies. Soft. Comput. 18(8), 1603–1614 (2014)
Radi, A., Poli, R.: Discovering efficient learning rules for feedforward neural networks using genetic programming. In: Abraham, A., Jain, L.C., Kacprzyk, J. (eds.) Recent Advances in Intelligent Paradigms and Applications, pp. 133–159. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-7908-1770-6_7
Stanley, K.O., D’Ambrosio, D.B., Gauci, J.: A hypercube-based encoding for evolving large-scale neural networks. Artif. Life 15(2), 185–212 (2009)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)
Whitley, D., Starkweather, T., Bogart, C.: Genetic algorithms and neural networks: optimizing connections and connectivity. Parallel Comput. 14(3), 347–361 (1990)
Acknowledgments
This work is partially funded by: Fundação para a Ciência e Tecnologia (FCT), Portugal, under the grant SFRH/BD/114865/2016, and is based upon work from COST Action CA15140: ImAppNIO, supported by COST (European Cooperation in Science and Technology): www.cost.eu.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Assunção, F., Lourenço, N., Machado, P., Ribeiro, B. (2018). Using GP Is NEAT: Evolving Compositional Pattern Production Functions. In: Castelli, M., Sekanina, L., Zhang, M., Cagnoni, S., García-Sánchez, P. (eds) Genetic Programming. EuroGP 2018. Lecture Notes in Computer Science(), vol 10781. Springer, Cham. https://doi.org/10.1007/978-3-319-77553-1_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-77553-1_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77552-4
Online ISBN: 978-3-319-77553-1
eBook Packages: Computer ScienceComputer Science (R0)