Abstract
This work uses Push GP to automatically design both local and population-based optimisers for continuous-valued problems. The optimisers are trained on a single function optimisation landscape, using random transformations to discourage overfitting. They are then tested for generality on larger versions of the same problem, and on other continuous-valued problems. In most cases, the optimisers generalise well to the larger problems. Surprisingly, some of them also generalise very well to previously unseen problems, outperforming existing general purpose optimisers such as CMA-ES. Analysis of the behaviour of the evolved optimisers indicates a range of interesting optimisation strategies that are not found within conventional optimisers, suggesting that this approach could be useful for discovering novel and effective forms of optimisation in an automated manner.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Andrychowicz, M., et al.: Learning to learn by gradient descent by gradient descent. In: Advances in Neural Information Processing Systems (2016)
Auger, A., Hansen, N.: A restart CMA evolution strategy with increasing population size. In: Proceedings of the IEEE Congress on Evolutionary Computation, IEEE CEC 2005, vol. 2, pp. 1769–1776. IEEE (2005)
Bogdanova, A., Junior, J.P., Aranha, C.: Franken-swarm: grammatical evolution for the automatic generation of swarm-like meta-heuristics. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO 2019, pp. 411–412. ACM (2019)
Edmonds, B.: Meta-genetic programming: Co-evolving the operators of variation. Technical report CPM Report 98–32, Manchester Metropolitan University (1998)
Goldman, B.W., Tauritz, D.R.: Self-configuring crossover. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO 2011, pp. 575–582. ACM (2011)
Kantschik, W., Dittrich, P., Brameier, M., Banzhaf, W.: Meta-Evolution in Graph GP. In: Poli, R., Nordin, P., Langdon, W.B., Fogarty, T.C. (eds.) EuroGP 1999. LNCS, vol. 1598, pp. 15–28. Springer, Heidelberg (1999). https://doi.org/10.1007/3-540-48885-5_2
Langdon, W.B.: Genetic Programming And Data Structures: Genetic Programming + Data Structures = Automatic Programming!. Springer, New York (2012). https://doi.org/10.1007/978-1-4615-5731-9
Lones, M.A.: Instruction-level design of local optimisers using push GP. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO 2019, pp. 1487–1494. ACM (2019)
Lones, M.A.: Mitigating metaphors: a comprehensible guide to recent nature-inspired algorithms. SN Comput. Sci. 1(1), 49 (2020)
Martin, M.A., Tauritz, D.R.: Evolving black-box search algorithms employing genetic programming. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO 2013, pp. 1497–1504. ACM (2013)
Metz, L., Maheswaranathan, N., Nixon, J., Freeman, D., Sohl-dickstein, J.: Learned optimizers that outperform SGD on wall-clock and test loss. In: Proceedings of the 2nd Workshop on Meta-Learning. MetaLearn 2018 (2018)
Oltean, M.: Evolving evolutionary algorithms using linear genetic programming. Evol. Comput. 13(3), 387–410 (2005)
Richter, S.N., Tauritz, D.R.: The automated design of probabilistic selection methods for evolutionary algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO 2018, pp. 1545–1552. ACM (2018)
Ronkkonen, J., Kukkonen, S., Price, K.V.: Real-parameter optimization with differential evolution. In: Proceedings of the IEEE Congress on Evolutionary Computation, IEEE CEC 2005, vol. 1, pp. 506–513. IEEE (2005)
Ryser-Welch, P., Miller, J.F., Swan, J., Trefzer, M.A.: Iterative cartesian genetic programming: creating general algorithms for solving travelling salesman problems. In: Heywood, M.I., McDermott, J., Castelli, M., Costa, E., Sim, K. (eds.) EuroGP 2016. LNCS, vol. 9594, pp. 294–310. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30668-1_19
Sörensen, K.: Metaheuristics–the metaphor exposed. Int. Trans. Oper. Res. 22(1), 3–18 (2015)
Spector, L.: Autoconstructive evolution: push, pushGP, and pushpop. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2019, vol. 137 (2001)
Spector, L., Perry, C., Klein, J., Keijzer, M.: Push 3.0 programming language description. Technical report, HC-CSTR-2004-02, School of Cognitive Science, Hampshire College (2004)
Spector, L., Robinson, A.: Genetic programming and autoconstructive evolution with the push programming language. Genet. Program Evolvable Mach. 3(1), 7–40 (2002)
Suganthan, P.N., et al.: Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization, KanGAL report, 2005005 (2005)
Wichrowska, O., et al.: Learned optimizers that scale and generalize. In: Proceedings of the 34th International Conference on Machine Learning-Volume 70, ICML 2017, pp. 3751–3760 (2017)
Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)
Woodward, J.R., Swan, J.: The automatic generation of mutation operators for genetic algorithms. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 2012, pp. 67–74. ACM (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lones, M.A. (2020). Optimising Optimisers with Push GP. In: Hu, T., Lourenço, N., Medvet, E., Divina, F. (eds) Genetic Programming. EuroGP 2020. Lecture Notes in Computer Science(), vol 12101. Springer, Cham. https://doi.org/10.1007/978-3-030-44094-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-44094-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44093-0
Online ISBN: 978-3-030-44094-7
eBook Packages: Computer ScienceComputer Science (R0)