Bogdan Burlacu
ABSTRACT
Genetic Programming (GP) is a dynamic field of research where empirical testing plays an important role in validating new ideas and algorithms. The ability to easily prototype new algorithms by reusing key components and quickly obtain results is therefore important for the researcher.
In this paper we introduce Operon, a C++ GP framework focused on performance, modularity and usability, featuring an efficient linear tree encoding and a scalable concurrency model where each logical thread is responsible for generating a new individual. We model the GP evolutionary process around the concept of an offspring generator, a streaming operator that defines how new individuals are obtained. The approach allows different algorithmic variants to be expressed using high-level constructs within the same generational basic loop. The evaluation routine supports both scalar and dual numbers, making it possible to calculate model derivatives via automatic differentiation. This functionality allows seamless integration with gradient-based local search approaches.
We discuss the design choices behind the proposed framework and compare against two other popular GP frameworks, DEAP and HeuristicLab. We empirically show that Operon is competitive in terms of solution quality, while being able to generate results significantly faster.
Supplemental Material
Available for Download
Supplemental material.
- Michael Affenzeller and Stefan Wagner. 2005. Offspring Selection: A New Self-Adaptive Selection Scheme for Genetic Algorithms. In Adaptive and Natural Computing Algorithms (Springer Computer Science), B. Ribeiro, R. F. Albrecht, A. Dobnikar, D. W. Pearson, and N. C. Steele (Eds.). Springer, 218--221.Google Scholar
- Sameer Agarwal, Keir Mierle, and Others. 2018. Ceres Solver. http://ceres-solver.org. (2018).Google Scholar
- Lee Altenberg. 1994. The Evolution of Evolvability in Genetic Programming. In Advances in Genetic Programming, Kenneth E. Kinnear, Jr. (Ed.). MIT Press, Chapter 3, 47--74.Google ScholarDigital Library
- L. Breiman, J. Friedman, C.J. Stone, and R.A. Olshen. 1984. Classification and Regression Trees. Taylor & Francis.Google Scholar
- Thomas F. Brooks, D. Stuart Pope, and Michael A. Marcolini. 1989. Airfoil self-noise and prediction.Google Scholar
- Darren M. Chitty. 2012. Fast parallel genetic programming: multi-core CPU versus many-core GPU. Soft Computing 16, 10 (Oct. 2012), 1795--1814. https://doi.org/ Google ScholarDigital Library
- Vinícius Veloso de Melo, Álvaro Luiz Fazenda, Léo Françoso Dal Piccol Sotto, and Giovanni Iacca. 2020. A MIMD Interpreter for Genetic Programming. In Applications of Evolutionary Computation, Pedro A. Castillo, Juan Luis Jiménez Laredo, and Francisco Fernández de Vega (Eds.). Springer International Publishing, Cham, 645--658.Google Scholar
- R. Duncan. 1990. A survey of parallel computer architectures. Computer 23, 2 (Feb 1990), 5--16. Google ScholarCross Ref
- Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau, and Christian Gagné. 2012. DEAP: Evolutionary Algorithms Made Easy. Journal of Machine Learning Research 13 (jul 2012), 2171--2175.Google ScholarDigital Library
- Jerome H. Friedman. 1991. Multivariate Adaptive Regression Splines. Ann. Statist. 19, 1 (03 1991), 1--67. Google ScholarCross Ref
- Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. 1995. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Longman Publishing Co., Inc., USA.Google ScholarDigital Library
- Gaël Guennebaud, Benoît Jacob, et al. 2010. Eigen v3. http://eigen.tuxfamily.org. (2010).Google Scholar
- Maarten Keijzer and James Foster. 2007. Crossover Bias in Genetic Programming. In Proceedings of the 10th European Conference on Genetic Programming (Lecture Notes in Computer Science), Marc Ebner, Michael O'Neill, Anikó Ekárt, Leonardo Vanneschi, and Anna Isabel Esparcia-Alcázar (Eds.), Vol. 4445. Springer, Valencia, Spain, 33--43. Google ScholarCross Ref
- Donald E. Knuth. 1997. The Art of Computer Programming, Volume 2 (3rd Ed.): Seminumerical Algorithms. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.Google ScholarDigital Library
- I. Kolar, P.W. Michor, and J. Slovak. 1993. Natural Operations in Differential Geometry. Springer. https://books.google.at/books?id=4m9x1uL71L4CGoogle Scholar
- Michael Kommenda, Bogdan Burlacu, Gabriel Kronberger, and Michael Affenzeller. 2019. Parameter identification for symbolic regression using nonlinear least squares. Genetic Programming and Evolvable Machines (2019), 1--31.Google Scholar
- Arthur Kordon. 2008. Evolutionary Computation in the Chemical Industry. Vol. 86. 245--262. Google ScholarCross Ref
- John R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA, USA.Google ScholarDigital Library
- Sean Luke. 1998. ECJ Evolutionary Computation Library. (1998). Available for free at http://cs.gmu.edu/~eclab/projects/ecj/.Google Scholar
- Nicholas Nethercote and Julian Seward. 2007. Valgrind: A Framework for Heavyweight Dynamic Binary Instrumentation. In Proceedings of ACM SIGPLAN 2007 Conference on Programming Language Design and Implementation (PLDI 2007). San Diego, California, USA, 89--100.Google ScholarDigital Library
- Travis Oliphant. 2006. NumPy: A guide to NumPy. USA: Trelgol Publishing. (2006). http://www.numpy.org/ [Online; accessed <today>].Google Scholar
- Ludo Pagie and Paulien Hogeweg. 1997. Evolutionary Consequences of Coevolving Targets. Evolutionary Computation 5, 4 (1997), 401--418. arXiv:https://doi.org/10.1162/evco.1997.5.4.401 Google ScholarDigital Library
- Riccardo Poli. 2003. A Simple but Theoretically-Motivated Method to Control Bloat in Genetic Programming. In Genetic Programming, Conor Ryan, Terence Soule, Maarten Keijzer, Edward Tsang, Riccardo Poli, and Ernesto Costa (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 204--217.Google Scholar
- W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery. 1992. Numerical Recipes in C. Cambridge University Press, Cambridge.Google Scholar
- Boris Schäling. 2011. The boost C++ libraries. Boris Schäling.Google Scholar
- Erich Schubert and Michael Gertz. 2018. Numerically Stable Parallel Computation of (Co-)Variance. In Proceedings of the 30th International Conference on Scientific and Statistical Database Management (SSDBM '18). ACM, New York, NY, USA, Article 10, 12 pages. Google ScholarDigital Library
- Erich Schubert and Arthur Zimek. 2019. ELKI: A large open-source library for data analysis - ELKI Release 0.7.5 "Heidelberg". CoRR abs/1902.03616 (2019). arXiv:1902.03616 http://arxiv.org/abs/1902.03616Google Scholar
- Eric O. Scott and Sean Luke. 2019. ECJ at 20: toward a general metaheuristics toolkit. In GECCO '19: Proceedings of the Genetic and Evolutionary Computation Conference Companion. ACM, Prague, Czech Republic, 1391--1398. Google ScholarDigital Library
- W. Stallings. 2010. Computer Organization and Architecture: Designing for Performance. Prentice Hall. https://books.google.com/books?id=-7nM1DkWb1YCGoogle Scholar
- S. Wagner, G. Kronberger, A. Beham, M. Kommenda, A. Scheibenpflug, E. Pitzer, S. Vonolfen, M. Kofler, S. Winkler, V. Dorfer, and M. Affenzeller. 2012. Architecture and Design of the HeuristicLab Optimization Environment. In First Australian Conference on the Applications of Systems Engineering, ACASE (Topics in Intelligent Engineering and Informatics), Robin Braun, Zenon Chaczko, and Franz Pichler (Eds.), Vol. 6. Springer International Publishing, Sydney, Australia, 197--261. Selected and updated papers. Google ScholarCross Ref
- B. P. Welford. 1962. Note on a method for calculating corrected sums of squares and products. Technometrics (1962), 419--420.Google Scholar
- Pawel Widera, Jonathan M. Garibaldi, and Natalio Krasnogor. 2010. GP challenge: evolving energy function for protein structure prediction. Genetic Programming and Evolvable Machines 11, 1 (March 2010), 61--88. https://doi.org/ Google ScholarDigital Library
- I.-C. Yeh. 1998. Modeling of strength of high-performance concrete using artificial neural networks. Cement and Concrete Research 28, 12 (1998), 1797 -- 1808. Google Scholar
Index Terms
- Operon C++: an efficient genetic programming framework for symbolic regression
Recommendations
A Comparison of three evolutionary strategies for multiobjective genetic programming
We report what we believe to be the first comparative study of multi-objective genetic programming (GP) algorithms on benchmark symbolic regression and machine learning problems. We compare the Strength Pareto Evolutionary Algorithm (SPEA2), the Non-...
An Investigation of Geometric Semantic GP with Linear Scaling
GECCO '23: Proceedings of the Genetic and Evolutionary Computation ConferenceGeometric semantic genetic programming (GSGP) and linear scaling (LS) have both, independently, shown the ability to outperform standard genetic programming (GP) for symbolic regression. GSGP uses geometric semantic genetic operators, different from ...
Genetic programming performance prediction and its application for symbolic regression problems
Highlights- Theoretical analysis of GP performance prediction.
- Suggestion of upper bounds ...
AbstractPredicting the performance of Genetic Programming (GP) helps us identify whether it is an appropriate approach to solve the problem at hand. However, previous studies show that measuring the difficulty of a problem for GP and ...
Comments