ABSTRACT
Vectorial Genetic Programming (Vec-GP) extends regular GP by allowing vectorial input features (e.g. time series data), while retaining the expressiveness and interpretability of regular GP. The availability of raw vectorial data during training, not only enables Vec-GP to select appropriate aggregation functions itself, but also allows Vec-GP to extract segments from vectors prior to aggregation (like windows for time series data). This is a critical factor in many machine learning applications, as vectors can be very long and only small segments may be relevant. However, allowing aggregation over segments within GP models makes the training more complicated. We explore the use of common evolutionary algorithms to help GP identify appropriate segments, which we analyze using a simplified problem that focuses on optimizing aggregation segments on fixed data. Since the studied algorithms are to be used in GP for local optimization (e.g. as mutation operator), we evaluate not only the quality of the solutions, but also take into account the convergence speed and anytime performance. Among the evaluated algorithms, CMA-ES, PSO and ALPS show the most promising results, which would be prime candidates for evaluation within GP.
- Michael Affenzeller and Stefan Wagner. 2005. Offspring selection: A new self-adaptive selection scheme for genetic algorithms. In Adaptive and Natural Computing Algorithms: Proceedings of the International Conference in Coimbra, Portugal, 2005. Springer, 218--221.Google ScholarCross Ref
- Irene Azzali, Leonardo Vanneschi, Sara Silva, Illya Bakurov, and Marion Giacobini. 2019. A Vectorial Approach to Genetic Programming. EuroGP (2019), 213--227.Google Scholar
- Maximilian Christ, Nils Braun, Julius Neuffer, and Andreas W. Kempa-Liehr. 2018. Time Series FeatuRe Extraction on basis of Scalable Hypothesis tests (tsfresh - A Python package). Neurocomputing 307 (2018), 72--77.Google ScholarDigital Library
- Russell Eberhart and James Kennedy. 1995. Particle swarm optimization. In Proceedings of the IEEE international conference on neural networks, Vol. 4. Citeseer, 1942--1948.Google Scholar
- Philipp Fleck, Stephan Winkler, Michael Kommenda, and Michael Affenzeller. 2021. Garmmar-based Vectorial Genetig Programming for Symbolic Regression. In Genetic Programming Theory and Practice XVIII, Wolfgang Banzhaf, Leonardo Trujillo, Stephan Winkler, and Bill Worzel (Eds.). Springer Nature Singapore Pte Ltd.Google Scholar
- Philipp Fleck, Stephan Winkler, Michael Kommenda, and Michael Affenzeller. 2023. Vectorial Genetic Programming - Optimizing Segments for Feature Extraction. arXiv:2303.03200 Note: Submitted to Eurocast 2022, but was not published due to an error in the submission information system. Will be published in the Eurocast 2024 proceedings..Google Scholar
- Nikolaus Hansen. 2016. The CMA evolution strategy: A tutorial. arXiv preprint arXiv:1604.00772 (2016).Google Scholar
- Nikolaus Hansen, Anne Auger, Raymond Ros, Steffen Finck, and Petr Pošík. 2010. Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009. In Proceedings of the 12th annual conference companion on Genetic and evolutionary computation. 1689--1696.Google ScholarDigital Library
- Nikolaus Hansen, Anne Auger, Raymond Ros, Olaf Mersmann, Tea Tušar, and Dimo Brockhoff. 2021. COCO: A platform for comparing continuous optimizers in a black-box setting. Optimization Methods and Software 36, 1 (2021), 114--144.Google ScholarCross Ref
- Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, Nathaniel J. Smith, Robert Kern, Matti Picus, Stephan Hoyer, Marten H. van Kerkwijk, Matthew Brett, Allan Haldane, Jaime Fernández del Río, Mark Wiebe, Pearu Peterson, Pierre Gérard-Marchant, Kevin Sheppard, Tyler Reddy, Warren Weckesser, Hameer Abbasi, Christoph Gohlke, and Travis E. Oliphant. 2020. Array programming with NumPy. Nature 585, 7825 (Sept. 2020), 357--362.Google ScholarCross Ref
- Gregory S Hornby. 2006. ALPS: the age-layered population structure for reducing the problem of premature convergence. In Proceedings of the 8th annual conference on Genetic and evolutionary computation. 815--822.Google ScholarDigital Library
- Holland Jh. 1975. Adaptation in natural and artificial systems. Ann Arbor (1975).Google Scholar
- John R Koza. 1993. Genetic Programming: On the Programming of Computers by Means of Natural Selection (Complex Adaptive Systems). A Bradford Book 1 (1993), 18.Google Scholar
- Riccardo Poli, William B. Langdon, and Nicholas Freitag McPhee. 2008. A field guide to genetic programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk.Google Scholar
- Ingo Rechenberg. 1965. Cybernetic solution path of an experimental problem. Roy. Aircr. Establ., Libr. transl. 1122 (1965).Google Scholar
- Alice Zheng and Amanda Casari. 2018. Feature engineering for machine learning: principles and techniques for data scientists. " O'Reilly Media, Inc.".Google Scholar
Index Terms
- Evolutionary Algorithms for Segment Optimization in Vectorial GP
Recommendations
Neural network crossover in genetic algorithms using genetic programming
AbstractThe use of genetic algorithms (GAs) to evolve neural network (NN) weights has risen in popularity in recent years, particularly when used together with gradient descent as a mutation operator. However, crossover operators are often omitted from ...
HMXT-GP: an information-theoretic approach to genetic programming that maintains diversity
SAC '11: Proceedings of the 2011 ACM Symposium on Applied ComputingThis paper applies a recent information--theoretic approach to controlling Genetic Algorithms (GAs) called HMXT to tree--based Genetic Programming (GP). HMXT, in a GA domain, requires the setting of selection thresholds in a population and the ...
A Comparison of three evolutionary strategies for multiobjective genetic programming
We report what we believe to be the first comparative study of multi-objective genetic programming (GP) algorithms on benchmark symbolic regression and machine learning problems. We compare the Strength Pareto Evolutionary Algorithm (SPEA2), the Non-...
Comments