ABSTRACT
In iterative supervised learning algorithms it is common to reach a point in the search where no further induction seems to be possible with the available data. If the search is continued beyond this point, the risk of overfitting increases significantly. Following the recent developments in inductive semantic stochastic methods, this paper studies the feasibility of using information gathered from the semantic neighborhood to decide when to stop the search. Two semantic stopping criteria are proposed and experimentally assessed in Geometric Semantic Genetic Programming (GSGP) and in the Semantic Learning Machine (SLM) algorithm (the equivalent algorithm for neural networks). The experiments are performed on real-world high-dimensional regression datasets. The results show that the proposed semantic stopping criteria are able to detect stopping points that result in a competitive generalization for both GSGP and SLM. This approach also yields computationally efficient algorithms as it allows the evolution of neural networks in less than 3 seconds on average, and of GP trees in at most 10 seconds. The usage of the proposed semantic stopping criteria in conjunction with the computation of optimal mutation/learning steps also results in small trees and neural networks.
- Qi Chen, Bing Xue, Lin Shang, and Mengjie Zhang. 2016. Improving Generalisation of Genetic Programming for Symbolic Regression with Structural Risk Minimisation. In Proceedings of the 2016 on Genetic and Evolutionary Computation Conference. ACM, 709--716. Google ScholarDigital Library
- Agoston E Eiben and Márk Jelasity. 2002. A critical note on experimental research methodology in EC. In Proceedings of the 2002 Congress on Evolutionary Computation (CEC 2002), Vol. 58. 2--587.Google ScholarCross Ref
- Ivo Gonçalves and Sara Silva. 2013. Balancing learning and overfitting in genetic programming with interleaved sampling of training data. In Genetic Programming. Springer, 73--84. Google ScholarDigital Library
- Ivo Gonçalves, Sara Silva, and Carlos M Fonseca. 2015. On the generalization ability of geometric semantic genetic programming. In Genetic Programming. Springer, 41--52.Google Scholar
- Ivo Gonçalves, Sara Silva, and Carlos M. Fonseca. 2015. Semantic Learning Machine: A Feedforward Neural Network Construction Algorithm Inspired by Geometric Semantic Genetic Programming. In Progress in Artificial Intelligence. Lecture Notes in Computer Science, Vol. 9273. Springer, 280--285.Google Scholar
- Ivo Gonçalves, Sara Silva, Joana B. Melo, and João M. B. Carreiras. 2012. Random sampling technique for overfitting control in genetic programming. In Genetic Programming. Springer, 218--229. Google ScholarDigital Library
- Ivo Gonçalves. 2017. An Exploration of Generalization and Overfitting in Genetic Programming: Standard and Geometric Semantic Approaches. Ph.D. Dissertation. Department of Informatics Engineering, University of Coimbra, Portugal.Google Scholar
- Ivo Gonçalves and Sara Silva. 2011. Experiments on Controlling Overfitting in Genetic Programming. In Proceedings of the 15th Portuguese Conference on Artificial Intelligence: Progress in Artificial Intelligence (EPIA 2011).Google Scholar
- Michael Kommenda, Michael Affenzeller, Bogdan Burlacu, Gabriel Kronberger, and Stephan M Winkler. 2014. Genetic programming with data migration for symbolic regression. In Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary Computation. ACM, 1361--1366. Google ScholarDigital Library
- John R. Koza. 1992. Genetic Programming: On the Programming of Computers by Means of Natural Selection (Complex Adaptive Systems) (1 ed.). The MIT Press. Google ScholarDigital Library
- Ibrahim Kushchu. 2002. An Evaluation of Evolutionary Generalisation in Genetic Programming. Artif. Intell. Rev. 18 (September 2002), 3--14. Issue 1. Google ScholarDigital Library
- Alberto Moraglio. 2007. Towards a Geometric Unification of Evolutionary Algorithms. Ph.D. Dissertation. Department of Computer Science, University of Essex, UK.Google Scholar
- Alberto Moraglio, Krzysztof Krawiec, and Colin G Johnson. 2012. Geometric semantic genetic programming. In Parallel Problem Solving from Nature-PPSN XII. Springer, 21--31. Google ScholarDigital Library
Recommendations
C-Mantec: A novel constructive neural network algorithm incorporating competition between neurons
C-Mantec is a novel neural network constructive algorithm that combines competition between neurons with a stable modified perceptron learning rule. The neuron learning is governed by the thermal perceptron rule that ensures stability of the acquired ...
Monitoring MLP's free parameters for generalization
AIKED'09: Proceedings of the 8th WSEAS international conference on Artificial intelligence, knowledge engineering and data basesGeneralization is one of major concerns for neural network training. In common practice, the number of weights in a MLP network is assumed to be the number of free parameters. This assumption leads to a conclusion: large MLP networks will generalize ...
Evolving Multilayer Perceptrons
This paper proposes a new version of a method (G-Prop, genetic backpropagation) that attempts to solve the problem of finding appropriate initial weights and learning parameters for a single hidden layer Multilayer Perceptron (MLP) by combining an ...
Comments