Abstract
The problem of overfitting (focusing closely on examples at the loss of generalization power) is encountered in all supervised machine learning schemes. This study is dedicated to explore some aspects of overfitting in the particular case of genetic programming. After recalling the causes usually invoked to explain overfitting such as hypothesis complexity or noisy learning examples, we test and compare the resistance to overfitting on three variants of genetic programming algorithms (basic GP, sizefair crossover GP and GP with boosting) on two benchmarks, a symbolic regression and a classification problem. We propose guidelines based on these results to help reduce overfitting with genetic programming.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Koza, J.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. The MIT Press, Cambridge (1992)
Langdon, W.B.: Size fair and homologous tree crossover for tree genetic programming. In: Genetic Programing and Evolvable Machines, Boston, vol. 1, pp. 95–119. Kluwer Academic Publishers, Dordrecht (2000)
Mitchell, T.M.: Machine Learning. McGraw-Hill, New York (1997)
Ng, A.Y.: Preventing overfitting of cross-validation data. In: Proc. 14th International Conference on Machine Learning, pp. 245–253. Morgan Kaufmann, San Francisco (1997)
Cavaretta, M.J., Chellapilla, K.: Data mining using genetic programming: The implications of parsimony on generalization error. In: Angeline, P.J., Michalewicz, Z., Schoenauer, M., Yao, X., Zalzala, A. (eds.) Proceedings of the Congress of Evolutionary Computation, Mayflower Hotel, Washington D.C., USA, vol. 2, pp. 1330–1337. IEEE Press, Los Alamitos (1999)
Paris, G., Robilliard, D., Fonlupt, C.: Applying boosting techniques to genetic programming. In: [14], pp. 267–278 (2001)
Paris, G., Robilliard, D., Fonlupt, C.: Genetic programming with boosting for ambiguities in regression problems. In: [15], pp. 183–193 (2003)
Iba, H.: Bagging, boosting, and bloating in genetic programming. In: [?], pp. 1053–1060 (1999)
Langdon, W.B.: Combining decision trees and neural networks for drug discovery. In: [17], pp. 60–70 (2002)
Freund, Y., Schapire, R.: A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence, 771–780 (1999)
Freund, Y., Schapire, Y.M., R.: Why averaging classifiers can protect against overfitting. In: Proceeding of the Eighth international WOrkshop on Artificial Intelligence and Statistics (2001)
Luke, S.: When short runs beat long runs. In: Proceedings of GECCO 2001, pp. 74–80. Morgan Kaufmann, San Francisco (2001)
Domingos, P.: The role of Occam’s razor in knowledge discovery. Data Mining and Knowledge Discovery 3, 409–425 (1999)
Collet, P., Fonlupt, C., Hao, J.-K., Lutton, E., SChoenauer, M. (eds.): EA 2001. LNCS, vol. 2310. Springer, Heidelberg (2002)
Ryan, C., Soule, T., Keijzer, M., Tsang, E., Poli, R., Costa, E. (eds.): EuroGP 2003. LNCS, vol. 2610. Springer, Heidelberg (2003)
Banzhaf, W., Daida, J., Eiben, A., Garzon, M., Honavar, V., Jakiela, M., Smith, R. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, USA, July 1999, Morgan-Kaufmann, San Francisco (1999)
Foster, J.A., Lutton, E., Miller, J., Ryan, C., Tettamanzi, A.G.B. (eds.) EuroGP 2002. LNCS, vol. 2278, Springer, Heidelberg (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Paris, G., Robilliard, D., Fonlupt, C. (2004). Exploring Overfitting in Genetic Programming. In: Liardet, P., Collet, P., Fonlupt, C., Lutton, E., Schoenauer, M. (eds) Artificial Evolution. EA 2003. Lecture Notes in Computer Science, vol 2936. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-24621-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-540-24621-3_22
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-21523-3
Online ISBN: 978-3-540-24621-3
eBook Packages: Springer Book Archive