Abstract
The decomposition of regression error into bias and variance terms provides insight into the generalization capability of modeling methods. The paper offers an introduction to bias/variance decomposition of mean squared error, as well as a presentation of experimental results of the application of genetic programming. Finally ensemble methods such as bagging and boosting are discussed that can reduce the generalization error in genetic programming.
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bishop, C.M.: Neural Networks for Pattern Recognition. Clarandon Press, Oxford (1995)
Breiman, L.: Bagging predictors. Machine Learning 26, 123–140 (1996)
Clack, C., Yu, T.: Performance enhanced genetic programming. In: Angeline, P.J., McDonnell, J.R., Reynolds, R.G., Eberhart, R. (eds.) EP 1997. LNCS, vol. 1213, Springer, Heidelberg (1997)
Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proceedings of the International Conference on Machine Learning, pp. 148–156 (1996)
Geman, S., Bienenstock, E., Doursat, R.: Neural networks and the bias/variance dilemma. Neural Computation 4, 1–58 (1992)
Hansen, L.K., Salamon, P.: Neural network ensembles. IEEE Transactions on pattern analysis and machine intelligence 12, 993–1001 (1990)
Iba, H.: Bagging, boosting, and bloating in genetic programming. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, USA, July 13-17, vol. 2, pp. 1053–1060. Morgan Kaufmann, San Francisco (1999)
Keijzer, M., Babovic, V.: Dimensionally aware genetic programming. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, USA, July 13-17, vol. 2, pp. 1069–1076. Morgan Kaufmann, San Francisco (1999)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
Langdon, W.B., Soule, T., Poli, R., Foster, J.A.: The evolution of size and shape. In: Spector, L., Langdon, W.B., O’Reilly, U.-M., Angeline, P.J. (eds.) Advances in Genetic Programming, vol. 3, ch. 8, pp. 163–190. MIT Press, Cambridge (1999)
Martin, L., Moal, F., Vrain, C.: Declarative expression of biases in genetic programming. In: Banzhaf, W., Daida, J., Eiben, A.E., Garzon, M.H., Honavar, V., Jakiela, M., Smith, R.E. (eds.) Proceedings of the Genetic and Evolutionary Computation Conference, Orlando, Florida, USA, July 13-17, vol. 1, pp. 401–408. Morgan Kaufmann, San Francisco (1999)
McPhee, N.F., Miller, J.D.: Accurate replication in genetic programming. In: Eshelman, L. (ed.) Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA 1995), Pittsburgh, PA, USA, July 15-19, pp. 303–309. Morgan Kaufmann, San Francisco (1995)
Montana, D.J.: Strongly typed genetic programming. Evolutionary Computation 3(2), 199–230 (1995)
Nikias, C.L., Shao, M.: Signal Processing with Alpha-Stable Distributions and Applications. John Wiley and Sons, Chichester (1995)
Nordin, P., Banzhaf, W.: Complexity compression and evolution. In: Eshelman, L. (ed.) Genetic Algorithms: Proceedings of the Sixth International Conference (ICGA 1995), Pittsburgh, PA, USA, July 15-19, pp. 310–317. Morgan Kaufmann, San Francisco (1995)
Ryan, C., Collins, J.J., Neill, M.O.: Grammatical evolution: Evolving programs for an arbitrary language. In: Banzhaf, W., Poli, R., Schoenauer, M., Fogarty, T.C. (eds.) EuroGP 1998. LNCS, vol. 1391, pp. 83–95. Springer, Heidelberg (1998)
Whigham, P.A.: rammatical Bias for Evolutionary Learning. PhD thesis, School of Computer Science, University College, University of New South Wales, Australian Defence Force Academy, October 14 (1996)
Wolpert, D.: Stacked generalization. Neural Networks 5, 241–259 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Keijzer, M., Babovic, V. (2000). Genetic Programming, Ensemble Methods and the Bias/Variance Tradeoff – Introductory Investigations. In: Poli, R., Banzhaf, W., Langdon, W.B., Miller, J., Nordin, P., Fogarty, T.C. (eds) Genetic Programming. EuroGP 2000. Lecture Notes in Computer Science, vol 1802. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-46239-2_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-46239-2_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67339-2
Online ISBN: 978-3-540-46239-2
eBook Packages: Springer Book Archive