Abstract
Probabilistic models are widely used in evolutionary and related algorithms. In Genetic Programming (GP), the Probabilistic Prototype Tree (PPT) is often used as a model representation. Drift due to sampling bias is a widely recognised problem, and may be serious, particularly in dependent probability models. While this has been closely studied in independent probability models, and more recently in probabilistic dependency models, it has received little attention in systems with strict dependence between probabilistic variables such as arise in PPT representation. Here, we investigate this issue, and present results suggesting that the drift effect in such models may be particularly severe – so severe as to cast doubt on their scalability. We present a preliminary analysis through a factor representation of the joint probability distribution. We suggest future directions for research aiming to overcome this problem.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Colorni, A., Dorigo, M., Maniezzo, V., et al.: Distributed Optimization by Ant Colonies. In: Varela, F.J., Bourgine, P. (eds.) Toward a Practice of Autonomous Systems: Proceedings of the First European Conference on Artificial Life, pp. 134–142. MIT Press, Cambridge (1991)
Baluja, S.: Population-based incremental learning: A method for integrating genetic searching based function optimization. Technical Report CMU-CS-94-163, Computer Science Dept., Carnegie Mellon University, Pittsburgh, PA, USA (1994)
Mühlenbein, H., Mahnig, T.: The factorized distribution algorithm for additively decomposed functions. In: Proceedings of the 1999 Congress on Evolutionary Computation, pp. 752–759. IEEE Press, Los Alamitos (1999)
Pelikan, M., Goldberg, D., Cantu-Paz, E.: BOA: The Bayesian optimization algorithm. In: Proceedings of the Genetic and Evolutionary Computation Conference, GECCO 1999, vol. 1, pp. 525–532 (1999)
Salustowicz, R., Schmidhuber, J.: Probabilistic incremental program evolution. Evolutionary Computation 5(2), 123–141 (1997)
Koza, J.: Genetic programming: on the programming of computers by means of natural selection. The MIT Press, Cambridge (1992)
Yanai, K., Iba, H.: Estimation of distribution programming based on bayesian network. In: Proceedings of the Congress on Evolutionary Computation, Canberra, Australia, pp. 1618–1625 (December 2003)
Sastry, K., Goldberg, D.E.: Probabilistic model building and competent genetic programming. In: Riolo, R.L., Worzel, B. (eds.) Genetic Programming Theory and Practise, pp. 205–220. Kluwer, Dordrecht (2003)
Hasegawa, Y., Iba, H.: A bayesian network approach to program generation. IEEE Transactions on Evolutionary Computation 12(6), 750–764 (2008)
Looks, M., Goertzel, B., Pennachin, C.: Learning computer programs with the bayesian optimization algorithm. In: GECCO 2005: Proceedings of the 2005 Conference on Genetic and Evolutionary Computation, pp. 747–748. ACM, New York (2005)
Roux, O., Fonlupt, C.: Ant programming: or how to use ants for automatic programming. In: Proceedings of the Second International Conference on Ant Algorithms (ANTS 2000), Belgium (2000)
Harik, G., Lobo, F., Sastry, K.: Linkage Learning via Probabilistic Modeling in the Extended Compact Genetic Algorithm (ECGA). In: Scalable Optimization via Probabilistic Modeling, vol. 33, pp. 39–61. Springer, Heidelberg (2006)
Schaffer, J., Eshelman, L., Offutt, D.: Spurious correlations and premature convergence in genetic algorithms. Foundations of Genetic Algorithms, pp. 102–112 (1991)
Gathercole, C., Ross, P.: An adverse interaction between crossover and restricted tree depth in genetic programming. In: GECCO 1996: Proceedings of the First Annual Conference on Genetic Programming, pp. 291–296. MIT Press, Cambridge (1996)
Whigham, P.A.: Grammatically-based genetic programming. In: Rosca, J. (ed.) Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, pp. 33–41 (1995)
Henrion, M.: Propagating uncertainty in bayesian networks by probabilistic logic sampling. In: Uncertainty in Artificial Intelligence 2 (UAI 1986), pp. 149–163. North Holland, Amsterdam (1986)
Shapiro, J.L.: Diversity loss in general estimation of distribution algorithms. In: Runarsson, T.P., Beyer, H.-G., Burke, E.K., Merelo-Guervós, J.J., Whitley, L.D., Yao, X. (eds.) PPSN 2006. LNCS, vol. 4193, pp. 92–101. Springer, Heidelberg (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kim, K., McKay, B.(.I.)., Punithan, D. (2010). Sampling Bias in Estimation of Distribution Algorithms for Genetic Programming Using Prototype Trees. In: Zhang, BT., Orgun, M.A. (eds) PRICAI 2010: Trends in Artificial Intelligence. PRICAI 2010. Lecture Notes in Computer Science(), vol 6230. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15246-7_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-15246-7_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15245-0
Online ISBN: 978-3-642-15246-7
eBook Packages: Computer ScienceComputer Science (R0)