Skip to main content
SpringerLink
Log in

A review on probabilistic graphical models in evolutionary computation

  • Published:
Journal of Heuristics Aims and scope Submit manuscript

Abstract

Thanks to their inherent properties, probabilistic graphical models are one of the prime candidates for machine learning and decision making tasks especially in uncertain domains. Their capabilities, like representation, inference and learning, if used effectively, can greatly help to build intelligent systems that are able to act accordingly in different problem domains. Evolutionary algorithms is one such discipline that has employed probabilistic graphical models to improve the search for optimal solutions in complex problems. This paper shows how probabilistic graphical models have been used in evolutionary algorithms to improve their performance in solving complex problems. Specifically, we give a survey of probabilistic model building-based evolutionary algorithms, called estimation of distribution algorithms, and compare different methods for probabilistic modeling in these algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4

Similar content being viewed by others

References

  • Ahn, C.W., An, J., Yoo, J.C.: Estimation of particle swarm distribution algorithms: combining the benefits of PSO and EDAs. Inf. Sci. 192, 109–119 (2012)

    Article  Google Scholar 

  • Ahn, C., Ramakrishna, R., Goldberg, D.: Real-coded Bayesian optimization algorithm: bringing the strength of BOA into the continuous world. In: 6th Annual Conference on Genetic and Evolutionary Computation (GECCO’04), pp. 840–851. Springer, Berlin (2004)

    Google Scholar 

  • Akaike, H.: A new look at the statistical model identification. IEEE Trans. Autom. Control 19(6), 716–723 (1974)

    Article  MathSciNet  MATH  Google Scholar 

  • Alden, M.E.: MARLEDA: effective distribution estimation through Markov random fields. Ph.D. Thesis, The University of Texas at Austin (2007)

  • Baluja, S.: Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning. Tech. Rep. CMU-CS-94-163, Carnegie-Mellon University (1994)

  • Baluja, S., Davies, S.: Using optimal dependency-trees for combinational optimization. In: 14th International Conference on Machine Learning, pp. 30–38. Morgan Kaufmann, San Mateo (1997)

    Google Scholar 

  • Bengoetxea, E., Larrañaga, P.: EDA-PSO: a hybrid paradigm combining estimation of distribution algorithms and particle swarm optimization. In: Swarm Intelligence. Lecture Notes in Computer Science, vol. 6234, pp. 416–423. Springer, Berlin (2010)

    Chapter  Google Scholar 

  • Bosman, P.A.N., Grahl, J.: Matching inductive search bias and problem structure in continuous estimation of distribution algorithms. Eur. J. Oper. Res. 185(3), 1246–1264 (2008)

    Article  MATH  Google Scholar 

  • Bosman, P.A.N., Grahl, J., Thierens, D.: Enhancing the performance of maximum-likelihood Gaussian EDAs using anticipated mean shift. In: 10th International Conference on Parallel Problem Solving from Nature (PPSN X), pp. 133–143. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Bosman, P.A.N., Thierens, D.: Advancing continuous IDEAs with mixture distributions and factorization selection metrics. In: Optimization by building and using probabilistic models (OBUPM) Workshop at the Genetic and Evolutionary Computation Conference (GECCO’01), pp. 208–212. ACM, New York (2001)

    Google Scholar 

  • Bosman, P.A.N., de Jong, E.: Adaptation of a success story in GAs: Estimation-of-distribution algorithms for tree-based optimization problems. In: Success in Evolutionary Computation. Studies in Computational Intelligence, vol. 92, pp. 3–18. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Bosman, P.A.N., Thierens, D.: Linkage information processing in distribution estimation algorithms. In: Genetic and Evolutionary Computation Conference (GECCO’99), pp. 60–67. Morgan Kaufmann, San Mateo (1999)

    Google Scholar 

  • Bosman, P.A.N., Thierens, D.: Continuous iterated density estimation evolutionary algorithms within the IDEA framework. In: Genetic and Evolutionary Computation Conference (GECCO’00) Workshop, pp. 197–200 (2000a)

    Google Scholar 

  • Bosman, P.A.N., Thierens, D.: Expanding from discrete to continuous estimation of distribution algorithms: the IDEA. In: 6th International Conference on Parallel Problem Solving from Nature (PPSN VI), pp. 767–776. Springer, Berlin (2000b)

    Chapter  Google Scholar 

  • Bouckaert, R.R.: Bayesian belief networks: from construction to inference. Ph.D. Thesis, Universiteit Utrecht, Faculteit Wiskunde en Informatica (1995)

  • Brownlee, A., McCall, J., Zhang, Q., Brown, D.: Approaches to selection and their effect on fitness modelling in an estimation of distribution algorithm. In: IEEE Congress on Evolutionary Computation (CEC 2008)—IEEE World Congress on Computational Intelligence, pp. 2621–2628. IEEE Comput. Soc., Los Alamitos (2008)

    Chapter  Google Scholar 

  • Brownlee, A.E.I.: Multivariate Markov networks for fitness modelling in an estimation of distribution algorithm. Ph.D. Thesis, The Robert Gordon University. School of Computing (2009)

  • Buntine, W.: Theory refinement on Bayesian networks. In: 7th Conference on Uncertainty in Artificial Intelligence (UAI’91), vol. 91, pp. 52–60. Morgan Kaufmann, San Mateo (1991)

    Google Scholar 

  • Chickering, D.: Learning Bayesian networks is NP-complete. In: Learning from Data: Artificial Intelligence and Statistics V. Lecture Notes in Statistics, vol. 112, pp. 121–130. Springer, Berlin (1996)

    Google Scholar 

  • Chickering, D., Geiger, D., Heckerman, D.: Learning Bayesian networks is NP-hard. Tech. Rep. MSR-TR-94-17, Microsoft Research (1994)

  • Chickering, D., Heckerman, D., Meek, C.: Large-sample learning of Bayesian networks is NP-hard. J. Mach. Learn. Res. 5, 1287–1330 (2004)

    MathSciNet  MATH  Google Scholar 

  • Cho, D.Y., Zhang, B.T.: Evolutionary optimization by distribution estimation with mixtures of factor analyzers. In: IEEE Congress on Evolutionary Computation (CEC’02), vol. 2, pp. 1396–1401. IEEE Comput. Soc., Los Alamitos (2002)

    Google Scholar 

  • Cho, D.Y., Zhang, B.T.: Evolutionary continuous optimization by distribution estimation with variational Bayesian independent component analyzers mixture model. In: Parallel Problem Solving from Nature (PPSN VIII). Lecture Notes in Computer Science, vol. 3242, pp. 212–221. Springer, Berlin (2004)

    Chapter  Google Scholar 

  • Cooper, G., Herskovits, E.: A Bayesian method for the induction of probabilistic networks from data. Mach. Learn. 9(4), 309–347 (1992)

    MATH  Google Scholar 

  • Costa, M., Minisci, E.: MOPED: a multi-objective Parzen-based estimation of distribution algorithm for continuous problems. In: Evolutionary Multi-Criterion Optimization. Lecture Notes in Computer Science, vol. 2632, p. 71. Springer, Berlin (2003)

    Chapter  Google Scholar 

  • Cramer, N.L.: A representation for the adaptive generation of simple sequential programs. In: First International Conference on Genetic Algorithms, pp. 183–187. Erlbaum, Hillsdale (1985)

    Google Scholar 

  • Cuesta-Infante, A., Santana, R., Hidalgo, J.I., Bielza, C., Larrañaga, P.: Bivariate empirical and n-variate Archimedean copulas in estimation of distribution algorithms. In: IEEE Congress on Evolutionary Computation (CEC’10) (2010)

    Google Scholar 

  • Dawid, A.P.: Conditional independence in statistical theory. J. R. Stat. Soc. B 41(1), 1–31 (1979)

    MathSciNet  MATH  Google Scholar 

  • De Bonet, J., Isbell, C., Viola, P.M.: Finding optima by estimating probability densities. Adv. Neural Inf. Process. Syst. 9, 424–430 (1997)

    Google Scholar 

  • de Castro, P.A.D., Zuben, F.J.V.: BAIS: a Bayesian artificial immune system for the effective handling of building blocks. Inf. Sci. 179(10), 1426–1440 (2009)

    Article  Google Scholar 

  • Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. B 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  • Ding, N., Zhou, S., Sun, Z.: Histogram-based estimation of distribution algorithm: a competent method for continuous optimization. J. Comput. Sci. Technol. 23(1), 35–43 (2008)

    Article  Google Scholar 

  • Echegoyen, C., Mendiburu, A., Santana, R., Lozano, J.: Analyzing the probability of the optimum in EDAs based on Bayesian networks. In: IEEE Congress on Evolutionary Computation (CEC’09), pp. 1652–1659 (2009)

    Chapter  Google Scholar 

  • Etxeberria, R., Larrañaga, P.: Global optimization using Bayesian networks. In: Second Symposium on Artificial Intelligence (CIMAF-99), pp. 332–339 (1999)

    Google Scholar 

  • Fogel, L.J.: Artificial Intelligence Through Simulated Evolution. Wiley, New York (1966)

    MATH  Google Scholar 

  • Frey, B.J., Dueck, D.: Mixture modeling by affinity propagation. In: Advances in Neural Information Processing Systems, vol. 18, pp. 379–386. MIT Press, Cambridge (2006)

    Google Scholar 

  • Frydenberg, M.: The chain graph Markov property. Scand. J. Stat. 17(4), 333–353 (1990)

    MathSciNet  MATH  Google Scholar 

  • Gámez, J., Mateo, J., Puerta, J.E.: Estimation of dependency networks algorithm. In: Bio-inspired Modeling of Cognitive Tasks. Lecture Notes in Computer Science, vol. 4527, pp. 427–436. Springer, Berlin (2007)

    Chapter  Google Scholar 

  • Geiger, D., Heckerman, D.: Learning Gaussian networks. In: 10th Conference on Uncertainty in Artificial Intelligence (UAI’94), pp. 235–243 (1994)

    Google Scholar 

  • Geman, S., Geman, D.: Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984)

    Article  MATH  Google Scholar 

  • Goldberg, D.E.: The Design of Innovation: Lessons from and for Competent Genetic Algorithms. Kluwer Academic, Norwell (2002)

    MATH  Google Scholar 

  • González, C., Lozano, J., Larrañaga, P.: Mathematical modelling of UMDAc algorithm with tournament selection. Behaviour on linear and quadratic functions. Int. J. Approx. Reason. 31(3), 313–340 (2002)

    Article  MATH  Google Scholar 

  • Grahl, J., Bosman, P.A.N., Rothlauf, F.: The correlation-triggered adaptive variance scaling IDEA. In: 8th Annual Conference on Genetic and Evolutionary Computation (GECCO’06), pp. 397–404. ACM, New York (2006)

    Chapter  Google Scholar 

  • Grünwald, P.: The minimum description length principle and reasoning under uncertainty. Ph.D. Thesis, University of Amsterdam (1998)

  • Hansen, N.: The CMA evolution strategy: a comparing review. In: (Lozano et al. 2006), pp. 75–102 (2006)

  • Harik, G., Cantú-Paz, E., Goldberg, D., Miller, B.: The gambler’s ruin problem, genetic algorithms, and the sizing of populations. Evol. Comput. 7(3), 231–253 (1999)

    Article  Google Scholar 

  • Harik, G.R., Lobo, F.G., Sastry, K.: Linkage learning via probabilistic modeling in the Extended Compact Genetic Algorithm (ECGA). In: (Pelikan et al. 2006), pp. 39–61 (2006). Chap. 3

  • Harik, G., Lobo, F., Goldberg, D.: The compact genetic algorithm. IEEE Trans. Evol. Comput. 3(4), 287–297 (1999)

    Article  Google Scholar 

  • Hasegawa, Y., Iba, H.: A Bayesian network approach to program generation. IEEE Trans. Evol. Comput. 12(6), 750–764 (2008)

    Article  Google Scholar 

  • Heckerman, D., Geiger, D., Chickering, D.: Learning Bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20(3), 197–243 (1995)

    MATH  Google Scholar 

  • Heckerman, D., Chickering, D.M., Meek, C., Rounthwaite, R., Kadie, C.: Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1, 49–75 (2001)

    MATH  Google Scholar 

  • Holland, J.: Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor (1975)

    Google Scholar 

  • Hong, Y., Zhu, G., Kwong, S., Ren, Q.: Estimation of distribution algorithms making use of both high quality and low quality individuals. In: IEEE International Conference on Fuzzy Systems (FUZZ-IEEE’09), pp. 1806–1813. IEEE Comput. Soc., Los Alamitos (2009)

    Google Scholar 

  • Karshenas, H., Nikanjam, A., Helmi, B.H., Rahmani, A.T.: Combinatorial effects of local structures and scoring metrics in Bayesian optimization algorithm. In: First ACM/SIGEVO Summit on Genetic and Evolutionary Computation (GEC’09), pp. 263–270. ACM, New York (2009)

    Chapter  Google Scholar 

  • Karshenas, H., Santana, R., Bielza, C., Larrañaga, P.: Multi-objective optimization with joint probabilistic modeling of objectives and variables. In: Evolutionary Multi-Criterion Optimization. Lecture Notes in Computer Science, vol. 6576, pp. 298–312. Springer, Berlin (2011)

    Chapter  Google Scholar 

  • Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. MIT Press, Cambridge (2009)

    Google Scholar 

  • Koza, J.R.: Genetic Programming: on the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)

    MATH  Google Scholar 

  • Larrañaga, P., Etxeberria, R., Lozano, J., Pena, J.: Optimization by learning and simulation of Bayesian and Gaussian networks. Tech. Rep. EHU-KZAAIK-IK-4/99, Intelligent Systems Group, Department of Computer Science and Artificial Intelligence, University of the Basque Country (1999)

  • Larrañaga, P., Etxeberria, R., Lozano, J., Peña, J.: Combinatonal optimization by learning and simulation of Bayesian networks. In: 16th Conference on Uncertainty in Artificial Intelligence (UAI’00), pp. 343–352. Morgan Kaufmann, San Mateo (2000a)

    Google Scholar 

  • Larrañaga, P., Etxeberria, R., Lozano, J., Peña, J.: Optimization in continuous domains by learning and simulation of Gaussian networks. In: Conference on Genetic and Evolutionary Computation (GECCO’00) Workshop Program, pp. 201–204. Morgan Kaufmann, San Mateo (2000b)

    Google Scholar 

  • Larrañaga, P., Lozano, J. (eds.): Estimation of Distribution Algorithms: a New Tool for Evolutionary Computation. Kluwer Academic, Norwell (2001)

    Google Scholar 

  • Larrañaga, P., Moral, S.: Probabilistic graphical models in artificial intelligence. Appl. Soft Comput. 11(2), 1511–1528 (2011)

    Article  Google Scholar 

  • Lauritzen, S.L., Spiegelhalter, D.J.: Local computations with probabilities on graphical structures and their application to expert systems. J. R. Stat. Soc. B 50(2), 157–224 (1988)

    MathSciNet  MATH  Google Scholar 

  • Li, B., Zhong, R.T., Wang, X.J., Zhuang, Z.Q.: Continuous optimization based-on boosting Gaussian mixture model. In: 18th International Conference on Pattern Recognition (ICPR’06), vol. 1, pp. 1192–1195 (2006)

    Google Scholar 

  • Lima, C., Pelikan, M., Goldberg, D., Lobo, F., Sastry, K., Hauschild, M.: Influence of selection and replacement strategies on linkage learning in BOA. In: CEC 2007, IEEE Congress on Evolutionary Computation, pp. 1083–1090 (2007)

    Chapter  Google Scholar 

  • Lima, C., Pelikan, M., Lobo, F., Goldberg, D.: Loopy substructural local search for the Bayesian optimization algorithm. In: Engineering Stochastic Local Search Algorithms. Designing, Implementing and Analyzing Effective Heuristics. Lecture Notes in Computer Science, vol. 5752, pp. 61–75. Springer, Berlin (2009)

    Chapter  Google Scholar 

  • Lozano, J., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.): Towards a New Evolutionary Computation: Advances on Estimation of Distribution Algorithms. Studies in Fuzziness and Soft Computing, vol. 192. Springer, Berlin (2006)

    MATH  Google Scholar 

  • Luo, N., Qian, F.: Evolutionary algorithm using kernel density estimation model in continuous domain. In: 7th Asian Control Conference (ASCC’09), pp. 1526–1531 (2009)

    Google Scholar 

  • Martí, L., García, J., Berlanga, A., Coello, C.A.C., Molina, J.M.: MB-GNG: addressing drawbacks in multi-objective optimization estimation of distribution algorithms. Oper. Res. Lett. 39(2), 150–154 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • McKay, R., Hoai, N., Whigham, P., Shan, Y., O’Neill, M.: Grammar-based genetic programming: a survey. Genet. Program. Evol. Mach. 11, 365–396 (2010)

    Article  Google Scholar 

  • Mendiburu, A., Santana, R., Lozano, J.A.: Introducing belief propagation in estimation of distribution algorithms: a parallel framework. Tech. Rep. EHU-KAT-IK-11-07, Intelligent Systems Group, University of the Basque Country (2007)

  • Michalski, R.S.: Learnable evolution model: evolutionary processes guided by machine learning. Mach. Learn. 38, 9–40 (2000)

    Article  MATH  Google Scholar 

  • Miquélez, T., Bengoetxea, E., Larrañaga, P.: Evolutionary computation based on Bayesian classifiers. Int. J. Appl. Math. Comput. Sci. 14(3), 335–350 (2004)

    MathSciNet  MATH  Google Scholar 

  • Miquélez, T., Bengoetxea, E., Larrañaga, P.: Evolutionary Bayesian classifier-based optimization in continuous domains. In: 6th International Conference on Simulated Evolution and Learning (SEAL’06). Lecture Notes in Computer Science, vol. 4247, pp. 529–536. Springer, Berlin (2006)

    Chapter  Google Scholar 

  • Mühlenbein, H., Mahnig, T.: FDA–A scalable evolutionary algorithm for the optimization of additively decomposed functions. Evol. Comput. 7(4), 353–376 (1999)

    Article  Google Scholar 

  • Mühlenbein, H., Mahnig, T., Ochoa Rodríguez, A.: Schemata, distributions and graphical models in evolutionary optimization. J. Heuristics 5(2), 215–247 (1999)

    Article  MATH  Google Scholar 

  • Mühlenbein, H., Paaß, G.: From recombination of genes to the estimation of distributions I. Binary parameters. In: 4th International Conference on Parallel Problem Solving from Nature (PPSN IV). Lecture Notes in Computer Science, vol. 1141, pp. 178–187. Springer, Berlin (1996)

    Chapter  Google Scholar 

  • Očenášek, J., Schwarz, J.: Estimation distribution algorithm for mixed continuous-discrete optimization problems. In: Intelligent Technologies: Theory and Applications: New Trends in Intelligent Technologies, pp. 227–232. IOS Press, Amsterdam (2002)

    Google Scholar 

  • Očenášek, J., Kern, S., Hansen, N., Koumoutsakos, P.: A mixed Bayesian optimization algorithm with variance adaptation. In: Parallel Problem Solving from Nature (PPSN VIII). Lecture Notes in Computer Science, vol. 3242, pp. 352–361. Springer, Berlin (2004)

    Chapter  Google Scholar 

  • Pearl, J.: Bayesian networks: a model of self-activated memory for evidential reasoning. In: 7th Conference of the Cognitive Science Society, pp. 329–334 (1985)

    Google Scholar 

  • Pelikan, M., Goldberg, D., Lobo, F.: A survey of optimization by building and using probabilistic models. Comput. Optim. Appl. 21(1), 5–20 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Pelikan, M., Mühlenbein, H.: The bivariate marginal distribution algorithm. In: Advances in Soft Computing-Engineering Design and Manufacturing, pp. 521–535 (1999)

    Google Scholar 

  • Pelikan, M., Sastry, K., Cantú-Paz, E. (eds.): Scalable Optimization via Probabilistic Modeling: from Algorithms to Applications. Springer, Berlin (2006)

    MATH  Google Scholar 

  • Pelikan, M., Sastry, K., Goldberg, D.: Sporadic model building for efficiency enhancement of the hierarchical BOA. Genet. Program. Evol. Mach. 9(1), 53–84 (2008)

    Article  Google Scholar 

  • Pelikan, M.: Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms, 1st edn. Studies in Fuzziness and Soft Computing, vol. 170. Springer, Berlin (2005)

    MATH  Google Scholar 

  • Pelikan, M., Goldberg, D.: Genetic algorithms, clustering, and the breaking of symmetry. In: Parallel Problem Solving from Nature (PPSN VI). Lecture Notes in Computer Science, vol. 1917, pp. 385–394. Springer, Berlin (2000)

    Chapter  Google Scholar 

  • Pelikan, M., Goldberg, D.E., Cantú-Paz, E.B.: The Bayesian optimization algorithm. In: Conference on Genetic and Evolutionary Computation (GECCO’99), vol. 1, pp. 525–532. Morgan Kaufmann, San Mateo (1999)

    Google Scholar 

  • Pelikan, M., Hartmann, A.: Searching for ground states of Ising spin glasses with hierarchical BOA and cluster exact approximation. In: (Pelikan et al. 2006), pp. 333–349 (2006)

  • Pelikan, M., Sastry, K.: Fitness inheritance in the Bayesian optimization algorithm. In: Conference on Genetic and Evolutionary Computation (GECCO’04). Lecture Notes in Computer Science, vol. 3103, pp. 48–59. Springer, Berlin (2004)

    Google Scholar 

  • Peña, J.M., Lozano, J.A., Larrañaga, P.: Globally multimodal problem optimization via an estimation of distribution algorithm based on unsupervised learning of Bayesian networks. Evol. Comput. 13(1), 43–66 (2005)

    Article  Google Scholar 

  • Pošík, P.: Preventing premature convergence in a simple EDA via global step size setting. In: 10th International Conference on Parallel Problem Solving from Nature (PPSN X). Lecture Notes in Computer Science, vol. 5199, pp. 549–558. Springer, Berlin (2008)

    Chapter  Google Scholar 

  • Pošík, P.: BBOB-benchmarking a simple estimation of distribution algorithm with cauchy distribution. In: 11th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO’09), pp. 2309–2314. ACM, New York (2009a)

    Google Scholar 

  • Pošík, P.: Stochastic local search techniques with unimodal continuous distributions: a survey. In: EvoWorkshops on Applications of Evolutionary Computing (EvoWorkshops’09), pp. 685–694. Springer, Berlin (2009b)

    Google Scholar 

  • Rechenberg, I.: Evolutionsstrategie-Optimierung Technischer Systeme nach Prinzipien der Biologischen Evolution. Ph.D. Thesis, reprinted by Fromman-Holzboog (1973)

  • Rissanen, J.: Modeling by shortest data description. Automatica 14(5), 465–471 (1978)

    Article  MATH  Google Scholar 

  • Robinson, R.: Counting unlabeled acyclic digraphs. In: Combinatorial Mathematics V. Lecture Notes in Mathematics, vol. 622, pp. 28–43. Springer, Berlin (1977)

    Chapter  Google Scholar 

  • Salinas-Gutiérrez, R., Hernández-Aguirre, A., Villa-Diharce, E.: Using copulas in estimation of distribution algorithms. In: Advances in Artificial Intelligence (MICAI’09). Lecture Notes in Computer Science, vol. 5845, pp. 658–668. Springer, Berlin (2009)

    Google Scholar 

  • Sałustowicz, R.P., Schmidhuber, J.: Probabilistic incremental program evolution: stochastic search through program space. In: 9th European Conference on Machine Learning (ECML’97). Lecture Notes in Computer Science, vol. 1224, pp. 213–220. Springer, Berlin (1997)

    Google Scholar 

  • Santana, R., Bielza, C., Lozano, J., Larrañaga, P.: Mining probabilistic models learned by EDAs in the optimization of multi-objective problems. In: 11th Annual Conference on Genetic and Evolutionary Computation (GECCO’09), pp. 445–452. ACM, New York (2009a)

    Chapter  Google Scholar 

  • Santana, R., Larrañaga, P., Lozano, J.: Research topics in discrete estimation of distribution algorithms based on factorizations. Memet. Comput. 1(1), 35–54 (2009b)

    Article  Google Scholar 

  • Santana, R., Larrañaga, P., Lozano, J.: Learning factorizations in estimation of distribution algorithms using affinity propagation. Evol. Comput. 18(4), 515–546 (2010)

    Article  Google Scholar 

  • Santana, R.: A Markov network based factorized distribution algorithm for optimization. In: 14th European Conference on Machine Learning (ECML’03). Lecture Notes in Computer Science, vol. 2837, pp. 337–348. Springer, Berlin (2003)

    Google Scholar 

  • Santana, R.: Estimation of distribution algorithms with Kikuchi approximations. Evol. Comput. 13, 67–97 (2005)

    Article  Google Scholar 

  • Santana, R.: Estimation of distribution algorithms: from available implementations to potential developments. In: 13th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO’11), pp. 679–686. ACM, New York (2011)

    Google Scholar 

  • Santana, R., Bielza, C., Larrañaga, P., Lozano, J.A., Echegoyen, C., Mendiburu, A., Armañanzas, R., Shakya, S.: Mateda-2.0: estimation of distribution algorithms in MATLAB. J. Stat. Softw. 35(7), 1–30 (2010)

    Google Scholar 

  • Sastry, K., Goldberg, D.E.: Probabilistic model building and competent genetic programming. In: Genetic Programming Theory and Practice, pp. 205–220. Kluwer Academic, Norwell (2003). Chap. 13

    Chapter  Google Scholar 

  • Sastry, K., Pelikan, M., Goldberg, D.: Efficiency enhancement of genetic algorithms via building-block-wise fitness estimation. In: IEEE Congress on Evolutionary Computation (CEC’04), vol. 1, pp. 720–727 (2004)

    Google Scholar 

  • Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)

    Article  MATH  Google Scholar 

  • Sebag, M., Ducoulombier, A.: Extending population-based incremental learning to continuous search spaces. In: 5th International Conference on Parallel Problem Solving from Nature (PPSN V). Lecture Notes in Computer Science, vol. 1498, pp. 418–427. Springer, Berlin (1998)

    Chapter  Google Scholar 

  • Shakya, S.: DEUM: a framework for an estimation of distribution algorithm based on Markov random fields. Ph.D. Thesis, The Robert Gordon University (2006)

  • Shakya, S., Santana, R. (eds.): Markov Networks in Evolutionary Computation. Adaptation, Learning, and Optimization, vol. 14. Springer, Berlin (2012)

    MATH  Google Scholar 

  • Shan, Y., McKay, R., Essam, D., Abbass, H.: a survey of probabilistic model building genetic programming. In: (Pelikan et al. 2006), pp. 121–160 (2006)

  • Spirtes, P., Glymour, C.: An algorithm for fast recovery of sparse causal graphs. Soc. Sci. Comput. Rev. 9(1), 62–72 (1991)

    Article  Google Scholar 

  • Spirtes, P., Glymour, C., Scheines, R.: Causation, Prediction, and Search, 2nd edn. MIT Press, Cambridge (2001)

    MATH  Google Scholar 

  • Sun, J., Zhang, Q., Tsang, E.: DE/EDA: a new evolutionary algorithm for global optimization. Inf. Sci. 169(3–4), 249–262 (2005)

    Article  MathSciNet  Google Scholar 

  • Thierens, D.: The linkage tree genetic algorithm. In: Parallel Problem Solving from Nature (PPSN XI). Lecture Notes in Computer Science, vol. 6238, pp. 264–273. Springer, Berlin (2011)

    Google Scholar 

  • Tsutsui, S., Pelikan, M., Goldberg, D.: Node histogram vs. edge histogram: a comparison of pmbgas in permutation domains. Tech. Rep. 2006009, Missouri Estimation of Distribution Algorithms Laboratory (MEDAL), Department of Mathematics and Computer Science, University of Missouri–St. Louis (2006)

  • Tsutsui, S.: Probabilistic model-building genetic algorithms in permutation representation domain using edge histogram. In: Parallel Problem Solving from Nature (PPSN VII). Lecture Notes in Computer Science, vol. 2439, pp. 224–233. Springer, Berlin (2002)

    Google Scholar 

  • Tsutsui, S., Pelikan, M., Goldberg, D.E.: Evolutionary algorithm using marginal histogram in continuous domain. In: Optimization by Building and Using Probabilistic Models (OBUPM) Workshop—Conference on Genetic and Evolutionary Computation (GECCO’01), pp. 230–233 (2001)

    Google Scholar 

  • Valdez-Peña, S.I., Hernández-Aguirre, A., Botello-Rionda, S.: Approximating the search distribution to the selection distribution in EDAs. In: 11th Annual Conference on Genetic and Evolutionary Computation (GECCO’09), pp. 461–468. ACM, New York (2009)

    Chapter  Google Scholar 

  • Wang, L.F., Zeng, J.C.: Estimation of distribution algorithm based on copula theory. In: Exploitation of Linkage Learning in Evolutionary Algorithms. Evolutionary Learning and Optimization, vol. 3, pp. 139–162. Springer, Berlin (2010)

    Chapter  Google Scholar 

  • Wang, L.F., Zeng, J.C., Hong, Y.: Estimation of distribution algorithm based on Archimedean copulas. In: First ACM/SIGEVO Summit on Genetic and Evolutionary Computation (GEC’09), pp. 993–996. ACM, New York (2009)

    Chapter  Google Scholar 

  • Wang, X., Wang, H.: Evolutionary optimization with Markov random field prior. IEEE Trans. Evol. Comput. 8(6), 567–579 (2004)

    Article  Google Scholar 

  • Weise, T., Niemczyk, S., Chiong, R., Wan, M.: A framework for multi-model EDAs with model recombination. In: Applications of Evolutionary Computation. Lecture Notes in Computer Science, vol. 6624, pp. 304–313. Springer, Berlin (2011)

    Chapter  Google Scholar 

  • Xiao, J., Yan, Y., Zhang, J.: HPBILc: a histogram-based EDA for continuous optimization. Appl. Math. Comput. 215(3), 973–982 (2009)

    Article  MATH  Google Scholar 

  • Yanai, K., Iba, H.: Estimation of distribution programming based on Bayesian network. In: IEEE Congress on Evolutionary Computation (CEC’03), vol. 3, pp. 1618–1625 (2003)

    Google Scholar 

  • Zhang, Q.: On stability of fixed points of limit models of univariate marginal distribution algorithm and factorized distribution algorithm. IEEE Trans. Evol. Comput. 8(1), 80–93 (2004)

    Article  Google Scholar 

  • Zhang, Q., Zhou, A., Jin, Y.: RM-MEDA: a regularity model based multiobjective estimation of distribution algorithm. IEEE Trans. Evol. Comput. 12(1), 41–63 (2008)

    Article  Google Scholar 

Download references

Acknowledgements

This work has been partially supported by TIN2010-20900-C04-04, Consolider Ingenio 2010-CSD2007-00018 and Cajal Blue Brain projects (Spanish Ministry of Science and Innovation).

Author information

Authors and Affiliations

  1. Computational Intelligence Group, Facultad de Informática, Universidad Politécnica de Madrid, Campus de Montegancedo, 28660, Boadilla del Monte, Madrid, Spain

    Pedro Larrañaga, Hossein Karshenas & Concha Bielza

  2. Intelligent System Group, Department of Computer Science and Artificial Intelligence, University of the Basque Country, Paseo Manuel de Lardizbal 1, 20080, San Sebastin, Donostia, Spain

    Roberto Santana

Authors
  1. Pedro Larrañaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Karshenas
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Concha Bielza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Roberto Santana
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Karshenas.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Larrañaga, P., Karshenas, H., Bielza, C. et al. A review on probabilistic graphical models in evolutionary computation. J Heuristics 18, 795–819 (2012). https://doi.org/10.1007/s10732-012-9208-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10732-012-9208-4

Keywords

Search

Navigation