Abstract
A few schema theorems for genetic programming (GP) have been proposed in the literature in the last few years. Since they consider schema survival and disruption only, they can only provide a lower bound for the expected value of the number of instances of a given schema at the next generation rather than an exact value. This paper presents theoretical results for GP with one-point crossover which overcome this problem. First, we give an exact formulation for the expected number of instances of a schema at the next generation in terms of microscopic quantities. Due to this formulation we are then able to provide an improved version of an earlier GP schema theorem in which some (but not all) schema creation events are accounted for. Then, we extend this result to obtain an exact formulation in terms of macroscopic quantities which makes all the mechanisms of schema creation explicit. This theorem allows the exact formulation of the notion of effective fitness in GP and opens the way to future work on GP convergence, population sizing, operator biases, and bloat, to mention only some of the possibilities.
Similar content being viewed by others
References
L. Altenberg, “Emergent phenomena in genetic programming,” in Evolutionary Programming-Proc. Third Ann. Conf., A. V. Sebald and L. J. Fogel (eds.), World Scientific Publishing: Singapore, 1994, pp. 233–241.
L. Altenberg, “The schema theorem and Price's theorem,” in Foundations of Genetic Algorithms, L. D. Whitley and M. D. Vose (eds.), Estes Park, Colorado, USA, 31 July-2 Aug, 1994. Morgan Kaufmann: San Francisco, 1995, vol. 3, pp. 23–49.
T. Bäck and D. B. Fogel, “Glossary,” in Evolutionary Computation 1: Basic Algorithms and Operators, T. Bäck, D. B. Fogel, and T. Michalewicz, (eds.), Institute of Physics Publishing: Bristol and Philadelphia, 2000.
W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic Programming-An Introduction; On the Automatic Evolution of Computer Programs and its Applications, Morgan Kaufmann: San Francisco, 1998.
S. W. Chung and R. A. Perez, “The schema theorem considered insufficient,” in Proc. Sixth IEEE Int. Conf. Tools with Artif. Intell., New Orleans, IEEE Press: Piscataway, NJ, Nov 6–9 1994, pp. 748–751.
T. E. Davis and J. C. Principe, “AMarkov chain framework for the simple genetic algorithm,” Evol. Comput., vol. 1(3) pp. 269–288, 1993.
K. A. De Jong, W. M. Spears, and D. F. Gordon, “Using Markov chains to analyze GAFOs,” in Foundations of Genetic Algorithms, L. D. Whitley and M. D. Vose (eds.), Morgan Kaufmann: San Francisco, CA, 1995, vol. 3, pp. 115–137.
D. B. Fogel and A. Ghozeil, “Schema processing under proportional selection in the presence of random effects,” IEEE Trans. Evol. Comput., vol. 1(4), pp. 290–293, 1997.
D. B. Fogel and A. Ghozeil, “The schema theorem and the misallocation of trials in the presence of stochastic effects,” in Evolutionary Programming VII: Proc. of the 7th Ann. Conf. on Evolutionary Programming, V. W. Porto, N. Saravanan, D. Waagen, and A. E. Eiben (eds.), Springer: Berlin, 1998, pp. 313–321.
D. E. Goldberg. Genetic algorithms and Walsh fuctions: II. Deception and its analysis. Complex Systems, vol. 3(2) pp. 153–171, 1989.
D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley: Reading, MA, 1989.
J. J. Grefenstette, “Deception considered harmful,” in Foundations of Genetic Algorithms, L. D. Whitley (ed.), San Mateo, CA, Morgan Kaufman: San Francisco, 1993, vol. 2.
J. Holland, Adaptation in Natural and Artificial Systems, University of Michigan Press: Ann Arbor, USA, 1975.
J. H. Holland, “Building blocks, cohort genetic algorithms, and hyperplane-defined functions,” Evol. Comput., vol. 8(4) pp. 373–391, 2000.
J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection, MIT Press: Cambridge, MA, 1992.
W. B. Langdon, “Size fair and homologous tree genetic programming crossovers,” in Proc. Genetic and Evolutionary Computation Conference, W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (eds.), Orlando, FL, USA, Morgan Kaufmann: San Francisco, July 13–17 1999, vol. 2, pp. 1092–1097.
W. B. Langdon, “Size fair and homologous tree genetic programming crossovers,” Genet. Program. Evol. Mach., vol. 1(1/2) pp. 95–119, 2000.
W. B. Langdon and R. Poli, “Why ants are hard,” in Genetic Programming 1998: Proc. Third Ann. Conf., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), University of Wisconsin, Madison, Wisconsin, USA, Morgan Kaufmann: San Francisco, July 22–25 1998, pp. 193–201.
N. F. McPhee and R. Poli, “Aschema theory analysis of the evolution of size in genetic programming with linear representations,” in Genetic Programming, Proceedings of EuroGP 2001, LNCS, Milan, Springer-Verlag: Berlin, April 18–20 2001, vol. 2038, pp. 108–125.
N. F. McPhee, R. Poli, and J. E. Rowe, “Aschema theory analysis of mutation size biases in genetic programming with linear representations,” Technical Report CSRP–00–24, University of Birmingham, School of Computer Science, December 2000.
A. E. Nix and M. D. Vose, “Modeling genetic algorithms with Markov chains,” Ann. Math. Artif. Intell., vol. 5 pp. 79–88, 1992.
P. Nordin and W. Banzhaf, “Complexity compression and evolution,” in Genetic Algorithms: Proc. Sixth Int. Conf. (ICGA95), L. Eshelman (ed.), Pittsburgh, PA, USA, Morgan Kaufmann: San Francisco, July 15–19 1995, pp. 310–317.
P. Nordin, F. Francone, and W. Banzhaf, “Explicitly defined introns and destructive crossover in genetic programming,” in Proceedings of the Workshop on Genetic Programming: From Theory to Real-World Applications, J. P. Rosca (ed.), Tahoe City, California, USA, July 9 1995, pp. 6–22.
U.-M. O'Reilly and F. Oppacher, “The troubling aspects of a building block hypothesis for genetic programming,” in Foundations of Genetic Algorithms, L. D. Whitley and M. D. Vose (eds.), Estes Park, Colorado, USA, Morgan Kaufmann: San Francisco, 31 July-2 August 1994, 1995, vol. 3, pp. 73–88.
J. Page, R. Poli, and W. B. Langdon, “Smooth uniform crossover with smooth point mutation in genetic programming: Apreliminary study,” in Genetic Programming, Proceedings of EuroGP'99, R. Poli, P. Nordin, W. B. Langdon, and T. C. Fogarty (eds.), volume 1598 of LNCS, Goteborg, Sweden, Springer-Verlag: Berlin, May 26–27 1999, pp. 39–49.
R. Poli, “Schema theorems without expectations,” in Proceedings of the Genetic and Evolutionary Computation Conference, W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (eds.), Orlando, FL, USA, Morgan Kaufmann: San Francisco, July 13–17 1999, vol. 1, p. 806.
R. Poli, “Exact schema theorem and effective fitness for GP with one-point crossover,” in Proc. Genet. Evol. Comput. Conf., D. Whitley, D. Goldberg, E. Cantu-Paz, L. Spector, I. Parmee, and H.-G. Beyer (eds.), Las Vegas, Morgan Kaufmann: San Francisco, July 2000, pp. 469–476.
R. Poli, “Hyperschema theory for GP with one-point crossover, building blocks, and some new results in GAtheory,” in Genetic Programming, Proceedings of EuroGP'2000, R. Poli, W. Banzhaf, W. B. Langdon, J. F. Miller, P. Nordin, and T. C. Fogarty (eds.), volume 1802 of LNCS, Edinburgh, Springer-Verlag: Berlin, April 15–16 2000, pp. 163–180.
R. Poli, “Recursive conditional schema theorem, convergence and population sizing in genetic algorithms,” in Proceedings of the Foundations of Genetic Algorithms Workshop (FOGA 6), W. M. Spears and W. Martin (eds.), Charlottesville, VA, USA, July 2000, in press.
R. Poli. “Why the schema theorem is correct also in the presence of stochastic effects,” in Proc. Congr. Evol. Comput. (CEC 2000), San Diego, USA, IEEE: Piscataway, NJ, July 2000, pp. 487–492.
R. Poli, “General schema theory for genetic programming with subtree-swapping crossover,” in Genetic Programming, Proceedings of EuroGP 2001, LNCS, Milan, Springer-Verlag: Berlin, April 18–20 2001, vol. 2038, pp. 143–159.
R. Poli and W. B. Langdon, “An experimental analysis of schema creation, propagation and disruption in genetic programming,” in Genetic Algorithms: Proc. Seventh Int. Conf., T. Back (ed.), Michigan State University, East Lansing, MI, USA, Morgan Kaufmann: San Francisco, July 19–23 1997, pp. 18–25.
R. Poli and W. B. Langdon, “Anew schema theory for genetic programming with one-point crossover and point mutation,” in Genetic Programming 1997: Proc. Sec. Annu. Conf., J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (eds.), Stanford University, CA, USA, Morgan Kaufmann: San Francisco, July 13–16 1997, pp. 278–285.
R. Poli and W. B. Langdon, “On the search properties of different crossover operators in genetic programming,” in Genetic Programming 1998: Proc. Third Annu. Conf., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), University of Wisconsin, Madison, Wisconsin, USA, Morgan Kaufmann: San Francisco, July 22–25 1998, pp. 293–301.
R. Poli and W. B. Langdon, “Areview of theoretical and experimental results on schemata in genetic programming,” in Proceedings of the First European Workshop on Genetic Programming, W. Banzhaf, R. Poli, M. Schoenauer, and T. C. Fogarty (eds.), volume 1391 of LNCS, Paris, Springer-Verlag: Berlin, April 14–15 1998, pp. 1–15.
R. Poli and W. B. Langdon, “Schema theory for genetic programming with one-point crossover and point mutation,” Evol. Comput., vol. 6(3) pp. 231–252, 1998.
R. Poli, W. B. Langdon, and U.-M. O'Reilly, “Analysis of schema variance and short term extinction likelihoods,” in Genetic Programming 1998: Proc. Third Annu. Conf., J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Garzon, D. E. Goldberg, H. Iba, and R. Riolo (eds.), University of Wisconsin, Madison, Wisconsin, USA, Morgan Kaufmann: San Francisco, July 22–25 1998, pp. 284–292.
R. Poli and N. F. McPhee, “Exact GP schema theory for headless chicken crossover and subtree mutation,” Technical Report CSRP–00–23, University of Birmingham, School of Computer Science, December 2000.
R. Poli and N. F. McPhee, “Exact schema theorems for GP with one-point and standard crossover operating on linear structures and their application to the study of the evolution of size,” in Genetic Programming, Proceedings of EuroGP 2001, LNCS, Milan, Springer-Verlag: Berlin, April 18–20 2001, vol. 2038, pp. 126–142.
R. Poli and J. Page, “Solving high-order boolean parity problems with smooth uniform crossover, sub-machine code GP and demes,” Genet. Program. Evol. Mach., vol. 1(1/2) pp. 37–56, 2000.
R. Poli, J. Page, and W. B. Langdon, “Smooth uniform crossover, sub-machine code GP and demes: Arecipe for solving high-order boolean parity problems,” in Proc. Genet. Evol. Comput. Conf., W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (eds.), Orlando, FL, USA, Morgan Kaufmann: San Francisco, July 13–17 1999, vol. 2, pp. 1162–1169.
A. Prögel-Bennett and J. L. Shapiro, “An analysis of genetic algorithms using statistical mechanics,” Physical Review Letters, vol. 72 pp. 1305–1309, 1994.
N. J. Radcliffe, “Schema processing,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: NY, 1997, pp. B2–5/1–10.
J. P. Rosca, “Analysis of complexity drift in genetic programming,” in Genetic Programming 1997: Proc. Second Annu. Conf., J. R. Koza, K. Deb, M. Dorigo, D. B. Fogel, M. Garzon, H. Iba, and R. L. Riolo (eds.), Stanford University, CA, USA, Morgan Kaufmann: San Francisco, July 13–16 1997, pp. 286–294.
J. E. Rowe, “Population fixed-points for functions of unitation,” in Foundations of Genetic Algorithms, W. Banzhaf and C. Reeves (eds.), Morgan Kaufmann: San Francisco, 1999, vol. 5 pp. 69–84.
G. Rudolph, “Convergence analysis of canonical genetic algorithm,” IEEE Trans. Neural Networks, vol. 5(1) pp. 96–101, 1994.
G. Rudolph, “Genetic algorithms,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: NY, 1997, pp. B2.4–20–27.
G. Rudolph, “Models of stochastic convergence,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: NY, 1997, pp. B2.3–1–3.
G. Rudolph, “Stochastic processes,” in Handbook of Evolutionary Computation, T. Baeck, D. B. Fogel, and Z. Michalewicz (eds.), IOP Press: New York, 1997, pp. B2.2–1–8.
W. M. Spears, “Aggregating models of evolutionary algorithms,” in Proc. Congr. Evol. Comput., P. J. Angeline, Z. Michalewicz, M. Schoenauer, X. Yao, and A. Zalzala, (eds.), Mayflower Hotel, Washington D.C., USA, IEEE Press: Piscataway, NJ, July 6–9 1999, vol. 1, pp. 631–638.
M. R. Spiegel, Probability and Statistics, McGraw-Hill: New York, 1975.
C. R. Stephens and J. M. Vargas, “Effective fitness as an alternative paradigm for evolutionary computation I: general formalism,” Genet. Program. Evol. Mach., vol. 1(4) pp. 363–378, 2000.
C. R. Stephens and J. M. Vargas, “Effective fitness as an alternative paradigm for evolutionary computation II: examples and applications,” Genet. Program. Evol. Mach., 2001, vol. 2(1), March, pp. 1–32.
C. R. Stephens and H. Waelbroeck, “Effective degrees of freedom in genetic algorithms and the block hypothesis,” in Proc. Seventh Int. Conf. Genet. Algo. (ICGA97), T. Bäck (ed.), East Lansing, Morgan Kaufmann: San Francisco, 1997, pp. 34–40.
C. R. Stephens and H. Waelbroeck, “Schemata evolution and building blocks,” Evol. Comput., vol. 7(2) pp. 109–124, 1999.
C. R. Stephens, H. Waelbroeck, and R. Aguirre, “Schemata as building blocks: Does size matter? in Foundations of Genetic Algorithms, W. Banzhaf and C. Reeves (eds.), Morgan Kaufmann: San Francisco, CA, 1999, vol. 5, pp. 117–133.
M. D. Vose, The Simple Genetic Algorithm: Foundations and Theory. MIT Press: Cambridge, MA, 1999.
P. A. Whigham, “A schema theorem for context-free grammars,” in 1995 IEEE Conf. Evol. Comput., Perth, Australia, IEEE Press: Piscataway, NJ, 29 November-1 December 1995, vol. 1, pp. 178–181.
P. A. Whigham, “Grammatical bias for evolutionary learning,” PhD thesis, School of Computer Science, University College, University of New South Wales, Australian Defence Force Academy, October 14 1996.
D. fWhitley, “Agenetic algorithm tutorial,” Technical Report CS–93–103, Department of Computer Science, Colorado State University, August 1993.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Poli, R. Exact Schema Theory for Genetic Programming and Variable-Length Genetic Algorithms with One-Point Crossover. Genetic Programming and Evolvable Machines 2, 123–163 (2001). https://doi.org/10.1023/A:1011552313821
Issue Date:
DOI: https://doi.org/10.1023/A:1011552313821