Skip to main content
SpringerLink
Log in

Differentiable Genetic Programming

  • Conference paper
  • First Online:
Genetic Programming (EuroGP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10196))

Included in the following conference series:

Abstract

We introduce the use of high order automatic differentiation, implemented via the algebra of truncated Taylor polynomials, in genetic programming. Using the Cartesian Genetic Programming encoding we obtain a high-order Taylor representation of the program output that is then used to back-propagate errors during learning. The resulting machine learning framework is called differentiable Cartesian Genetic Programming (dCGP). In the context of symbolic regression, dCGP offers a new approach to the long unsolved problem of constant representation in GP expressions. On several problems of increasing complexity we find that dCGP is able to find the exact form of the symbolic expression as well as the constants values. We also demonstrate the use of dCGP to solve a large class of differential equations and to find prime integrals of dynamical systems, presenting, in both cases, results that confirm the efficacy of our approach.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The exact details on all these experiments can be found in two IPython notebooks available here: https://goo.gl/iH5GAR, https://goo.gl/0TFsSv.

  2. 2.

    The exact details on all these experiments can be found in the IPython notebook available here: https://goo.gl/8fOzYM.

  3. 3.

    The full details on these experiments can be found in an IPython notebook available here: https://goo.gl/wnCkO9.

  4. 4.

    The full details on our experiments can be found in an IPython notebook available here: https://goo.gl/ATrQR5.

References

  1. Auger, A., Hansen, N.: Performance evaluation of an advanced local search evolutionary algorithm. In: 2005 IEEE Congress on Evolutionary Computation, vol. 2, pp. 1777–1784. IEEE (2005)

    Google Scholar 

  2. Bertz, M.: Modern Map Methods in Particle Beam Physics, vol. 108. Academic Press, Cambridge (1999)

    Google Scholar 

  3. Borisov, A.V., Kholmskaya, A., Mamaev, I.S.: S.V. Kovalevskaya top and generalizations of integrable systems. Regul. Chaotic Dyn. 6(1), 1–16 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  4. Casasayas, J., Nunes, A., Tufillaro, N.: Swinging Atwood’s machine: integrability and dynamics. J. de Phys. 51(16), 1693–1702 (1990)

    Article  MathSciNet  Google Scholar 

  5. Cerny, B.M., Nelson, P.C., Zhou, C.: Using differential evolution for symbolic regression and numerical constant creation. In: Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation, pp. 1195–1202. ACM (2008)

    Google Scholar 

  6. Graves, A., Wayne, G., Danihelka, I.: Neural turing machines. arXiv preprint arXiv:1410.5401 (2014)

  7. Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S.G., et al.: Hybrid computing using a neural network with dynamic external memory. Nature 538(7626), 471–476 (2016)

    Article  Google Scholar 

  8. Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a new evolutionary computation. SFSC, vol. 192, pp. 75–102. Springer, Heidelberg (2006). doi:10.1007/3-540-32494-1_4

    Chapter  Google Scholar 

  9. Izzo, D.: dCGP: first release, November 2016. https://doi.org/10.5281/zenodo.164627

  10. Izzo, D., Biscani, F.: AuDi: first release, November 2016. https://doi.org/10.5281/zenodo.164628

  11. Khan, M.M., Ahmad, A.M., Khan, G.M., Miller, J.F.: Fast learning neural networks using cartesian genetic programming. Neurocomputing 121, 274–289 (2013)

    Article  Google Scholar 

  12. Koza, J.: Tutorial on advanced genetic programming, at genetic programming 1997, Palo Alto, CA (1997)

    Google Scholar 

  13. Miller, J.F.: Cartesian genetic programming. In: Miller, J.F. (ed.) Cartesian Genetic Programming. NCS, pp. 17–34. Springer, Heidelberg (2011). doi:10.1007/978-3-642-17310-3_2

    Chapter  Google Scholar 

  14. O’Neill, M., Vanneschi, L., Gustafson, S., Banzhaf, W.: Open issues in genetic programming. Genet. Program. Evolvable Mach. 11(3–4), 339–363 (2010)

    Article  Google Scholar 

  15. Schmidt, M., Lipson, H.: Distilling free-form natural laws from experimental data. Science 324(5923), 81–85 (2009)

    Article  Google Scholar 

  16. Schmidt, M., Lipson, H.: Symbolic regression of implicit equations. In: Riolo, R., O’Reilly, U.-M., McConaghy, T. (eds.) Genetic Programming Theory and Practice VII. GEC, pp. 73–85. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  17. Topchy, A., Punch, W.F.: Faster genetic programming based on local gradient search of numeric leaf values. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2001), pp. 155–162 (2001)

    Google Scholar 

  18. Tsoulos, I.G., Lagaris, I.E.: Solving differential equations with genetic programming. Genet. Program. Evolvable Mach. 7(1), 33–54 (2006)

    Article  Google Scholar 

  19. Turner, A.J., Miller, J.F.: Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks. In: Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 1005–1012. ACM (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Advanced Concepts Team, European Space Agency, Noordwijk, The Netherlands

    Dario Izzo & Alessio Mereta

  2. Interdisciplinary Center for Scientific Computing, Heidelberg University, Heidelberg, Germany

    Francesco Biscani

Authors
  1. Dario Izzo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Francesco Biscani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alessio Mereta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dario Izzo .

Editor information

Editors and Affiliations

  1. University College Dublin , Dublin, Ireland

    James McDermott

  2. Universidade Nova de Lisboa , Lisbon, Portugal

    Mauro Castelli

  3. Brno University of Technology , Brno, Czech Republic

    Lukas Sekanina

  4. Vrije Universiteit Amsterdam , Amsterdam, The Netherlands

    Evert Haasdijk

  5. University of Cádiz , Cádiz, Spain

    Pablo García-Sánchez

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Izzo, D., Biscani, F., Mereta, A. (2017). Differentiable Genetic Programming. In: McDermott, J., Castelli, M., Sekanina, L., Haasdijk, E., García-Sánchez, P. (eds) Genetic Programming. EuroGP 2017. Lecture Notes in Computer Science(), vol 10196. Springer, Cham. https://doi.org/10.1007/978-3-319-55696-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-55696-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-55695-6

  • Online ISBN: 978-3-319-55696-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation