Abstract
Genetic programming (GP) has developed to the point where it is a credible candidate for the ‘black box’ modeling of real systems. Wider application, however, could greatly benefit from its seamless embedding in conventional optimization schemes, which are most efficiently carried out using gradient-based methods. This paper describes the development of a method to automatically differentiate GP trees using a series of tree transformation rules; the resulting method can be applied an unlimited number of times to obtain higher derivatives of the function approximated by the original, trained GP tree. We demonstrate the utility of our method using a number of illustrative gradient-based optimizations that embed GP models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We have used NLopt version 2.5 downloadable from https://github.com/stevengj/nlopt/archive/v2.5.0.tar.gz.
References
Bartholomew-Biggs, M.C., Brown, S., Christianson, B., Dixon, L.C.W.: Automatic differentiation of algorithms. J. Comput. Appl. Math. 124, 171–190 (2000). https://doi.org/10.1016/S0377-0427(00)00422-2
Baydin, A.G., Pearlmutter, B.A., Radul, A.A., Siskind, J.M.: Automatic differentiation in machine learning: a survey. J. Mach. Learn. Res. 18(1), 5595–5637 (2017). http://jmlr.org/papers/v18/17-468.html
Camacho, E.F., Bordons, C.: Model Predictive Control, 2nd edn. Springer, London (2004)
Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. Society for Industrial and Applied Mathematics (2009). https://doi.org/10.1137/1.9780898718768
Gandomi, A.H., Alavi, A.H., Ryan, C. (eds.): Handbook of Genetic Programming Applications. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-20883-1
Goldberg, D.E.: Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley, Reading (1989)
Hock, W., Schittkowski, K.: Test Examples for Nonlinear Programming Codes. Lecture Notes in Economics and Mathematical Systems, vol. 187. Springer, Heidelberg (1981). https://doi.org/10.1007/978-3-642-48320-2
Izzo, D., Biscani, F., Mereta, A.: Differentiable genetic programming. In: 20th European Conference (EuroGP 2017), Amsterdam, The Netherlands, pp. 35–51 (2017). https://doi.org/10.1007/978-3-319-55696-3_3
Johnson, S.G.: The NLopt nonlinear-optimization package (2019). http://ab-initio.mit.edu/nlopt
Keijzer, M.: Improving symbolic regression with interval arithmetic and linear scaling. In: Ryan, C., Soule, T., Keijzer, M., Tsang, E., Poli, R., Costa, E. (eds.) EuroGP 2003. LNCS, vol. 2610, pp. 70–82. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36599-0_7
Kraft, D.: Algorithm 733: TOMP-Fortran modules for optimal control calculations. ACM Trans. Math. Softw. 20(3), 262–281 (1994). https://doi.org/10.1145/192115.192124
Mousavi Astarabadi, S.S., Ebadzadeh, M.M.: Avoiding overfitting in symbolic regression using the first order derivative of GP trees. In: Genetic and Evolutionary Computation Conference (GECCO Companion 2015), Madrid, Spain, pp. 1441–1442, 11–15 July 2015. https://doi.org/10.1145/2739482.2764662
Ni, J., Drieberg, R.H., Rockett, P.I.: The use of an analytic quotient operator in genetic programming. IEEE Trans. Evol. Comput. 17(1), 146–152 (2013). https://doi.org/10.1109/TEVC.2012.2195319
Rockett, P., Hathway, E.A.: Model-predictive control for non-domestic buildings: critical review and prospects. Build. Res. Inform. 45(5), 556–571 (2017). https://doi.org/10.1080/09613218.2016.1139885
Rudin, W.: Principles of Mathematical Analysis. McGraw-Hill, New York (1976)
Wächter, A., Biegler, L.T.: On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming. Math. Program. 106(1), 25–57 (2006). https://doi.org/10.1007/s10107-004-0559-y
Waltz, R.A., Morales, J.L., Nocedal, J., Orban, D.: An interior algorithm for nonlinear optimization that combines line search and trust region steps. Math. Program. 107(3), 391–408 (2006). https://doi.org/10.1007/s10107-004-0560-5
Acknowledgements
We gratefully acknowledge supported by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant EP/N022351/1.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Rockett, P., Lopes, Y.K., Dou, T., Hathway, E.A. (2019). d(Tree)-by-dx: Automatic and Exact Differentiation of Genetic Programming Trees. In: Pérez García, H., Sánchez González, L., Castejón Limas, M., Quintián Pardo, H., Corchado Rodríguez, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2019. Lecture Notes in Computer Science(), vol 11734. Springer, Cham. https://doi.org/10.1007/978-3-030-29859-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-29859-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-29858-6
Online ISBN: 978-3-030-29859-3
eBook Packages: Computer ScienceComputer Science (R0)