Abstract
Local search methods can harmoniously work with global search methods such as Evolutionary Algorithms (EAs); however, particularly in Genetic Programming (GP), concerns remain about the additional cost of local search (LS). One successful such system is Chameleon, which tunes internal GP nodes and addresses cost concerns by employing a number of strategies to make its LS both effective and inexpensive. Expense is reduced by an innovative approach to parsimony pressure whereby smaller trees are rewarded with local search opportunities more often than bigger trees.
This paper investigates three new extensions to Chameleon’s original simple setup, seeking ways for an even more effective local search. These are: trying alternative, more cost-reflective parsimony measures such as visitation length instead of tree size; varying the reward function to more gently incentivize parsimony than that in the original setup; and having more tuning earlier in runs when smaller trees can be tuned more cheaply and effectively. These strategies were tested on a varied suite of 16 difficult artificial and real-world regression problems. All of these techniques improved performance.
We show that these strategies successfully combined to cumulatively improve average test RMSE by 31% over the original and already effective Chameleon tuning scheme. A minimum of 64 simulations were run on every problem/tuning setup combination.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Applegate, D., Cook, W., Rohe, A.: Chained Lin-Kernighan for large traveling salesman problems. INFORMS Journal on Computing 15(1), 82–92 (2003)
Azad, R.M.A., Ryan, C.: Abstract functions and lifetime learning in genetic programming for symbolic regression. In: Genetic and Evolutionary Computation Conference (GECCO-2010), pp. 893–900. ACM (2010)
Atif, R.M., Azad, C.R.: A simple approach to lifetime learning in genetic programming based symbolic regression. Evolutionary Computation 22(2), 287–317 (2014)
Bhardwaj, A., Tiwari, A.: A Novel Genetic Programming Based Classifier Design Using a New Constructive Crossover Operator with a Local Search Technique. In: Huang, D.-S., Bevilacqua, V., Figueroa, J.C., Premaratne, P. (eds.) ICIC 2013. LNCS, vol. 7995, pp. 86–95. Springer, Heidelberg (2013)
Clark-Carter, D.: Geometric mean. In: Encyclopedia of Statistics in Behavioral Science. Wiley (2005)
Frank, A., Asuncion, A., et al.: UCI machine learning repository (2010)
Fukunaga, A.S.: Evolving local search heuristics for SAT using genetic programming. In: Deb, K., Tari, Z. (eds.) GECCO 2004. LNCS, vol. 3103, pp. 483–494. Springer, Heidelberg (2004)
Keijzer, M., Foster, J.: Crossover bias in genetic programming. In: Ebner, M., O’Neill, M., Ekárt, A., Vanneschi, L., Esparcia-Alcázar, A.I. (eds.) EuroGP 2007. LNCS, vol. 4445, pp. 33–44. Springer, Heidelberg (2007)
Kommenda, M., Kronberger, G., et al.: Effects of constant optimization by nonlinear least squares minimization in symbolic regression. In: Genetic and Evolutionary Computation Conference (GECCO 2013), pp. 1121–1128. ACM (2013)
Krawiec, K.: Genetic programming with local improvement for visual learning from examples. In: Skarbek, W. (ed.) CAIP 2001. LNCS, vol. 2124, pp. 209–216. Springer, Heidelberg (2001)
Majeed, H., Ryan, C.: On the constructiveness of context-aware crossover. In: Genetic and Evolutionary Computation Conference (GECCO-2007), pp. 1659–1666. ACM (2007)
McDermott, J., White, D.R., et al.: Genetic programming needs better benchmarks. In: Genetic and Evolutionary Computation Conference (GECCO 2012), pp. 791–798. ACM (2012)
McPhee, N.F., Hopper, N.J.: Analysis of genetic diversity through population history. In: Genetic and Evolutionary Computation Conference (GECCO 1999), vol. 2, pp. 1112–1120 (1999)
Moscato, P., Cotta, C.: A gentle introduction to memetic algorithms. In: Handbook of metaheuristics, pp. 105–144. Springer (2003)
O’Reilly, U.-M., Oppacher, F.: Program search with a hierarchical variable length representation: Genetic programming, simulated annealing and hill climbing. In: Davidor, Y., Männer, R., Schwefel, H.-P. (eds.) PPSN 1994. LNCS, vol. 866, pp. 397–406. Springer, Heidelberg (1994)
O’Reilly, U.-M., Oppacher, F.: A comparative analysis of genetic programming. In: Advances in Genetic Programming 2, ch. 2, pp. 23–44. MIT Press (1996)
Radcliffe, N.J., Surry, P.D.: Formal memetic algorithms. In: Fogarty, T.C. (ed.) AISB-WS 1994. LNCS, vol. 865, pp. 1–16. Springer, Heidelberg (1994)
Scoble, A., Johnston, M., Zhang, M.: Local search in parallel linear genetic programming for multiclass classification. In: Thielscher, M., Zhang, D. (eds.) AI 2012. LNCS, vol. 7691, pp. 373–384. Springer, Heidelberg (2012)
Smits, G.F., Kotanchek, M.: Pareto-front exploitation in symbolic regression. In: Genetic Programming Theory and Practice II, pp. 283–299. Springer (2005)
Williams, C.B.: The use of logarithms in the interpretation of certain entomological problems. Annals of Applied Biology 24(2), 404–414 (1937)
Zhang, M., Gao, X., Lou, W.: A new crossover operator in genetic programming for object classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 37(5), 1332–1343 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Lane, F., Azad, R.M.A., Ryan, C. (2014). On Effective and Inexpensive Local Search Techniques in Genetic Programming Regression. In: Bartz-Beielstein, T., Branke, J., Filipič, B., Smith, J. (eds) Parallel Problem Solving from Nature – PPSN XIII. PPSN 2014. Lecture Notes in Computer Science, vol 8672. Springer, Cham. https://doi.org/10.1007/978-3-319-10762-2_44
Download citation
DOI: https://doi.org/10.1007/978-3-319-10762-2_44
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10761-5
Online ISBN: 978-3-319-10762-2
eBook Packages: Computer ScienceComputer Science (R0)