Symbolic Regression with Genetic Programming: Synthetic Data Generation and Model Selection for Interpolation and Extrapolation

Created by W.Langdon from gp-bibliography.bib Revision:1.9194

@PhdThesis{Wulandari:thesis,

• author = "Fitria Wulandari",
• title = "Symbolic Regression with Genetic Programming: Synthetic Data Generation and Model Selection for Interpolation and Extrapolation",
• school = "School of Computer Science, University of Galway",
• year = "2026",
• address = "Ireland",
• month = "8 " # may,
• keywords = "genetic algorithms, genetic programming",
• URL = "https://hdl.handle.net/10379/19621",
• URL = "https://researchrepository.universityofgalway.ie/entities/publication/1201a290-5183-4423-aa30-d51be0d48d11",
• URL = "https://researchrepository.universityofgalway.ie/server/api/core/bitstreams/2b58cc87-0925-4555-8107-1740e87a3f76/content",
• DOI = "10.13025/30409",
• size = "222 pages",
• abstract = "Symbolic regression (SR) uses genetic programming (GP) to find mathematical expressions that fit data directly, without assuming a fixed model structure in advance. Unlike neural networks (NN) and random forests (RF), SR produces explicit and readable equations that domain experts can be inspect, interpret, understand, and trust. However, three practical problems limit its usefulness in real applications. First, SR can struggle when training data is scarce or unevenly distributed. Second, GP-evolved expressions often fail to generalise beyond the training distribution, producing unreliable predictions in sparse regions of the input space. Third, GP returns a set of candidate equations rather than a single model, and choosing the best one is not straightforward.

  This thesis addresses both problems. For the first, we test whether synthetic data can help GP find better equations when real data is limited. A teacher model, such as NN or RF, generates synthetic labels by querying a trained model on new inputs, and GP learns from this augmented dataset. Experiments across benchmark datasets show that this helps in some conditions but not all. Using a NN or RF as the teacher, with GP as the student, works more reliably than using GP as the teacher. For the second, we extend this framework to extrapolation by targeting sparse regions of the input space identified using kernel density estimation (KDE). KDE is better than geometric bounding boxes for identifying where synthetic data should be placed, and targeting those regions improves extrapolation performance more consistently than interpolation

  For the third problem, we first compare five standard model selection criteria (MSE, AIC, BIC, MDL, and PSM) across 20 benchmark datasets and find that none of them works reliably across all cases. We then develop hybrid criteria that extend any standard metric with three penalty components: (i) extrapolation divergence, which measures how much predictions change outside the training region; (ii) interpolation sensitivity, which measures how much predictions vary under small input perturbations within the training region; and (iii) extrapolation sensitivity, which measures the same prediction variance but in sparse regions outside the training data. Experiments across 20 datasets and 30 runs per dataset show that these hybrid penalties consistently improve model selection for extrapolation tasks. Statistical testing shows that the extrapolation divergence penalty shifts most significantly when the evaluation target is extrapolation rather than interpolation, making it the most reliably activated component. For interpolation tasks, the interpolation sensitivity weight tends to dominate in the best-performing configurations. Two configurations are carried forward as practical recommendations: one optimised for stability across runs, and one that wins the most individual run-by-run comparisons when selecting models for extrapolation.

  Both configurations are then applied directly to a real energy system problem, predicting a global energy system cost from a large integrated assessment model under a delayed climate policy scenario, without any further tuning. Both configurations improve median test error by approximately 98 percent over the standard baseline, which selected a single-feature linear model of climate sensitivity as the best model on nearly every run. The selected equations are readable and use physically meaningful input variables such as climate sensitivity, discount rate, demand elasticity, and biomass potential. Overall, the thesis contributes the first systematic comparison of model selection criteria for SR, a hybrid selection framework with tested practical recommendations, and a real-world validation on an energy system surrogate modelling problem.",

• notes = "Research in interpretable machine learning, symbolic regression, genetic programming, and statistical model selection

  supervisors: James McDermott and Colm O'Riordan",

}

Genetic Programming entries for Fitria Wulandari Ramlan

Citations