ABSTRACT
This Hof-off-the-Press paper summarizes our recently published work, "SR-Forest: A Genetic Programming based Heterogeneous Ensemble Learning Method," published in IEEE Transactions on Evolutionary Computation [4]. This paper presents SR-Forest, a novel genetic programming-based heterogeneous ensemble learning method, which combines the strengths of decision trees and genetic programming-based symbolic regression methods. Rather than treating genetic programming-based symbolic regression methods as competitors to random forests, we propose to enhance the performance of random forests by incorporating genetic programming as a complementary technique. We introduce a guided mutation operator, a multi-fidelity evaluation strategy, and an ensemble selection mechanism to accelerate the search process, reduce computational costs, and improve predictive performance. Experimental results on a regression benchmark with 120 datasets show that SR-Forest outperforms 25 existing symbolic regression and ensemble learning methods. Moreover, we demonstrate the effectiveness of SR-Forest on an XGBoost hyperparameter performance prediction task, which is an important application area of ensemble learning methods. Overall, SR-Forest provides a promising approach to solving regression problems and can serve as a valuable tool in real-world applications.
- Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5--32.Google ScholarDigital Library
- Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 785--794.Google ScholarDigital Library
- William La Cava, Tilak Raj Singh, James Taggart, Srinivas Suri, and Jason H Moore. 2018. Learning concise representations for regression by evolving networks of trees. In International Conference on Learning Representations.Google Scholar
- Hengzhe Zhang, Aimin Zhou, Qi Chen, Bing Xue, and Mengjie Zhang. 2023. SR-Forest: A Genetic Programming based Heterogeneous Ensemble Learning Method. IEEE Transactions on Evolutionary Computation (2023).Google Scholar
- Hengzhe Zhang, Aimin Zhou, and Hu Zhang. 2022. An Evolutionary Forest for Regression. IEEE Transactions on Evolutionary Computation 26, 4 (2022), 735--749.Google ScholarCross Ref
Index Terms
- Genetic Programming-based Evolutionary Feature Construction for Heterogeneous Ensemble Learning [Hot of the Press]
Recommendations
Building boosted classification tree ensemble with genetic programming
GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference CompanionAdaptive boosting (AdaBoost) is a method for building classification ensemble, which combines multiple classifiers built in an iterative process of reweighting instances. This method proves to be a very effective classification method, therefore it was ...
Ensemble Genetic Programming
Genetic ProgrammingAbstractEnsemble learning is a powerful paradigm that has been used in the top state-of-the-art machine learning methods like Random Forests and XGBoost. Inspired by the success of such methods, we have developed a new Genetic Programming method called ...
An automated ensemble learning framework using genetic programming for image classification
GECCO '19: Proceedings of the Genetic and Evolutionary Computation ConferenceAn ensemble consists of multiple learners and can achieve a better generalisation performance than a single learner. Genetic programming (GP) has been applied to construct ensembles using different strategies such as bagging and boosting. However, no GP-...
Comments