Abstract
AdaBoost is one of the most commonly used and most successful approaches for generating ensemble classifiers. However, AdaBoost is limited in that it requires independent training cases and can only use voting as a cooperation mechanism. This paper compares AdaBoost to Orthogonal Evolution of Teams (OET), an approach for generating ensembles that allows for a much wider range of problems and cooperation mechanisms. The set of test problems includes problems with significant amounts of noise in the form of erroneous training cases and problems with adjustable levels of epistasis. The results demonstrate that OET is a suitable alternative to AdaBoost for generating ensembles. Over the set of all tested problems OET with a hierarchical cooperation mechanism, rather than voting, is slightly more likely to produce better results. This is most apparent on the problems with very high levels of noise - suggesting that the hierarchical approach is less subject to over-fitting than voting techniques. The results also suggest that there are specific problems and features of problems that make thembetter suited for different training algorithms and different cooperation mechanisms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Asuncion, A. and Newman, D.J. (2007). UCI machine learning repository.
Brameier, Markus and Banzhaf, Wolfgang (2001). Evolving teams of predictors with linear genetic programming. Genetic Programming and Evolvable Machines, 2(4):381–408.
Freund, Y., Schapire, R., and Abe, N. (1999). A short introduction to boosting. JOURNAL-JAPANESE SOCIETY FOR ARTIFICIAL INTELLIGENCE, 14:771–780.
Haynes, Thomas, Sen, Sandip, Schoenefeld, Dale, and Wainwright, Roger (1995). Evolving a team. In Siegel, Eric V. and Koza, John, editors, Working Notes of the AAAI-95 Fall Symposium on GP, pages 23–30. AAAI Press.
Heckendorn, Robert B. (2002). Embedded landscapes. Evolutionary Computation, 10(4):345–376.
Imamura, Kosuke (2002). N-version Genetic Programming: A probabilistic Optimal Ensemble. PhD thesis, University of Idaho.
Imamura, Kosuke, Heckendorn, Robert B., Soule, Terence, and Foster, James A. (2004). Behavioral diversity and a probabilistically optimal gp ensemble. Genetic Programming and Evolvable Machines, 4:235–253.
Just, M.A. and Carpenter, P.A. (1993). The intensity dimension of thought: Pupillometric indices of sentence processing. Canadian Journal of Experimental Psychology, 47(2):310–339.
Kishore, JK, Patnaik, LM, Mani, V., and Agrawal, VK (2000). Application of genetic programming for multicategory pattern classification. IEEE Transactions on Evolutionary Computation, 4(3):242–258.
Lew, R., P., Dyre B., Soule, T., Werner, S., and Ragsdale, S.A. (2010).Assessing mental workload from skin conductance and pupillometry using wavelets and genetic programming. In Proceedings of the 54th Annual Meeting of the Human Factors and Ergonomics Society.
Luke, Sean and Spector, Lee (1996). Evolving teamwork and coordination with genetic programming. InKoza, John R., Goldberg, David E., Fogel, David B., and Riolo, Rick R., editors, Genetic Programming 1996: Proceedings of the First Annual Conference on Genetic Programming, pages 150–156. Cambridge, MA: MIT Press.
Mease, D. and Wyner, A. (2008). Evidence contrary to the statistical view of boosting. The Journal of Machine Learning Research, 9:131–156.
Muni, DP, Pal, NR, and Das, J. (2004). A novel approach to design classifiers using genetic programming. IEEE transactions on evolutionary computation, 8(2):183–196.
Nakayama,M. and Katsukura,M. (2007). Feasibility of assessing usability with pupillary responses. Proc. of AUIC 2007, 15, 22.
Paul, T.K. and Iba, H. (2009). Prediction of cancer classwithmajority voting genetic programming classifier using gene expression data. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB), 6(2):353–367.
Platel, Michael Defoin, Chami, Malik, Clergue, Manuel, and Collard, Philippe (2005). Teams of genetic predictors for inverse problem solving. In Proceeding of the 8th European Conference on Genetic Programming – EuroGP 2005.
Polikar, R. (2006). Ensemble based systems in decision making. IEEE Circuits and systems magazine, 6(3):21–45.
Schapire, R.E., Freund, Y., Bartlett, P., and Lee, W.S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. Annals of statistics, 26(5):1651–1686.
Sigillito, V G, Wing, S P, Hutton, L V, and Baker, K B (1989). Classification of radar returns from the ionosphere using neural networks. Johns Hopkins APL Tech. Dig, vol. 10:262–266. in.
Soule, T. and Heckendorn, R.B. (2007a). Improving Performance and Cooperation in Multi-Agent Systems. In Proceedings of the Genetic Programming Theory and Practice Workshop. Springer.
Soule, Terence (1999). Voting teams: A cooperative approach to non-typical problems. In Banzhaf, Wolfgang, Daida, Jason, Eiben, Agoston E., Garzon, Max H., Honavar, Vasant, Jakiela, Mark, and Smith, Robert E., editors, Proceedings of the Genetic and Evolutionary Computation Conference, pages 916–922, Orlando, Florida, USA. Morgan Kaufmann.
Soule, Terence and Heckendorn, Robert B. (2007b). Evolutionary optimization of cooperative heterogeneous teams. In SPIE Defense and Security Symposium, volume 6563.
Soule,Terence and Komireddy, Pavankumarreddy (2006).Orthogonal evolution of teams: A class of algorithms for evolving teams with inversely correlated errors. In Riolo, Rick L., Soule, Terence, and Worzel, Bill, editors, Genetic Programming Theory andPractice IV, volume 5 ofGenetic andEvolutionary Computation, chapter 8, pages –. Springer, Ann Arbor.
Thomason, Russell, Heckendorn, Robert B., and Soule, Terence (2008). Training time and team composition robustness in evolved multi-agent systems. In O’Neill, Michael, Vanneschi, Leonardo, Gustafson, Steven, Esparcia Alcazar, Anna Isabel, De Falco, Ivanoe, Della Cioppa, Antonio, and Tarantino, Ernesto, editors, Proceedings of the 11th European Conference on Genetic Programming, EuroGP 2008, volume 4971 of Lecture Notes in Computer Science, pages 1–12, Naples. Springer.
Tsanas, A., Little, M.A., McSharry, P.E., and Ramig, L.O. (2009). Accurate telemonitoring of Parkinson’s disease progression by non-invasive speech tests. Scientific Commons.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Soule, T., Heckendorn, R.B., Dyre, B., Lew, R. (2011). Ensemble Classifiers: AdaBoost and Orthogonal Evolution of Teams. In: Riolo, R., McConaghy, T., Vladislavleva, E. (eds) Genetic Programming Theory and Practice VIII. Genetic and Evolutionary Computation, vol 8. Springer, New York, NY. https://doi.org/10.1007/978-1-4419-7747-2_4
Download citation
DOI: https://doi.org/10.1007/978-1-4419-7747-2_4
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-7746-5
Online ISBN: 978-1-4419-7747-2
eBook Packages: Computer ScienceComputer Science (R0)