Abstract
The goal of the chapter is to construct high quality classifiers through applying collective computational techniques to the field of machine learning. Among the computational intelligence techniques one can distinguish a class referred to as the collective computational intelligence. The chapter proposes and reviews a family of ensemble classifiers constructed from expression trees. We propose to construct classifiers using collective computational intelligence paradigms at two levels. At the lower level the so-called weak classifiers are produced taking advantage of the benefits of cooperation between individuals evolved iteratively using gene expression programming and cellular evolutionary algorithms. At the upper level, cooperating individuals, which in our case are expression trees, are combined with a view to achieve better classification results through exploiting the collective intelligence property. Expression trees are induced using gene expression programming and cellular evolutionary algorithm. Ensemble classifiers are constructed from the weak classifiers obtained at the lower level of collaboration. To construct ensemble classifiers several standard techniques including majority voting, boosting and Dempster-Shafer theory of evidence, are used. To validate the approach a computational experiment has been carried-out using several well known datasets. The experiment aimed at comparison of the proposed classifiers performance with that of several widely used and popular classifiers with some of them also built through applying some collective computational intelligence tools. Experiment results confirm that next generation collective computational intelligence techniques like gene expression programming and cellular evolutionary algorithms, when applied to the field of machine learning, can offer an advantage that can be attributed to their collaborative and synergetic features.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alba, E., Dorronsoro, B.: Cellular Genetic Algorithms. Springer Science, New York (2008)
Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, School of Information and Computer Science (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html
Battiti, R., Colla, A.M.: Democracy in neural nets: Voting schemes for classification. Neural Networks 7(4), 691–707 (1994)
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36(1-2), 105–139 (1999)
Bennett, P.N.: Building Reliable Metaclassifiers for Text Learning, Ph.D. Thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh (2006)
Bi, Y., Guan, J., Bell, D.: The combination of multiple classifiers using an evidentional reasoning approach. Artif. Intell. 172, 1731–1751 (2008)
Corder, G.W., Foreman, D.I.: Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach. J. Wiley, New Jersey (2009)
Dasarathy, B.V., Sheela, B.V.: Composite classifier system design: concepts and methodology. Proceedings of the IEEE 67(5), 708–713 (1979)
Drucker, H., Cortes, C., Jackel, L.D., LeCun, Y., Vapnik, V.: Boosting and other ensemble methods. Neural Computation 6(6), 1289–1301 (1994)
Duan, L., Tang, C., Zhang, T., Wei, D., Zhang, H.: Distance guided classification with gene expression programming. In: Li, X., Zaïane, O.R., Li, Z.-h. (eds.) ADMA 2006. LNCS (LNAI), vol. 4093, pp. 239–246. Springer, Heidelberg (2006)
Duda, R., Hart, P., Stork, D.: Pattern Classification. J. Wiley, New York (2001)
Engelbrecht, A.P.: Computational Intelligence. An Introduction. J. Wiley, Chichester (2007)
Fawcett, T.: ROC Graphs: Notes and Practical Considerations for Researchers, HP Labs Tech Report HPL-2003-4, Palo Alto, Ca (2003)
Ferreira, C.: Gene expression programming: a new adaptive algorithm for solving problems. Complex Systems 13(2), 87–129 (2001)
Ferreira, C.: Gene Expression Programming. SCI, vol. 21, pp. 337–380. Springer, Heidelberg (2006)
Freund, Y., Schapire, R.E.: Decision-theoretic generalization of on-line learning and application to boosting. Journal of Computer and System Science 55(1), 119–139 (1997)
Fulcher, J.: Computational Intelligence. An Introduction. SCI, vol. 115, pp. 3–78. Springer, Heidelberg (2006)
Gama, J.: Local cascade generalization. In: Proceedings of the 15th International Conference on Machine Learning, pp. 206–214 (1998)
Hartigan, J.A., Wong, M.A.: A k-means clustering algorithm. Applied Statistics 28(1), 100–108 (1979)
Ho, T.K., Hull, J.J., Srihari, S.N.: Decision combination in multiple classifier systems. IEEE Transactions on Pattern Recognition and Machine Intelligence 16(1), 66–75 (1994)
Huang, C.-L., Chen, M.-C., Wang, C.-J.: Credit scoring with a data mining approach based on support vector machines. Expert Systems with Applications 33, 847–856 (2007)
Jedrzejowicz, J., Jedrzejowicz, P.: GEP-induced expression trees as weak classifiers. In: Perner, P. (ed.) ICDM 2008. LNCS (LNAI), vol. 5077, pp. 129–141. Springer, Heidelberg (2008)
Jedrzejowicz, J., Jedrzejowicz, P.: A Family of GEP-induced ensemble classifiers. In: Nguyen, N.T., Kowalczyk, R., Chen, S.-M. (eds.) ICCCI 2009. LNCS, vol. 5796, pp. 641–652. Springer, Heidelberg (2009)
Jedrzejowicz, J., Jedrzejowicz, P.: Two ensemble classifiers constructed from GEP-induced expression trees. In: Jędrzejowicz, P., Nguyen, N.T., Howlet, R.J., Jain, L.C. (eds.) KES-AMSTA 2010. LNCS, vol. 6071, pp. 200–209. Springer, Heidelberg (2010)
Karakasis, V.K., Stafylopatis, A.: Data mining based on gene expression programming and clonal selection. In: Proc. IEEE Congress on Evolutionary Computation, pp. 514–521 (2006)
Kuncheva, L.I.: Classifier ensembles for changing environments. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 1–15. Springer, Heidelberg (2004)
Lam, L., Suen, C.Y.: Optimal combination of pattern classifiers. Pattern Recognition Letters 16(9), 945–954 (1995)
Last, M., Maimon, O.: A compact and accurate model for classification. IEEE Transactions on Knowledge and Data Engineering 16(2), 203–215 (2004)
Li, X., Zhou, C., Xiao, W., Nelson, P.C.: Prefix gene expression programming. In: Proc. Genetic and Evolutionary Computation Conference, Washington, pp. 25–31 (2005)
Pena Centeno, T., Lawrence, N.D.: Optimising kernel parameters and regularisation coefficients for non-linear discriminant analysis. Journal of Machine Learning Research 7, 455–491 (2006)
Polikar, R.: Ensemble based systems in decision making. IEEE Circuits and Systems Magazine 3, 22–43 (2006)
Van Rijsbergen, C.V.: Information Retrieval, 2nd edn. Butterworth, London (1979)
Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. The Annals of Statistics 26(5), 1651–1686 (1998)
Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)
Srinivasa, K.B., Singh, A., Thomas, A.O., Venugopal, K.R., Patnoik, L.M.: Generic feature extraction for classification using fuzzy c-means clustering. In: Proc. Intelligent Sensing and Information Processing Conference, pp. 33–38 (2005)
Torre, F.: Boosting Correct Least General Generalizations, Technical Report GRAppA-0104, Grenoble (2004)
Tsoumakas, G., Angelis, L., Vlahavas, I.: Selective fusion of heterogeneous classifiers. Intelligent Data Analysis 9(6), 511–525 (2005)
Wang, W., Li, Q., Han, S., Lin, H.: A preliminary study on constructing decision tree with gene expression programming, In: Proc. First International Conference on Innovative Computing, Information and Control, vol. 1, pp. 222–225 (2006)
Weinert, W.R., Lopes, H.S.: GEPCLASS: a classification rule discovery tool using gene expression programming, In: Li, X., Zaïane, O.R., Li, Z.-h. (eds.) ADMA 2006. LNCS (LNAI), vol. 4093, pp. 871–880. Springer, Heidelberg (2006)
Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)
Wolpert, D.H.: Stacked generalizatio. Neural Networks 5, 241–259 (1992)
Statlog Datasets: comparison of results, http://www.is.umk.pl/projects/datasets.html#Cleveland (accessed on December 27, 2007)
Zeng, T., Xiang, Y., Chen, P., Liu, Y.: A model of immune gene expression programming for rule mining. Journal of Universal Computer Science 13(7), 1239–1252 (2007)
Zhou, C., Xiao, W., Tirpak, T.M., Nelson, P.C.: Evolving accurate and compact classification rules with gene expression programming. IEEE Transactions on Evolutionary Computation 7(6), 519–531 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Jȩdrzejowicz, J., Jȩdrzejowicz, P. (2011). Constructing Ensemble Classifiers from GEP-Induced Expression Trees. In: Bessis, N., Xhafa, F. (eds) Next Generation Data Technologies for Collective Computational Intelligence. Studies in Computational Intelligence, vol 352. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20344-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-20344-2_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20343-5
Online ISBN: 978-3-642-20344-2
eBook Packages: EngineeringEngineering (R0)