Skip to main content
SpringerLink
Log in

Constructing Ensemble Classifiers from GEP-Induced Expression Trees

  • Chapter
Next Generation Data Technologies for Collective Computational Intelligence

Part of the book series: Studies in Computational Intelligence ((SCI,volume 352))

Abstract

The goal of the chapter is to construct high quality classifiers through applying collective computational techniques to the field of machine learning. Among the computational intelligence techniques one can distinguish a class referred to as the collective computational intelligence. The chapter proposes and reviews a family of ensemble classifiers constructed from expression trees. We propose to construct classifiers using collective computational intelligence paradigms at two levels. At the lower level the so-called weak classifiers are produced taking advantage of the benefits of cooperation between individuals evolved iteratively using gene expression programming and cellular evolutionary algorithms. At the upper level, cooperating individuals, which in our case are expression trees, are combined with a view to achieve better classification results through exploiting the collective intelligence property. Expression trees are induced using gene expression programming and cellular evolutionary algorithm. Ensemble classifiers are constructed from the weak classifiers obtained at the lower level of collaboration. To construct ensemble classifiers several standard techniques including majority voting, boosting and Dempster-Shafer theory of evidence, are used. To validate the approach a computational experiment has been carried-out using several well known datasets. The experiment aimed at comparison of the proposed classifiers performance with that of several widely used and popular classifiers with some of them also built through applying some collective computational intelligence tools. Experiment results confirm that next generation collective computational intelligence techniques like gene expression programming and cellular evolutionary algorithms, when applied to the field of machine learning, can offer an advantage that can be attributed to their collaborative and synergetic features.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alba, E., Dorronsoro, B.: Cellular Genetic Algorithms. Springer Science, New York (2008)

    MATH  Google Scholar 

  2. Asuncion, A., Newman, D.J.: UCI Machine Learning Repository. University of California, School of Information and Computer Science (2007), http://www.ics.uci.edu/~mlearn/MLRepository.html

  3. Battiti, R., Colla, A.M.: Democracy in neural nets: Voting schemes for classification. Neural Networks 7(4), 691–707 (1994)

    Article  Google Scholar 

  4. Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning 36(1-2), 105–139 (1999)

    Article  Google Scholar 

  5. Bennett, P.N.: Building Reliable Metaclassifiers for Text Learning, Ph.D. Thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh (2006)

    Google Scholar 

  6. Bi, Y., Guan, J., Bell, D.: The combination of multiple classifiers using an evidentional reasoning approach. Artif. Intell. 172, 1731–1751 (2008)

    Article  MATH  Google Scholar 

  7. Corder, G.W., Foreman, D.I.: Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach. J. Wiley, New Jersey (2009)

    MATH  Google Scholar 

  8. Dasarathy, B.V., Sheela, B.V.: Composite classifier system design: concepts and methodology. Proceedings of the IEEE 67(5), 708–713 (1979)

    Article  Google Scholar 

  9. Drucker, H., Cortes, C., Jackel, L.D., LeCun, Y., Vapnik, V.: Boosting and other ensemble methods. Neural Computation 6(6), 1289–1301 (1994)

    Article  MATH  Google Scholar 

  10. Duan, L., Tang, C., Zhang, T., Wei, D., Zhang, H.: Distance guided classification with gene expression programming. In: Li, X., Zaïane, O.R., Li, Z.-h. (eds.) ADMA 2006. LNCS (LNAI), vol. 4093, pp. 239–246. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  11. Duda, R., Hart, P., Stork, D.: Pattern Classification. J. Wiley, New York (2001)

    MATH  Google Scholar 

  12. Engelbrecht, A.P.: Computational Intelligence. An Introduction. J. Wiley, Chichester (2007)

    Google Scholar 

  13. Fawcett, T.: ROC Graphs: Notes and Practical Considerations for Researchers, HP Labs Tech Report HPL-2003-4, Palo Alto, Ca (2003)

    Google Scholar 

  14. Ferreira, C.: Gene expression programming: a new adaptive algorithm for solving problems. Complex Systems 13(2), 87–129 (2001)

    MATH  MathSciNet  Google Scholar 

  15. Ferreira, C.: Gene Expression Programming. SCI, vol. 21, pp. 337–380. Springer, Heidelberg (2006)

    Book  MATH  Google Scholar 

  16. Freund, Y., Schapire, R.E.: Decision-theoretic generalization of on-line learning and application to boosting. Journal of Computer and System Science 55(1), 119–139 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  17. Fulcher, J.: Computational Intelligence. An Introduction. SCI, vol. 115, pp. 3–78. Springer, Heidelberg (2006)

    Google Scholar 

  18. Gama, J.: Local cascade generalization. In: Proceedings of the 15th International Conference on Machine Learning, pp. 206–214 (1998)

    Google Scholar 

  19. Hartigan, J.A., Wong, M.A.: A k-means clustering algorithm. Applied Statistics 28(1), 100–108 (1979)

    Article  MATH  Google Scholar 

  20. Ho, T.K., Hull, J.J., Srihari, S.N.: Decision combination in multiple classifier systems. IEEE Transactions on Pattern Recognition and Machine Intelligence 16(1), 66–75 (1994)

    Article  Google Scholar 

  21. Huang, C.-L., Chen, M.-C., Wang, C.-J.: Credit scoring with a data mining approach based on support vector machines. Expert Systems with Applications 33, 847–856 (2007)

    Article  Google Scholar 

  22. Jedrzejowicz, J., Jedrzejowicz, P.: GEP-induced expression trees as weak classifiers. In: Perner, P. (ed.) ICDM 2008. LNCS (LNAI), vol. 5077, pp. 129–141. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  23. Jedrzejowicz, J., Jedrzejowicz, P.: A Family of GEP-induced ensemble classifiers. In: Nguyen, N.T., Kowalczyk, R., Chen, S.-M. (eds.) ICCCI 2009. LNCS, vol. 5796, pp. 641–652. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  24. Jedrzejowicz, J., Jedrzejowicz, P.: Two ensemble classifiers constructed from GEP-induced expression trees. In: Jędrzejowicz, P., Nguyen, N.T., Howlet, R.J., Jain, L.C. (eds.) KES-AMSTA 2010. LNCS, vol. 6071, pp. 200–209. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  25. Karakasis, V.K., Stafylopatis, A.: Data mining based on gene expression programming and clonal selection. In: Proc. IEEE Congress on Evolutionary Computation, pp. 514–521 (2006)

    Google Scholar 

  26. Kuncheva, L.I.: Classifier ensembles for changing environments. In: Roli, F., Kittler, J., Windeatt, T. (eds.) MCS 2004. LNCS, vol. 3077, pp. 1–15. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  27. Lam, L., Suen, C.Y.: Optimal combination of pattern classifiers. Pattern Recognition Letters 16(9), 945–954 (1995)

    Article  Google Scholar 

  28. Last, M., Maimon, O.: A compact and accurate model for classification. IEEE Transactions on Knowledge and Data Engineering 16(2), 203–215 (2004)

    Article  Google Scholar 

  29. Li, X., Zhou, C., Xiao, W., Nelson, P.C.: Prefix gene expression programming. In: Proc. Genetic and Evolutionary Computation Conference, Washington, pp. 25–31 (2005)

    Google Scholar 

  30. Pena Centeno, T., Lawrence, N.D.: Optimising kernel parameters and regularisation coefficients for non-linear discriminant analysis. Journal of Machine Learning Research 7, 455–491 (2006)

    MathSciNet  Google Scholar 

  31. Polikar, R.: Ensemble based systems in decision making. IEEE Circuits and Systems Magazine 3, 22–43 (2006)

    Google Scholar 

  32. Van Rijsbergen, C.V.: Information Retrieval, 2nd edn. Butterworth, London (1979)

    Google Scholar 

  33. Schapire, R.E., Freund, Y., Bartlett, P., Lee, W.S.: Boosting the margin: a new explanation for the effectiveness of voting methods. The Annals of Statistics 26(5), 1651–1686 (1998)

    Article  MATH  MathSciNet  Google Scholar 

  34. Shafer, G.: A Mathematical Theory of Evidence. Princeton University Press, Princeton (1976)

    MATH  Google Scholar 

  35. Srinivasa, K.B., Singh, A., Thomas, A.O., Venugopal, K.R., Patnoik, L.M.: Generic feature extraction for classification using fuzzy c-means clustering. In: Proc. Intelligent Sensing and Information Processing Conference, pp. 33–38 (2005)

    Google Scholar 

  36. Torre, F.: Boosting Correct Least General Generalizations, Technical Report GRAppA-0104, Grenoble (2004)

    Google Scholar 

  37. Tsoumakas, G., Angelis, L., Vlahavas, I.: Selective fusion of heterogeneous classifiers. Intelligent Data Analysis 9(6), 511–525 (2005)

    Google Scholar 

  38. Wang, W., Li, Q., Han, S., Lin, H.: A preliminary study on constructing decision tree with gene expression programming, In: Proc. First International Conference on Innovative Computing, Information and Control, vol. 1, pp. 222–225 (2006)

    Google Scholar 

  39. Weinert, W.R., Lopes, H.S.: GEPCLASS: a classification rule discovery tool using gene expression programming, In: Li, X., Zaïane, O.R., Li, Z.-h. (eds.) ADMA 2006. LNCS (LNAI), vol. 4093, pp. 871–880. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  40. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Francisco (2005)

    MATH  Google Scholar 

  41. Wolpert, D.H.: Stacked generalizatio. Neural Networks 5, 241–259 (1992)

    Article  Google Scholar 

  42. Statlog Datasets: comparison of results, http://www.is.umk.pl/projects/datasets.html#Cleveland (accessed on December 27, 2007)

  43. Zeng, T., Xiang, Y., Chen, P., Liu, Y.: A model of immune gene expression programming for rule mining. Journal of Universal Computer Science 13(7), 1239–1252 (2007)

    Google Scholar 

  44. Zhou, C., Xiao, W., Tirpak, T.M., Nelson, P.C.: Evolving accurate and compact classification rules with gene expression programming. IEEE Transactions on Evolutionary Computation 7(6), 519–531 (2003)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Informatics, Gdańsk University, Wita Stwosza 57, 80-952, Gdańsk, Poland

    Joanna Jȩdrzejowicz

  2. Department of Information Systems, Gdynia Maritime University, Morska 83, 81-225, Gdynia, Poland

    Piotr Jȩdrzejowicz

Authors
  1. Joanna Jȩdrzejowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Piotr Jȩdrzejowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computing & Maths, University of Derby, DE22 1GB, Derby, United Kingdom (UK)

    Nik Bessis

  2. Dept de Llenguatges i Sistemes Informàtics, Universitat Politècnica de Catalunya, Barcelona, Spain

    Fatos Xhafa

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Jȩdrzejowicz, J., Jȩdrzejowicz, P. (2011). Constructing Ensemble Classifiers from GEP-Induced Expression Trees. In: Bessis, N., Xhafa, F. (eds) Next Generation Data Technologies for Collective Computational Intelligence. Studies in Computational Intelligence, vol 352. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20344-2_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-20344-2_7

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-20343-5

  • Online ISBN: 978-3-642-20344-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation