Abstract
Classification is one of the most important machine learning tasks in science and engineering. However, it can be a difficult task, in particular when a high number of classes is involved. Genetic Programming, despite its recognized successfulness in so many different domains, is one of the machine learning methods that typically struggles, and often fails, to provide accurate solutions for multiclass classification problems. We present a novel algorithm for tree based GP that incorporates some ideas on the representation of the solution space in higher dimensions, and can be generalized to other types of GP. We test three variants of this new approach on a large set of benchmark problems from several different sources, and observe their competitiveness against the most successful state-of-the-art classifiers like Random Forests, Random Subspaces and Multilayer Perceptron.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alcala-Fdez J, Fernandez A, Luengo J, Derrac J, Garcia S, Sanchez L, Herrera F (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J Mult Valued Logic Soft Comput 17:2–3, 255–287
Bache K, Lichman M (2013) UCI machine learning repository, university of California, Irvine, school of information and computer sciences. http://archiveicsuciedu/ml
Bojarczuk CC, Lopes HS, Freitas AA (2000) Genetic programming for knowledge discovery in chest-pain diagnosis. IEEE Eng Med Biol Mag 19(4):38–44. http://ieeexplore.ieee.org/iel5/51/18543/00853480.pdf
Castelli M, Silva S, Vanneschi L, Cabral A, Vasconcelos MJ, Catarino L, Carreiras JMB (2013) Land cover/land use multiclass classification using gp with geometric semantic operators. In: EvoApplications’13. Springer, Berlin, pp 334–343
Espejo PG, Ventura S, Herrera F (2010) A survey on the application of genetic programming to classification. Trans Sys Man Cyber Part C 40(2):121–144
Falco ID, Cioppa AD, Tarantino E (2002) Discovering interesting classification rules with genetic programming. Appl Soft Comput 1(4):257–269
Haynes T (1998) Collective adaptation: the exchange of coding segments. Evol Comput 6(4): 311–338. doi:10.1162/evco.1998.6.4.311. http://dx.doi.org/10.1162/evco.1998.6.4.311
Hsu CW, Lin CJ (2002) A comparison of methods for multi-class support vector machines. IEEE Trans Neural Netw 13(2):415–425
Ingalalli V, Silva S, Castelli M, Vanneschi L (2014) A multi-dimensional genetic programming approach for multi-class classification problems. In: Nicolau M, et al. (eds) 17th European conference on genetic programming. Lecture notes in computer science, vol 8599. Springer, Granada, pp 48–60
Jabeen H, Baig AR (2013) Two-stage learning for multi-class classification using genetic programming. Neurocomputing 116:311–316
Kishore JK, Patnaik L, Mani V, Agrawal VK (2000) Application of genetic programming for multicategory pattern classification. IEEE Trans Evol Comput 4(3):242–258
Koza JR (1992) Genetic programming: volume 1, On the programming of computers by means of natural selection, vol 1. MIT Press, New York
Koza JR (2010) Human-competitive results produced by genetic programming. Genet Program Evolvable Mach 11(3–4):251–284
Langdon W, Poli R (2002) Foundations of genetic programming. Springer, Berlin
Li XM, Wang M, Cui LJ, Huang DM (2007) A new classification arithmetic for multi-image classification in genetic programming. In: International conference on machine learning and cybernetics, vol 3, pp 1683–1687, 2007
Lin JY, Ke HR, Chien BC, Yang WP (2007) Designing a classifier by a layered multi-population genetic programming approach. Pattern Recogn 40(8):2211–2225
Lin JY, Ke HR, Chien BC, Yang WP (2008) Classifier design with feature selection and feature extraction using layered genetic programming. Expert Syst Appl 34(2):1384–1393
Muñoz L, Silva S, Trujillo L (2015) M3gp—multiclass classification with gp. In: Machado P, Heywood MI, McDermott J, Castelli M, García-Sánchez P, Burelli P, Risi S, Sim K (eds) Genetic programming. Lecture notes in computer science, vol 9025. Springer International Publishing, Berlin, pp 78–91
Muni D, Pal N, Das J (2004) A novel approach to design classifiers using genetic programming. IEEE Trans Evol Comput 8(2):183–196
Poli R, Langdon WB, McPhee NF (2008) A field guide to genetic programming. Published via http://lulu.com and freely available at http://www.gp-field-guide.org.uk. http://www.gp-field-guide.org.uk. (With contributions by J. R. Koza)
Sakprasat S, Sinclair M (2007) Classification rule mining for automatic credit approval using genetic programming. In: IEEE congress on evolutionary computation, 2007. CEC 2007, pp 548–555
Shen S, Sandham W, Granat M, Dempsey MF, Patterson J (2003) A new approach to brain tumour diagnosis using fuzzy logic based genetic programming. In: Engineering in medicine and biology society, 2003. Proceedings of the 25th annual international conference of the IEEE (volume 1), vol 1, pp 870–873
Shiming Xiang FN, Zhang C (2008) Learning a mahalanobis distance metric for data clustering and classification. Pattern Recogn 41(2):3600–3612
Silva S (2011) Reassembling operator equalisation: A secret revealed. In: Proceedings of the 13th annual conference on genetic and evolutionary computation, GECCO ’11. ACM, New York, pp 1395–1402
Silva S, Tseng YT (2008) Classification of seafloor habitats using genetic programming. In: Applications of evolutionary computing. Lecture notes in computer science, vol 4974. Springer, Berlin, pp 315–324
Tackett WA (1993) Genetic programming for feature discovery and image discrimination. In: Proceedings of the 5th international conference on genetic algorithms. Morgan Kaufmann Publishers Inc, San Francisco, CA, pp 303–311
Tan KC, Tay A, Lee T, Heng CM (2002) Mining multiple comprehensible classification rules using genetic programming. In: Proceedings of the 2002 congress on evolutionary computation. 2002. CEC ’02, vol 2, pp 1302–1307
Teredesai A, Govindaraju V (2004) Issues in evolving gp based classifiers for a pattern recognition task. In: Congress on evolutionary computation, 2004. CEC2004, vol 1, pp 509–515
Winkler S, Affenzeller M, Wagner S (2007) Advanced genetic programming based machine learning. J Math Model Algorithm 6(3):455–480
Zhang M, Ciesielski V (1999) Genetic programming for multiple class object detection. In: Advanced topics in artificial intelligence. Lecture notes in computer science, vol 1747. Springer, Berlin, pp 180–192
Zhang M, Smart W (2004) Multiclass object classification using genetic programming. In: Applications of evolutionary computing. Lecture notes in computer science, vol 3005. Springer, Berlin, pp 369–378
Zhang Y, Rockett PI (2009) A generic multi-dimensional feature extraction method using multiobjective genetic programming. Evol Comput 17(1):89–115
Acknowledgements
This work was partially supported by FCT funds (Portugal) under contract UID/Multi/04046/2013 and projects PTDC/EEI-CTP/2975/2012 (MaSSGP), PTDC/DTP-FTO/1747/2012 (InteleGen) and EXPL/EMS-SIS/1954/2013 (CancerSys). Funding was also provided by CONACYT (Mexico) Basic Science Research Project No. 178323, DGEST (Mexico) Research Projects No. 5149.13-P and 5414.11-P, and FP7-Marie Curie-IRSES 2013 project ACoBSEC. Finally, the second author is supported by scholarship No. 372126 from CONACYT.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Silva, S., Muñoz, L., Trujillo, L., Ingalalli, V., Castelli, M., Vanneschi, L. (2016). Multiclass Classification Through Multidimensional Clustering. In: Riolo, R., Worzel, W., Kotanchek, M., Kordon, A. (eds) Genetic Programming Theory and Practice XIII. Genetic and Evolutionary Computation. Springer, Cham. https://doi.org/10.1007/978-3-319-34223-8_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-34223-8_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-34221-4
Online ISBN: 978-3-319-34223-8
eBook Packages: Computer ScienceComputer Science (R0)