Islam Elnabarawy
Daniel R. Tauritz
ABSTRACT
Fuzzy Adaptive Resonance Theory (ART) is a classic unsupervised learning algorithm. Its performance on a particular clustering problem is sensitive to the suitability of the category function for said problem. However, classic Fuzzy ART employs a fixed category function and thus is unable to benefit from the potential to adjust its category function. This paper presents an exploration into employing evolutionary computation for the automated design of category functions to obtain significantly enhanced Fuzzy ART performance through tailoring to specific problem classes. We employ a genetic programming powered hyper-heuristic approach where the category functions are constructed from a set of primitives constituting those of the original Fuzzy ART category function as well as additional hand-selected primitives. Results are presented for a set of experiments on benchmark classification tasks from the UCI Machine Learning Repository demonstrating that tailoring Fuzzy ART's category function can achieve statistically significant superior performance on the testing datasets in stratified 10-fold cross-validation procedures. We conclude with discussing the results and placing them in the context of being a first step towards automating the design of entirely new forms of ART.
- G. C. Anagnostopoulos and M. Georgiopoulos. 2001. Ellipsoid ART and ARTMAP for incremental clustering and classification. In Proceedings of the International Joint Conference on Neural Networks (IJCNN '01), Vol. 2. 1221--1226.Google Scholar
- Edmund K Burke, Michel Gendreau, Matthew Hyde, Gabriela Ochoa, Graham Kendall, Ender Özcan, and Rong Qu. 2013. Hyper-heuristics : A Survey of the State of the Art. Journal of the Operational Research Society (2013), 1695--1724.Google Scholar
- GA Carpenter, S Grossberg, and JH Reynolds. 1991. ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network. Neural networks (1991). Google ScholarDigital Library
- GA Carpenter, S Grossberg, and DB Rosen. 1991. Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system. Neural networks 4, 6 (1991), 759--771. Google ScholarDigital Library
- Gail A. Carpenter and Marin N. Gjaja. 1994. Fuzzy ART Choice Functions. Proceedings of the World Congress on Neural Networks (WCNN'94) 1993, 12 (1994).Google Scholar
- Gail A. Carpenter and Stephen Grossberg. 1987. A massively parallel architecture for a self-organizing neural pattern recognition machine. Computer Vision, Graphics, and Image Processing 37, 1 (jan 1987), 54--115. Google ScholarDigital Library
- Gail A. Carpenter, Stephen Grossberg, Natalya Markuzon, John H. Reynolds, and David B. Rosen. 1992. Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Transactions on Neural Networks 3, 5 (sep 1992), 698--713. Google ScholarDigital Library
- I Elnabarawy, D C Wunsch, and A M Abdelbar. 2016. Biclustering ARTMAP collaborative filtering recommender system. In 2016 International Joint Conference on Neural Networks (IJCNN). 2986--2991.Google ScholarCross Ref
- Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner, Marc Parizeau, and Christian Gagné. 2012. DEAP: Evolutionary Algorithms Made Easy. Journal of Machine Learning Research 13 (jul 2012), 2171--2175. Google ScholarDigital Library
- Stephen Grossberg. 1980. How does a brain build a cognitive code? Psychological Review 87, 1 (jan 1980), 1--51.Google ScholarCross Ref
- Mark Hall, Eibe Frank, Geoffrey Holmes, Bernhard Pfahringer, Peter Reutemann, and Ian H Witten. 2009. The WEKA data mining software: an update. ACM SIGKDD explorations newsletter 11, 1 (2009), 10--18. Google ScholarDigital Library
- Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of classification 2, 1 (1985), 193--218.Google ScholarCross Ref
- A. Kaylani, M. Georgiopoulos, M. Mollaghasemi, and G.C. Anagnostopoulos. 2009. AG-ART: An adaptive approach to evolving ART architectures. Neurocomputing 72, 10--12 (jun 2009), 2079--2092. Google ScholarDigital Library
- A Kaylani, M Georgiopoulos, M Mollaghasemi, G C Anagnostopoulos, and C Sentelle. 2010. An adaptive multiobjective approach to evolving ART architectures. IEEE Transactions on Neural Networks 21, 4 (apr 2010), 529--50. Google ScholarDigital Library
- John R Koza. 1992. Genetic programming: on the programming of computers by means of natural selection. Vol. 1. MIT press. Google ScholarDigital Library
- Rong Li, Timothy R. Mersch, Oriana X. Wen, Assem Kaylani, and Georgios C. Ana. 2010. Multi-objective memetic evolution of ART-based classifiers. In Proceedings of the IEEE Congress on Evolutionary Computation. 1--8.Google Scholar
- M Lichman. 2013. UCI Machine Learning Repository. (2013). http://archive.ics.uci.edu/mlGoogle Scholar
- David J Montana. 1995. Strongly Typed Genetic Programming. Evol. Comput. 3, 2 (jun 1995), 199--230. Google ScholarDigital Library
- Sanjib Kumar Nayak, Sarat Chandra Nayak, and H. S. Behera. 2016. Evolving low complex higher order neural network based classifiers for medical data classification. In 2nd International Conference on Computational Intelligence in Data Mining, ICCIDM 2015 (Advances in Intelligent Systems and Computing), Himansu Sekhar Behera and Durga Prasad Mohapatra (Eds.), Vol. 411. Springer India, New Delhi, 415--425.Google Scholar
- José Carlos Ortiz-Bayliss, Hugo Terashima-Marín, and Santiago Enrique Conant-Pablos. 2015. A Neuro-evolutionary Hyper-heuristic Approach for Constraint Satisfaction Problems. Cognitive Computation (dec 2015), 1--13.Google Scholar
- F Pedregosa, G Varoquaux, A Gramfort, V Michel, B Thirion, O Grisel, M Blondel, P Prettenhofer, R Weiss, V Dubourg, J Vanderplas, A Passos, D Cournapeau, M Brucher, M Perrot, and E Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825--2830. Google ScholarDigital Library
- Travis Bueter Sean Harris and Daniel R. Tauritz. 2015. A Comparison of Genetic Programming Variants for Hyper-Heuristics. In Proceedings of the 17th Annual Conference Companion on Genetic and Evolutionary Computation (GECCO '15). 1043--1050. Google ScholarDigital Library
- Shing Chiang Tan, Junzo Watada, Zuwairie Ibrahim, and Marzuki Khalid. 2015. Evolutionary Fuzzy ARTMAP neural networks for classification of semiconductor defects. IEEE Transactions on Neural Networks and Learning Systems 26, 5 (2015), 933--50.Google ScholarCross Ref
- James R Williamson. 1996. Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps. Neural networks 9, 5 (1996), 881--897. Google ScholarDigital Library
- Xin Yao. 1999. Evolving artificial neural networks. Proc. IEEE 87, 9 (sep 1999), 1423--1447. arXiv:1108.1530Google Scholar
- Rui Xu, Georgios C. Anagnostopoulos, and Donald C. Wunsch. 2007. Multiclass Cancer Classification Using Semisupervised Ellipsoid ARTMAP and Particle Swarm Optimization with Gene Expression Data. IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB) 4, 1 (jan 2007), 65--77. Google ScholarDigital Library
- Rui Xu and Donald C Wunsch II. 2011. BARTMAP: A viable structure for biclustering. Neural Networks 24, 7 (2011), 709--716. Google ScholarDigital Library
Index Terms
- Evolutionary computation for the automated design of category functions for fuzzy ART: an initial exploration
Recommendations
Uncertain fuzzy self-organization based clustering: interval type-2 fuzzy approach to adaptive resonance theory
AbstractConventional unsupervised learning algorithms require knowledge of the desired number of clusters beforehand. Even if such knowledge is not required in advance, empirical selection of the parameter values may limit the adaptive ...
A two-leveled symbiotic evolutionary algorithm for clustering problems
Because of its unsupervised nature, clustering is one of the most challenging problems, considered as a NP-hard grouping problem. Recently, several evolutionary algorithms (EAs) for clustering problems have been presented because of their efficiency for ...
Clustering interval-valued proximity data using belief functions
The problem of clustering objects based on interval-valued dissimilarities is tackled in the framework of the Dempster--Shafer theory of belief functions. The proposed method assigns to each object a basic belief assignment (or mass function) defined on ...
Comments