Andrew James Turner
ABSTRACT
Neuroevolution, the application of evolutionary algorithms to artificial neural networks (ANNs), is well-established in machine learning. Cartesian Genetic Programming (CGP) is a graph-based form of Genetic Programming which can easily represent ANNs. Cartesian Genetic Programming encoded ANNs (CGPANNs) can evolve every aspect of an ANN: weights, topology, arity and node transfer functions. This makes CGPANNs very suited to situations where appropriate configurations are not known in advance. The effectiveness of CGPANNs is compared with a large number of previous methods on three benchmark problems. The results show that CGPANNs perform as well as or better than many other approaches. We also discuss the strength and weaknesses of each of the three benchmarks.
- J. Abonyi and F. Szeifert. Supervised fuzzy clustering for the identification of fuzzy classifiers. Pattern Recognition Letters, 24(14):2195--2207, 2003. Google Scholar
Digital Library
- A. M. Ahmad and G. M. Khan. Bio-signal processing using cartesian genetic programming evolved artificial neural network (cgpann). In Frontiers of Information Technology (FIT), 2012 10th International Conference on, pages 261--268. IEEE, 2012. Google ScholarDigital Library
- F. Ahmad, N. Mat Isa, Z. Hussain, and S. Sulaiman. A genetic algorithm-based multi-objective optimization of an artificial neural network classifier for breast cancer diagnosis. Neural Computing & Applications, pages 1--9, 2012.Google Scholar
- A. Albrecht, G. Lappas, S. Vinterbo, C. Wong, and L. Ohno-Machado. Two applications of the LSA machine. In Proceedings of the 9th International Conference on Neural Information Processing, volume 1, pages 184--189. IEEE, 2002.Google ScholarCross Ref
- F. Gomez and R. Miikkulainen. Solving non-markovian control tasks with neuroevolution. In International Joint Conference on Artificial Intelligence, volume 16, pages 1356--1361, 1999. Google ScholarDigital Library
- F. Gomez, J. Schmidhuber, and R. Miikkulainen. Accelerated neural evolution through cooperatively coevolved synapses. The Journal of Machine Learning Research, 9:937--965, 2008. Google ScholarDigital Library
- F. J. Gomez and R. Miikkulainen. Robust non-linear control through neuroevolution. PhD thesis, 2003. Google ScholarDigital Library
- F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In Proceedings of the First Annual Conference on Genetic Programming, pages 81--89. MIT Press, 1996. Google ScholarDigital Library
- B. Guijarro-Berdiñas, O. Fontenla-Romero, B. Pérez-Sánchez, and P. Fraguela. A linear learning method for multilayer perceptrons using least-squares. Intelligent Data Engineering and Automated Learning, pages 365--374, 2007. Google ScholarDigital Library
- H. Hamilton, N. Shan, and N. Cercone. RIAC: A rule induction algorithm based on approximate classification. Technical report, University of Regina, 1996.Google Scholar
- M. Huang, Y. Hung, and W. Chen. Neural network classifier with entropy based feature selection on breast cancer diagnosis. Journal of medical systems, 34(5):865--873, 2010. Google ScholarDigital Library
- C. Igel. Neuroevolution for reinforcement learning using evolution strategies. In Evolutionary Computation, volume 4, pages 2588--2595. IEEE, 2003.Google Scholar
- S. Kamruzzaman, A. Hasan, A. Siddiquee, M. Mazumder, and E. Hoque. Medical diagnosis using neural network. In Proceedings of the International Conference on Electrical and Computer Engineering (ICECE-2004), pages 537--540, 2004.Google Scholar
- M. Karabatak and M. Ince. An expert system for detection of breast cancer based on association rules and neural network. Expert Systems with Applications, 36(2):3465--3469, 2009. Google ScholarDigital Library
- A. Kattan, R. Abdullah, and R. Salam. Harmony search based supervised training of artificial neural networks. In International Conference on Intelligent Systems, Modelling and Simulation (ISMS), pages 105--110. IEEE, 2010. Google ScholarDigital Library
- G. Khan, J. F. Miller, and D. Halliday. A developmental model of neural computation using cartesian genetic programming. In Genetic And Evolutionary Computation Conference: Proceedings of the 2007 GECCO conference companion on Genetic and evolutionary computation, volume 7, pages 2535--2542, 2007. Google ScholarDigital Library
- G. Khan, J. F. Miller, and D. Halliday. Developing neural structure of two agents that play checkers using cartesian genetic programming. In Proceedings of the 2008 GECCO conference on Genetic and evolutionary computation, pages 2169--2174. ACM, 2008. Google ScholarDigital Library
- M. Khan, G. Khan, and J. F. Miller. Efficient representation of recurrent neural networks for markovian/non-markovian non-linear control problems. In Intelligent Systems Design and Applications (ISDA), 2010 10th International Conference on, pages 615--620. IEEE, 2010.Google ScholarCross Ref
- M. Khan, G. Khan, and J. F. Miller. Evolution of optimal ANNs for non-linear control problems using cartesian genetic programming. In Proceedings of International Conference on Artificial Intelligence (ICAI 2010), 2010.Google Scholar
- M. M. Khan, G. M. Khan, and J. F. Miller. Evolution of neural networks using cartesian genetic programming. In Proceedings of IEEE World Congress on Computational Intelligence, 2010.Google ScholarCross Ref
- J. Koutnık, F. Gomez, and J. Schmidhuber. Evolving neural networks in compressed weight space. In Proceedings of the Conference on Genetic and Evolutionary Computation (GECCO-10), 2010. Google ScholarDigital Library
- J. R. Koza. Genetic Programming: On the Programming of Computers by Means of Natural Selection. The MIT Press, 1992. Google ScholarDigital Library
- K. Mangasarian. Neural network training via linear programming. Advances in Optimisation and Parallel Computing, pages 56--67, 1992.Google Scholar
- O. Mangasarian, R. Setiono, and W. Wolberg. Pattern recognition via linear programming: Theory and application to medical diagnosis. Large-scale numerical optimization, pages 22--31, 1990.Google Scholar
- T. Manning and P. Walsh. Improving the performance of cgpann for breast cancer diagnosis using crossover and radial basis functions. In Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics, pages 165--176. Springer, 2013. Google ScholarDigital Library
- J. F. Miller. An empirical study of the efficiency of learning boolean functions using a cartesian genetic programming approach. In Proceedings of the Genetic and Evolutionary Computation Conference, volume 2, pages 1135--1142. Citeseer, 1999.Google Scholar
- J. F. Miller. What bloat? cartesian genetic programming on boolean problems. In 2001 Genetic and Evolutionary Computation Conference Late Breaking Papers, pages 295--302, 2001.Google Scholar
- J. F. Miller, editor. Cartesian Genetic Programming. Springer, 2011.Google Scholar
- J. F. Miller and S. Smith. Redundancy and computational efficiency in cartesian genetic programming. Evolutionary Computation, IEEE Transactions on, 10(2):167--174, 2006. Google ScholarDigital Library
- J. F. Miller and P. Thomson. Cartesian genetic programming. In Proceedings of the Third European Conference on Genetic Programming (EuroGP2000), volume 10802, pages 121--132. Springer-Verlag, 2000. Google ScholarDigital Library
- D. Moriarty and R. Mikkulainen. Efficient reinforcement learning through symbiotic evolution. Machine learning, 22(1):11--32, 1996. Google ScholarDigital Library
- D. Nauck and R. Kruse. Obtaining interpretable fuzzy classification rules from medical data. Artificial intelligence in medicine, 16(2):149, 1999.Google Scholar
- C. Pena-Reyes and M. Sipper. A fuzzy-genetic approach to breast cancer diagnosis. Artificial intelligence in medicine, 17(2):131--155, 1999.Google Scholar
- N. Pokudom. Determine of appropriate neural networks structure using ant colony system. In ICCAS-SICE, 2009, pages 4522--4525. IEEE, 2009.Google Scholar
- D. Polani and R. Miikkulainen. Fast reinforcement learning through eugenic neuro-evolution. Technical report, University of Texas at Austin, Austin, TX, 1999. Google ScholarDigital Library
- K. Polat and S. Güneş. Breast cancer diagnosis using least square support vector machine. Digital Signal Processing, 17(4):694--701, 2007. Google ScholarDigital Library
- L. Prechelt. Proben1: A set of neural network benchmark problems and benchmarking rules. Fakultät für Informatik, Univ. Karlsruhe, Karlsruhe, Germany, Tech. Rep, 21:94, 1994.Google Scholar
- J. Pujol and R. Poli. Evolving the topology and the weights of neural networks using a dual representation. Applied Intelligence, 8(1):73--84, 1998. Google ScholarDigital Library
- A. Raad, A. Kalakech, and M. Ayache. Breast cancer classification using neural network approach: MLP and RBF. Networks, 7(8):9, 2012.Google Scholar
- M. Senapati, A. Mohanty, S. Dash, and P. Dash. Local linear wavelet neural network for breast cancer recognition. Neural Computing & Applications, pages 1--7, 2011.Google Scholar
- K. Stanley and R. Miikkulainen. Efficient evolution of neural network topologies. In Evolutionary Computation, 2002. CEC'02. Proceedings of the 2002 Congress on, volume 2, pages 1757--1762. IEEE, 2002. Google ScholarDigital Library
- K. O. Stanley. Efficient evolution of neural networks through complexification. PhD thesis, The University of Texas at Austin, 2004. Google ScholarDigital Library
- Y. Tsoy and V. Spitsyn. Using genetic algorithm with adaptive mutation mechanism for neural networks design and training. In Science and Technology. Proceedings. The 9th Russian-Korean International Symposium on, pages 709--714. IEEE, 2005.Google ScholarCross Ref
- V. K. Vassilev and J. F. Miller. The Advantages of Landscape Neutrality in Digital Circuit Evolution. In Proc. International Conference on Evolvable Systems, volume 1801 of LNCS, pages 252--263. Springer Verlag, 2000. Google ScholarDigital Library
- A. Wieland. Evolving neural network controllers for unstable systems. In Neural Networks, 1991., IJCNN-91-Seattle International Joint Conference on, volume 2, pages 667--673. IEEE, 1991.Google ScholarCross Ref
- J. Wu. MIMO CMAC neural network classifier for solving classification problems. Applied Soft Computing, 11(2):2326--2333, 2011. Google ScholarDigital Library
- X. Yao. Evolving artificial neural networks. Proceedings of the IEEE, 87(9):1423--1447, 1999.Google ScholarCross Ref
- X. Yao and Y. Liu. A new evolutionary system for evolving artificial neural networks. Neural Networks, IEEE Transactions on, 8(3):694--713, 1997. Google ScholarDigital Library
- L. Yingwei, N. Sundararajan, and P. Saratchandran. Performance evaluation of a sequential minimal radial bass function (RBF) neural network learning algorithm. Neural Networks, IEEE Transactions on, 9(2):308--318, 1998. Google ScholarDigital Library
- T. Yu and J. F. Miller. Neutrality and the evolvability of boolean function landscape. Genetic programming, pages 204--217, 2001. Google ScholarDigital Library
Index Terms
- Cartesian genetic programming encoded artificial neural networks: a comparison using three benchmarks
Recommendations
Recent Developments in Cartesian Genetic Programming and its Variants
Cartesian Genetic Programming (CGP) is a variant of Genetic Programming with several advantages. During the last one and a half decades, CGP has been further extended to several other forms with lots of promising advantages and applications. This ...
Recurrent Cartesian Genetic Programming of Artificial Neural Networks
Cartesian Genetic Programming of Artificial Neural Networks is a NeuroEvolutionary method based on Cartesian Genetic Programming. Cartesian Genetic Programming has recently been extended to allow recurrent connections. This work investigates applying ...
Neural network crossover in genetic algorithms using genetic programming
AbstractThe use of genetic algorithms (GAs) to evolve neural network (NN) weights has risen in popularity in recent years, particularly when used together with gradient descent as a mutation operator. However, crossover operators are often omitted from ...
Comments