Skip to main content
Log in

Evolutionary Learning of Modular Neural Networks with Genetic Programming

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Evolutionary design of neural networks has shown a great potential as a powerful optimization tool. However, most evolutionary neural networks have not taken advantage of the fact that they can evolve from modules. This paper presents a hybrid method of modular neural networks and genetic programming as a promising model for evolutionary learning. This paper describes the concepts and methodologies for the evolvable model of modular neural networks, which might not only develop new functionality spontaneously, but also grow and evolve its own structure autonomously. We show the potential of the method by applying an evolved modular network to a visual categorization task with handwritten digits. Sophisticated network architectures as well as functional subsystems emerge from an initial set of randomly-connected networks. Moreover, the evolved neural network has reproduced some of the characteristics of natural visual system, such as the organization of coarse and fine processing of stimuli in separate pathways.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S.A. Harp, “Towards the genetic synthesis of neural networks,” in Proc. 3rd Int. Conf. Genetic Algorithms and Their Applications, Morgan Kaufmann: San Mateo, CA, 1989, pp. 360- 369.

    Google Scholar 

  2. D. Whitley and T. Hanson, “Optimizing neural networks using faster, more accurate genetic search,” in Proc. 3rd Int. Conf. Genetic Algorithms and Their Applications, Morgan Kaufmann: San Mateo, CA, 1989, pp. 391- 396.

    Google Scholar 

  3. H. Kitano, “Designing neural networks using genetic algorithms with graph generation system,” Complex Systems, vol. 4, no.4, pp. 461- 476, 1990.

    Google Scholar 

  4. D.T. Cliff, I. Harvey, and P. Husbands, “Incremental evolution of neural network architectures for adaptive behavior,” Technical Report CSRP 256, University of Sussex School of Cognitive and Computing Science, 1992.

  5. X. Yao, “Evolutionary artificial neural networks,” Int. Journal of Neural Systems, vol. 4, no.3, pp. 203- 222, 1993.

    Google Scholar 

  6. S. Nolfi, O. Miglino, and D. Parisi, “Phenotypic plasticity in evolving neural networks: Evolving the control system for an autonomous agent,” Technical Report PCIA-94-04, Institute of Psychology, C.N.R., Rome, 1994.

    Google Scholar 

  7. S.-B. Cho and K. Shimohara, “Toward evolvable model of modularized neural networks,” in Proc. the 5th Annual Conf. Japanese Neural Network Society, Tsukuba, November 1994, pp. 117- 118.

  8. G.F. Miller, P.M. Todd, and S.U. Hedge, “Designing neural networks using genetic algorithms,” in Proc. 3rd Int. Conf. Genetic Algorithms and Their Applications, Morgan Kaufmann: San Mateo, CA, 1989, pp. 379- 384.

    Google Scholar 

  9. S.-B. Cho and J.H. Kim, “Combining multiple neural networks by fuzzy integral for robust classification,” IEEE Trans. Systems, Man, and Cybernetics, vol. 25, no.2, pp. 380- 384, 1995.

    Google Scholar 

  10. R.A. Jacobs, M.I. Jordan, S.J. Nowlan, and G.E. Hinton, “Adaptive mixtures of local experts,” Neural Computation, vol. 3, pp. 79- 87, 1991.

    Google Scholar 

  11. J.B. Hampshire II and A. Waibel, “The meta-pi network: Building distributed knowledge representations for robust multisource pattern recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no.7, pp. 751- 769, 1992.

    Google Scholar 

  12. F. Gruau and D. Whitley, “The cellular development of neural networks: The interaction of learning and evolution,” Research Report 93-04, Laboratoire de l'Informatique du Parallelisme, Ecole Normale Superieure de Lyon, 1993.

  13. S. Nolfi and D. Parisi, “Growing neural networks,” Report PCIA-91-15, Institute of Psychology, C.N.R., Rome, 1991.

    Google Scholar 

  14. E. Mjolsness, D.H. Sharp, and J. Reinitz, “A connectionist model of development,” Journal of Theoretical Biology, vol. 152, pp. 429- 453, 1991.

    Google Scholar 

  15. K. Fleischer and A.H. Barr, “A simulation testbed for the study of multicellular development: The multiple mechanisms of morphogenesis,” in Artificial Life III, edited by C.G. Langton, Addison-Wesley: Reading, MA, pp. 389- 416, 1994.

    Google Scholar 

  16. J.R. Koza, Genetic Programming on the Programming of Computers by Means of Natural Selection, The MIT Press, 1992.

  17. D. Whitley, “The GENITOR algorithm and selective pressure: Why rank-based allocation of reproductive trials is best,” in Proc. 3rd Int. Conf. Genetic Algorithms and Their Applications, Morgan Kaufmann: San Mateo, CA, 1989, pp. 116- 121

    Google Scholar 

  18. J.M.J. Murre, R.H. Phaf, and G. Wolters, “CALM: Categorizing and learning module,” Neural Networks, vol. 5, pp. 55- 82, 1992.

    Google Scholar 

  19. D.E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, Addison-Wesley, 1989.

  20. C.Y. Suen, C. Nadal, T. Mai, R. Legault, and L. Lam, “Recognition of handwritten numerals based on the concept of multiple experts,” in Proc. 1st Int. Workshop Frontiers in Handwriting Recognition, Montreal, Canada, 1990, pp. 131- 144.

Download references

Author information

Authors and Affiliations

Authors

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cho, SB., Shimohara, K. Evolutionary Learning of Modular Neural Networks with Genetic Programming. Applied Intelligence 9, 191–200 (1998). https://doi.org/10.1023/A:1008388118869

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008388118869

Navigation