Skip to main content
SpringerLink
Log in

Evolving the Topology and the Weights of Neural Networks Using a Dual Representation

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Evolutionary computation is a class of global search techniques based on the learning process of a population of potential solutions to a given problem, that has been successfully applied to a variety of problems. In this paper a new approach to the construction of neural networks based on evolutionary computation is presented. A linear chromosome combined to a graph representation of the network are used by genetic operators, which allow the evolution of the architecture and the weights simultaneously without the need of local weight optimization. This paper describes the approach, the operators and reports results of the application of this technique to several binary classification problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. S. Haykin. Neural networks, a comprehensive foundation. Macmillan College Publishing Company, Inc., 866 Third Avenue, New York, New York 10022, 1994.

  2. R. Reed. Pruning algorithms: a survey. IEEE Transactions on Neural Networks, 4(5):740- 747, 1993.

    Google Scholar 

  3. M. Frean. The upstart algorithm: a method for constructing and training feed-forward neural networks. Neural Computation, 2:198- 209, 1990.

    Google Scholar 

  4. S. E. Fahlman and C. Lebiere. The cascade-correlation learning architecture. In D. Touretzky, editor, Advances in Neural Information Processing Systems, volume 2, pages 524- 532. Morgan Kaufmann, 1990.

  5. D. Chen, C. Giles, G. Sun, H. Chen, Y. Less, and M. Goudreau. Constructive learning of recurrent neural networks. In IEEE International Conference on Neural Networks (ICNN), pages 1196- 1201, 1993.

  6. C. Campbell and C. Vicente. The target switch algorithm: a constructive learning procedure for feed-forward neural networks. Neural Computation, 7:1245- 1264, 1995.

    Google Scholar 

  7. X. Yao. An overview of evolutionary computation. Chinese Journal of Advanced Software Research, 3(1), 1996. To be published.

  8. D. Fogel. Evolutionary computation: toward a new philosophy of machine Intelligence. IEEE Press, Piscataway, NJ, USA, 1995.

    Google Scholar 

  9. D. Goldberg. Genetic algorithm in search, optimization and machine learning. Addison-Wesley, Reading, Massachusetts, 1989.

    Google Scholar 

  10. M. Mitchell. An introduction to genetic algorithms. MIT Press, Cambridge, Massachusetts, USA, 1996.

    Google Scholar 

  11. L. Davis. Handbook of Genetic Algorithms. Van Nostrand Rheinhold, New York, NY, 1991.

    Google Scholar 

  12. Michalewicz Zbigniew. Genetic Algorithms [plus] Data Structures = Evolution Programs. Springer-Verlag, Berlin, 1994.

    Google Scholar 

  13. S. Harp, T. Samad, and A. Guha. Toward the genetic synthesis of neural networks. In J. Schaffer, editor, Proceedings of the 3rd International Conference on Genetic Algorithms (ICGA), pages 360- 369, San Mateo, CA, USA, 1989.

  14. M. Mandischer. Evolving recurrent neural networks with non-binary encoding. In Proceedings of the 2nd IEEE Conference on Evolutionary Computation (ICEC), volume 2, pages 584- 589, Perth, Australia, Nov. 1995.

    Google Scholar 

  15. M. Mandischer. Representation and evolution of neural networks. In Proceedings of the International Conference on Artificial Neural Nets and Genetic Algorithms (ICANNGA), pages 643- 649, 1993.

  16. H. Kitano. Neurogenetic learning: an integrated method of designing and training neural networks using genetic algorithms. Physica D, 75:225- 238, 1994.

    Google Scholar 

  17. S. Fujita and H. Nishimura. An evolutionary approach to associative memory in recurrent neural networks. Neural Processing, 1(2), 1994.

  18. H. Braun and P. Zagorski. ENZO-M-a hybrid approach for optimizing neural networks by evolution and learning. In Y. Davidor, H. Schwefel, and H. Manner, editors, Parallel Problem Solving from Nature (PPSN3), volume 866. Springer-Verlag, 1994. Lecture Notes in Computer Science.

  19. X. Yao and J. Liu. Evolutionary artificial neural networks that learn and generalize well. In Proceedings of the 1996 IEEE International Conference on Neural Networks, Washington, DC, Jun. 1996. Submitted.

  20. V. Maniezzo. Genetic evolution of the topology and weight distribution of neural networks. IEEE Transactions on Neural Networks, 5(1):39- 53, Jan. 1994.

    Google Scholar 

  21. F. Gruau. Neural network synthesis using cellular encoding and the genetic algorithm. PhD thesis, Laboratoire de L'informatique du Parallélisme, Ecole Normale Supériere de Lyon, Lyon, France, 1994.

    Google Scholar 

  22. S. Nolfi and D. Parisi. Growing neural networks. Technical Report PCIA-95-15, Rome, Italy, Jun. 1991.

  23. B. T. Zhang and H. Mühlenbein. Evolving optimal neural networks using genetic algorithms with Occam's razor. Complex Systems, 7(3):199- 220, 1993.

    Google Scholar 

  24. P. J. Angeline, G. M. Saunders, and J. B. Pollack. An evolutionary algorithm that constructs recurrent neural networks. IEEE Transactions on Neural Networks, 5(1), 1994.

  25. D. Fogel. Using evolutionary programming to create neural networks that are capable of playing Tic Tac Toe. In IEEE International Conference on Neural Networks (ICNN). IEEE Press, 1993.

  26. J. McDonnell and D. Waagen. Evolving neural network connectivity. In Proceedings of IEEE International Conference on Neural Networks (ICNN), pages 863- 868, San Francisco, CA, USA, 1993.

  27. J. Holland. Adaptation in natural and artificial systems. University of Michigan Press, Ann Arbor, Michigan, 1975.

    Google Scholar 

  28. J. R. Koza. Genetic Programming, on the Programming of Computers by Means of Natural Selection. The MIT Press, Cambridge, Massachusetts, 1992.

    Google Scholar 

  29. B. Zhang and H. Muehlenbein. Genetic programming of minimal neural nets using Occam's razor. In S. Forrest, editor, Proceedings of the 5th international conference on genetic algorithms (ICGA'93), pages 342- 349. Morgan Kaufmann, 1993.

  30. B. Zhang and Muehlenbein. Synthesis of sigma-pi neural networks by the breeder genetic programming. In Proceedings of IEEE International Conference on Evolutionary Computation (ICEC),World Congress on Computational Intelligence, pages 318- 323, Orlando, Florida, USA, Jun. 1994. IEEE Computer Society Press.

  31. F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. Koza, D. Goldberg, D. Fogel, and R. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 81- 89, Stanford University, CA, USA, Jul. 1996. MIT Press.

    Google Scholar 

  32. R. Poli. Some steps towards a form of parallel distributed genetic programming. In Proceedings of the First On-line Workshop on Soft Computing, Aug. 1996.

  33. R. Poli. Discovery of symbolic, neuron-symbolic and neural networks with parallel distributed genetic programming. In 3rd International Conference on Artificial Neural Networks and Genetic Algorithms (ICANNGA), 1997.

  34. X. Yao and Y. Shi. A preliminary study on designing artificial neural networks using co-evolution. In Proceedings of the IEEE Singapore International Conference on Intelligent Control and Instrumentation, pages 149- 154, Jun. 1995.

  35. K. Tang, C. Chan, K. Man, and S. Kwong. Genetic structure for NN topology and weights optimization. In Proceedings of the International Conference on Genetic Algorithms in Engineering Systems: innovations and applications (GALESIA), pages 250- 255, Sept. 1995.

  36. D. Dasgupta and D. McGregor. Designing application-specific neural networks using the structured genetic algorithm. In L. Whitley and J. Schaffer, editors, Proceedings of International Workshop on Combinations of Genetic Algorithms and Neural Networks (COGANN), pages 87- 96. IEEE Computer Society Press, Jun. 1992.

  37. D. Whitley, T. Starkweather, and C. Bogart. Genetic algorithms and neural networks: Optimizing connections and connectivity. Parallel Computing, 14-3:347- 361, 1990.

    Google Scholar 

  38. D. Rumelhart and J. McClelland. Parallel Distributed Processing. IEEE Press, Piscataway, NJ, USA, 1986.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, The University of Birmingham, Birmingham, B15 2TT, UK

    João Carlos Figueira Pujol & Riccardo Poli

Authors
  1. João Carlos Figueira Pujol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Riccardo Poli
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Pujol, J.C.F., Poli, R. Evolving the Topology and the Weights of Neural Networks Using a Dual Representation. Applied Intelligence 8, 73–84 (1998). https://doi.org/10.1023/A:1008272615525

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1008272615525

Search

Navigation