Skip to main content

Advertisement

SpringerLink
Log in

Using Cartesian Genetic Programming Approach with New Crossover Technique to Design Convolutional Neural Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In image classification problems, Convolutional Neural Networks (CNNs) are deep neural networks that include a variety of different layers aimed at classifying images. Until today, the most promising and state-of-the-art method in image recognition tasks is CNN. Tuning the deep network with a large number of hyperparameters to maximize performance would be an excruciating task that requires lots of time and engineering efforts. To construct that high-performance architecture, experts should go through a lot of trial and error. Neural Architecture Search is a way to automatically fabricate an accurate network architecture. An evolutionary algorithm called Cartesian Genetic Programming (CGP) with a new crossover operation based on the multiple Sequence Alignment algorithm is proposed in this paper to construct an appropriate neural network without the burden of building manually. This new method has a remarkable improvement over a standard CGP only by adding a crossover operator. The datasets for training on the proposed method were CIFAR-10 and CIFAR-100. The results show that it achieves a good balance between accuracy and the number of trainable parameters compared to the other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

References

  1. Lecun Y, Bottou LI, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. IEEE

  2. Krizhevsky A, Hinton GE, Sutskever I (2012) ImageNet classification with deep convolutional neural networks. NIPS

  3. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. ICLR

  4. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: IEEE conference on computer vision and pattern recognition (CVPR). https://doi.org/10.1109/CVPR.2015.7298594

  5. He K, Zhang X, Ren X, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385v1. https://doi.org/10.48550/arXiv.1512.03385

  6. Chollet F (2017) Xception: deep learning with depthwise separable convolutions. arXiv:1610.02357v3. https://doi.org/10.48550/arXiv.1610.02357

  7. Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. arXiv:1611.05431v2. https://doi.org/10.48550/arXiv.1611.05431

  8. Wistuba M, R A, Pedapati T (2019) A survey on neural architecture search. arXiv: 1905.01392v2

  9. Ren P, Xiao Y, Chang X, Huang PY, Li Z, Chen X, Wang X (2020) A comprehensive survey of neural architecture search: challenges and solutions. ACM Comput Surv 54(4):1–34

    Article  Google Scholar 

  10. Elsken T, Metzen JH, Hutter F (2019) Neural architecture search: a survey. arXiv:1808.05377. https://doi.org/10.48550/arXiv.1808.05377

  11. Liu Y, Sun Y, Xue B, Zhang M, Yen G (2020) A survey on evolutionary neural architecture search. https://doi.org/10.48550/arXiv.2008.10937

  12. Sun Yanan, Xue Bing, Zhang Mengjie, Yen Gary G (2019) Completely automated CNN architecture design based on blocks. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2019.2919608

    Article  Google Scholar 

  13. Zoph B, Le QV (2017) Neural architecture search with reinforcement learning. arXiv:1611.01578. https://doi.org/10.48550/arXiv.1611.01578

  14. Miller JF, P T, Fogarty T (1997) Designing electronic circuits using evolutionary algorithms, arithmetic circuits: a case study. In: Wiley genetic algorithms and evolution strategies in engineering and computer science: recent advancements and industrial applications

  15. Miller JF, Thomson P (2000) Cartesian genetic programming. In: Springer-Verlag proceeding of the third European conference on genetic programming LNCS 1802

  16. Baker B, Gupta O, Naik N, Raskar R (2017) Designing neural network architectures using reinforcement learning. arXiv:1611.02167. https://doi.org/10.48550/arXiv.1611.02167

  17. Zhong Z, Yan J, Wu W, Shao J, Liu C-L (2018) Practical block-wise neural network architecture generation. arXiv:1708.05552. https://doi.org/10.48550/arXiv.1708.05552

  18. Miller GF, Todd PM, Hegde SU (1989) Designing neural networks using genetic algorithms. In: Proceedings of the third international conference on genetic algorithms. Morgan Kaufmann Publishers Inc., pp 379–384

  19. Stanley KO, Miikkulainen R (2002) Evolving neural networks through augmenting topologies. IEEE Evolut Comput. https://doi.org/10.1162/106365602320169811

    Article  Google Scholar 

  20. Breuel T, Shafait F (2010) Automlp: simple, effective, fully automated learning rate and size adjustment. In: The learning workshop, Utah

  21. Real E, Moore S, Selle A, Saxena S, Suematsu YL, Tan J, Le QV, Kurakin A (2017) Large-scale evolution of image classifiers. arXiv:1703.01041. https://doi.org/10.48550/arXiv.1703.01041

  22. Xie L, Yuille A (2017) Genetic CNN. In: Proceedings of the IEEE international conference on computer vision (ICCV)

  23. Liu H, Simonyan K, Vinyals O, Fernando C, Kavukcuoglu K (2018) Hierarchical representations for efficient architecture search. arXiv:1711.00436. https://doi.org/10.48550/arXiv.1711.00436

  24. Wistuba M, Rawat A, Pedapati T (2018) Deep learning architecture search by neuro-cell-based evolution with function-preserving mutations. In: Springer ECML PKDD, pp 243–258

  25. Yao X (1999) Evolving artificial neural networks. Proc. EEE 10(1109/5):784219. https://doi.org/10.1109/5.784219

    Article  Google Scholar 

  26. Suganuma M, Shirakawa S, Nagao T (2020) Designing convolutional neural network architectures using cartesian genetic programming. In: Deep neural evolution. Springer natural computing series, pp 185–208

  27. Clegg J, Walker JA, Miller JF (2007) A new crossover technique for cartesian genetic programming. In: GECCO, pp 1580–1587. https://doi.org/10.1145/1276958.1277276

  28. Hutt B, Warwick K (2007) Synapsing variable-length crossover: meaningful crossover for variable-length genomes. IEEE Trans Evolut Comput. https://doi.org/10.1109/TEVC.2006.878096

    Article  Google Scholar 

  29. Sung Wing-Kin (2010) Algorithms in bioinformatics: a practical introduction. CRC mathematical and computational biology series. Chapman and Hall/CRC Press

    MATH  Google Scholar 

  30. Cormen TH et al (2009) Introduction to algorithms, 3rd edn. The MIT Press

    MATH  Google Scholar 

  31. Gusfield D (1993) Efficient methods for multiple sequence alignment with guaranteed error bounds. Bull Math Biol 55(1):141–154

    Article  MathSciNet  MATH  Google Scholar 

  32. Schaffer JD (1989) Uniform crossover in genetic algorithms. In: Proceedings of the third international conference on genetic algorithms, pp 2–9

  33. The CIFAR-10 and CIFAR-100, https://www.cs.toronto.edu/~kriz/cifar.html

  34. Loshchilov I, Hutter F (2017) SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv:1608.03983. https://doi.org/10.48550/arXiv.1608.03983

  35. Ying C, Klein A, Christiansen E, Real E, Murphy K, Hutter F (2019) NAS-Bench-101: towards reproducible neural architecture search. arXiv: 1902.09635

  36. Dong X, Yang Y (2020) NAS-BENCH-201: extending the scope of reproducible neural architecture search. arXiv:2001.00326v2

  37. Huang G, Liu Z, van der Maaten L, Weinberger KQ (2018) Densely connected convolutional networks. https://doi.org/10.48550/arXiv.1608.06993

  38. Larsson G, Maire M, Shakhnarovich G (2017) FractalNet: ultra-deep neural networks without residuals. https://doi.org/10.48550/arXiv.1605.07648

  39. Huang G, Sun Y, Liu Z, Sedra D, Weinberger KQ (2016) Deep networks with stochastic depth. https://doi.org/10.48550/arXiv.1603.09382

  40. Ma N, Zhang X, Zheng H-T, Sun J (2018) ShuffleNet V2: Practical guidelines for efficient CNN architecture design. https://doi.org/10.48550/arXiv.1807.11164

  41. Elsken T, Metzen J-H, Hutter F (2017) Simple and efficient architecture search for convolutional neural networks. https://doi.org/10.48550/arXiv.1711.04528

  42. Zagoruyko S, Komodakis N (2018) Wide residual networks. https://doi.org/10.48550/arXiv.1605.07146

Download references

Funding

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Author information

Author notes

  1. A. Sharifi and M. Teshnehlab contributed equally to this work.

Authors and Affiliations

  1. Department of Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

    Ali Torabi & Arash Sharifi

  2. Department of Control Engineering, K.N. Toosi University of Technology, Tehran, Iran

    Mohammad Teshnehlab

Authors
  1. Ali Torabi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Arash Sharifi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohammad Teshnehlab
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Arash Sharifi.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Torabi, A., Sharifi, A. & Teshnehlab, M. Using Cartesian Genetic Programming Approach with New Crossover Technique to Design Convolutional Neural Networks. Neural Process Lett 55, 5451–5471 (2023). https://doi.org/10.1007/s11063-022-11093-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-11093-0

Keywords

Search

Navigation