Skip to main content
SpringerLink
Log in

A Melting Pot of Evolution and Learning

  • Chapter
  • First Online:
Genetic Programming Theory and Practice XX

Part of the book series: Genetic and Evolutionary Computation ((GEVO))

Abstract

We survey eight recent works by our group, involving the successful blending of evolutionary algorithms with machine learning and deep learning:

  1. 1.

    Binary and Multinomial Classification through Evolutionary Symbolic Regression,

  2. 2.

    Classy Ensemble: A Novel Ensemble Algorithm for Classification,

  3. 3.

    EC-KitY: Evolutionary Computation Tool Kit in Python,

  4. 4.

    Evolution of Activation Functions for Deep Learning-Based Image Classification,

  5. 5.

    Adaptive Combination of a Genetic Algorithm and Novelty Search for Deep Neuroevolution,

  6. 6.

    An Evolutionary, Gradient-Free, Query-Efficient, Black-Box Algorithm for Generating Adversarial Instances in Deep Networks,

  7. 7.

    Foiling Explanations in Deep Neural Networks,

  8. 8.

    Patch of Invisibility: Naturalistic Black-Box Adversarial Attacks on Object Detectors.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Sipper, M., Olson, R.S., Moore, J.H.: Evolutionary computation: the next major transition of artificial intelligence? BioData Min. 10(1), 26 (2017)

    Google Scholar 

  2. Sipper, M.: Binary and multinomial classification through evolutionary symbolic regression. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’22, pp. 300–303. Association for Computing Machinery, New York, NY, USA (2022a)

    Google Scholar 

  3. Sipper, M.: Classy ensemble: a novel ensemble algorithm for classification. https://arxiv.org/abs/2302.10580 (2023a)

  4. Sipper, M., Halperin, T., Tzruia, I., Elyasaf, A.: EC-KitY: Evolutionary computation tool kit in Python with seamless machine learning integration. SoftwareX 22, 101381 (2023b)

    Article  Google Scholar 

  5. Lapid, R., Sipper, M.: Evolution of activation functions for deep learning-based image classification. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, GECCO ’22, pp. 2113–2121. Association for Computing Machinery, New York, NY, USA (2022a)

    Google Scholar 

  6. Segal, E., Sipper, M.: Adaptive combination of a genetic algorithm and novelty search for deep neuroevolution. In: Bäck, T., van Stein, B., Wagner, C., Garibaldi, J.M., Lam, H.K., Cottrell, M., Doctor, F., Filipe, J., Warwick, K., Kacprzyk, J. (eds.) Proceedings of the 14th International Joint Conference on Computational Intelligence, IJCCI 2022, Valletta, Malta, October 24–26, 2022, pp. 143–150. SciTePress (2022)

    Google Scholar 

  7. Lapid, R., Haramaty, Z., Sipper, M.: An evolutionary, gradient-free, query-efficient, black-box algorithm for generating adversarial instances in deep convolutional neural networks. Algorithms 15(11), (2022b)

    Google Scholar 

  8. Tamam, S.V., Lapid, R., Sipper, M.: Foiling explanations in deep neural networks. https://arxiv.org/abs/2211.14860 (2022)

  9. Lapid, R., Sipper, M.: Patch of invisibility: naturalistic black-box adversarial attacks on object detectors. https://arxiv.org/abs/2303.04238 (2023)

  10. GPLearn. https://gplearn.readthedocs.io/ (2021). Accessed 30 April 2021

  11. Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  Google Scholar 

  12. Miller, J.F.: Cartesian Genetic Programming, pp. 17–34. Springer (2011)

    Google Scholar 

  13. Quade, M.: Cartesian. https://github.com/Ohjeah/cartesian (2020)

  14. Pena-Reyes, C.A., Sipper, M.: Fuzzy CoCo: a cooperative-coevolutionary approach to fuzzy modeling. IEEE Trans. Fuzzy Syst. 9(5), 727–737 (2001)

    Article  Google Scholar 

  15. Sipper, M., Moore, J.H.: OMNIREP: originating meaning by coevolving encodings and representations. Memetic Comput. 11(3), 251–261 (2019)

    Google Scholar 

  16. Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: A next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2623–2631 (2019)

    Google Scholar 

  17. Sipper, M., Moore, J.H.: Conservation machine learning. BioData Min. 13(1), 9 (2020)

    Google Scholar 

  18. Sipper, M., Moore, J.H.: Conservation machine learning: a case study of random forests. Nat. Sci. Rep. 11(1), 3629 (2021a)

    Google Scholar 

  19. Metevier, B., Saini, A.K., Spector, L.: Lexicase selection beyond genetic programming. In: Banzhaf, W., Spector, L., Sheneman, L. (eds.) Genetic Programming Theory and Practice XVI, pp. 123–136. Springer (2019)

    Google Scholar 

  20. Spector, L.: Assessment of problem modality by differential performance of lexicase selection in genetic programming: a preliminary report. In: Proceedings of the 14th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 401–408. ACM (2012)

    Google Scholar 

  21. Sipper, M., Moore, J.H.: Symbolic-regression boosting. Genet. Program. Evolvable Mach. 1–25 (2021b)

    Google Scholar 

  22. Sipper, M., Moore, J.H.: AddGBoost: a gradient boosting-style algorithm based on strong learners. Mach. Learn. Appl. 7, 100243 (2022b)

    Google Scholar 

  23. Sipper, M.: Combining deep learning with good old-fashioned machine learning. SN Comput. Sci. 4(1), 85 (2022c)

    Google Scholar 

  24. Agostinelli, F., Hoffman, M., Sadowski, P., Baldi, P.: Learning activation functions to improve deep neural networks (2014). arXiv:1412.6830

  25. Saha, S., Nagaraj, N., Mathur, A., Yedida, R.: Evolution of novel activation functions in neural network training with applications to classification of exoplanets (2019). arXiv:1906.01975

  26. Sharma, S., Sharma, S.: Activation functions in neural networks. Towar. Data Sci. 6(12), 310–316 (2017)

    Google Scholar 

  27. Sipper, M.: Neural networks with à la carte selection of activation functions. SN Comput. Sci. 2(6), 1–9 (2021)

    Article  Google Scholar 

  28. Miller, J.F., Harding, S.L.: Cartesian genetic programming. In: Proceedings of the 10th Annual Conference Companion on Genetic and Evolutionary Computation, GECCO ’08, pp. 2701–2726. Association for Computing Machinery, New York, NY, USA (2008). ISBN 9781605581316

    Google Scholar 

  29. Sutton, R.S., Barto, A.G.: Reinforcement Learning: an Introduction, 2nd edn. MIT press (2018)

    Google Scholar 

  30. Salimans, T., Ho, J., Chen, X., Sidor, S., Sutskever, I.: Evolution strategies as a scalable alternative to reinforcement learning (2017). arXiv:1703.03864

  31. Such, F.P., Madhavan, V., Conti, E., Lehman, J., Stanley, K.O., Clune, J.: Deep neuroevolution: genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning (2017). arXiv:1712.06567

  32. Lehman, J., Stanley, K.O.: Exploiting open-endedness to solve problems through the search for novelty. In: Proceedings of the Eleventh International Conference on Artificial Life (ALIFE). MIT Press, Cambridge, MA, (2008)

    Google Scholar 

  33. Wang, X., Jin, H., He, K.: Natural language adversarial attack and defense in word level (2019)

    Google Scholar 

  34. Morris, J.X., Lifland, E., Yoo, J.Y., Qi, Y.: Textattack: a framework for adversarial attacks in natural language processing. In: Proceedings of the 2020 EMNLP (2020)

    Google Scholar 

  35. Carlini, N., Wagner, D.: Audio adversarial examples: Targeted attacks on speech-to-text. In: 2018 IEEE Security and Privacy Workshops (SPW), pp. 1–7. IEEE (2018)

    Google Scholar 

  36. Schönherr, L., Kohls, K., Zeiler, S., Holz, T., Kolossa, D.: Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding (2018). arXiv:1808.05665

  37. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2014). arXiv:1412.6572

  38. Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: 2016 IEEE European Symposium on Security and Privacy (EuroS &P), pp. 372–387. IEEE (2016)

    Google Scholar 

  39. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 39–57. IEEE (2017)

    Google Scholar 

  40. Gu, S., Rigazio, L.: Towards deep neural network architectures robust to adversarial examples (2014). arXiv:1412.5068

  41. Moosavi-Dezfooli, S.-M., Fawzi, A., Frossard, P.: Deepfool: a simple and accurate method to fool deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2574–2582 (2016)

    Google Scholar 

  42. Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z.B., Swami, A.: Practical black-box attacks against machine learning. In: Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, pp. 506–519 (2017)

    Google Scholar 

  43. Van Lent, M., Fisher, W., Mancuso, M.: An explainable artificial intelligence system for small-unit tactical behavior. In: Proceedings of the National Conference on Artificial Intelligence, pp. 900–907. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2004

    Google Scholar 

  44. Došilović, F.K., Brčić, M., Hlupić, N.: Explainable artificial intelligence: a survey. In: 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), pp. 0210–0215. IEEE (2018)

    Google Scholar 

  45. Walia, S., Kumar, K., Agarwal, S., Kim, H.: Using xai for deep learning-based image manipulation detection with shapley additive explanation. Symmetry 14(8), 1611 (2022)

    Article  Google Scholar 

  46. Fidel, G., Bitton, R., Shabtai, A.: When explainability meets adversarial learning: Detecting adversarial examples using shap signatures. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2020)

    Google Scholar 

  47. Kao, C.-Y., Chen, J., Markert, K., Böttinger, K.: Rectifying adversarial inputs using xai techniques. In: 2022 30th European Signal Processing Conference (EUSIPCO), pp. 573–577. IEEE (2022)

    Google Scholar 

  48. Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., Song, D.: Robust physical-world attacks on deep learning visual classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1625–1634 (2018)

    Google Scholar 

  49. Wierstra, D., Schaul, T., Glasmachers, T., Sun, Y., Peters, J., Schmidhuber, J.: Natural evolution strategies. J. Mach. Learn. Res. 15(1), 949–980 (2014)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

This research was partially supported by the following grants: Israeli Innovation Authority through the Trust.AI consortium; Israeli Science Foundation grant no. 2714/19; Israeli Smart Transportation Research Center (ISTRC); Israeli Council for Higher Education (CHE) via the Data Science Research Center, Ben-Gurion University of the Negev, Israel.

Author information

Authors and Affiliations

  1. Department of Computer Science, Ben-Gurion University of the Negev, 8410501, Beer-Sheva, Israel

    Moshe Sipper, Tomer Halperin, Zvika Haramaty, Raz Lapid, Eyal Segal, Itai Tzruia & Snir Vitrack Tamam

  2. Department of Software and Information Systems Engineering, Ben-Gurion University of the Negev, 8410501, Beer-Sheva, Israel

    Achiya Elyasaf

  3. DeepKeep, Tel-Aviv, Israel

    Raz Lapid

Authors
  1. Moshe Sipper
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Achiya Elyasaf
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tomer Halperin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zvika Haramaty
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Raz Lapid
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Eyal Segal
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Itai Tzruia
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Snir Vitrack Tamam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Moshe Sipper .

Editor information

Editors and Affiliations

  1. School of Information, Communications and Media, University of Applied Sciences Upper Austria, Hagenberg, Austria

    Stephan Winkler

  2. Engineering Sciences Graduate Program, Tecnológico Nacional de México, IT de Tijuana, Tijuana, Baja California, Mexico

    Leonardo Trujillo

  3. Department of Computer Science and Engineering, Michigan State University, East Lansing, MI, USA

    Charles Ofria

  4. School of Computing, Queen's University, Kingston, ON, Canada

    Ting Hu

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Sipper, M. et al. (2024). A Melting Pot of Evolution and Learning. In: Winkler, S., Trujillo, L., Ofria, C., Hu, T. (eds) Genetic Programming Theory and Practice XX. Genetic and Evolutionary Computation. Springer, Singapore. https://doi.org/10.1007/978-981-99-8413-8_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8413-8_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8412-1

  • Online ISBN: 978-981-99-8413-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation