Skip to main content
Log in

Severe damage recovery in evolving soft robots through differentiable programming

  • Published:
Genetic Programming and Evolvable Machines Aims and scope Submit manuscript

Abstract

Biological systems are very robust to morphological damage, but artificial systems (robots) are currently not. In this paper we present a system based on neural cellular automata, in which locomoting robots are evolved and then given the ability to regenerate their morphology from damage through gradient-based training. Our approach thus combines the benefits of evolution to discover a wide range of different robot morphologies, with the efficiency of supervised training for robustness through differentiable update rules. The resulting neural cellular automata are able to grow virtual robots capable of regaining more than 80% of their functionality, even after severe types of morphological damage.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://academist-cf.com/projects/119?lang=en

References

  1. B.M. Carlson, Principles of Regenerative Biology (Elsevier/Academic Press, New York, 2011)

    Google Scholar 

  2. G.L. Wade, R.R. Westerfield, Basic Principles of Pruning Woody Plants (University of Georgia, 2009)

    Google Scholar 

  3. J.M. Davis, E.A. Estes, Spacing and pruning affect growth, yield, and economic returns of staked fresh-market tomatoes. J. Am. Soc. Hortic. Sci. 118(6), 719–725 (1993). https://doi.org/10.21273/JASHS.118.6.719

    Article  Google Scholar 

  4. H.T. Hartmann, D.E. Kester et al., Plant Propagation: Principles and Practices (Prentice-Hall, New Jersey, 1975)

    Google Scholar 

  5. W.A. Vieira, K.M. Wells, C.D. McCusker, Advancements to the axolotl model for regeneration and aging. Gerontology 66(3), 212–222 (2020). https://doi.org/10.1159/000504294

    Article  Google Scholar 

  6. M. Levin, J. Selberg, M. Rolandi, Endogenous bioelectrics in development, cancer, and regeneration: drugs and bioelectronic devices as electroceuticals for regenerative medicine. Iscience 22, 519–533 (2019). https://doi.org/10.1016/j.isci.2019.11.023

    Article  Google Scholar 

  7. M.C. Vogg, B. Galliot, C.D. Tsiairis, Model systems for regeneration: Hydra. Development 146(21), 177212 (2019). https://doi.org/10.1242/dev.177212

    Article  Google Scholar 

  8. N. Fausto, J.S. Campbell, K.J. Riehle, Liver regeneration. Hepatology 43(S1), 45–53 (2006).  https://doi.org/10.1002/hep.20969

    Article  Google Scholar 

  9. K. Horibe, K. Walker, S. Risi, Regenerating soft robots through neural cellular automata, in EuroGP, pp. 36–50 (2021)

  10. S. Sudhakaran, D. Grbic, S. Li, A. Katona, E. Najarro, C. Glanois, S. Risi, Growing 3d artefacts and functional machines with neural cellular automata. arXiv preprint arXiv:2103.08737 (2021)

  11. A. Mordvintsev, E. Randazzo, E. Niklasson, M. Levin, Growing neural cellular automata. Distill (2020). https://doi.org/10.23915/distill.00023.

  12. K. Sims, Evolving 3d morphology and behavior by competition. Artif. Life 1(4), 353–372 (1994). https://doi.org/10.1162/artl.1994.1.4.353

    Article  Google Scholar 

  13. K. Sims, Evolving virtual creaturesm, in Proceedings of the 21st Annual conference on computer graphics and interactive techniques, pp. 15–22 (1994). https://doi.org/10.1145/192161.192167

  14. F. Dellaert, R.D. Beer, Co-evolving body and brain in autonomous agents using a developmental model. Cleveland, OH 44106 (1994)

  15. P. Eggenberger, Evolving morphologies of simulated 3d organisms based on differential gene expression, in Proceedings of the Fourth European Conference on Artificial Life, pp. 205–213 (1997)

  16. E.H. Ostergaard, H.H. Lund, Evolving control for modular robotic units, in Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for The New Millennium (Cat. No. 03EX694), vol. 2, pp. 886–892 (IEEE, 2003)

  17. H. Lipson, J.B. Pollack, Automatic design and manufacture of robotic lifeforms. Nature 406(6799), 974–978 (2000). https://doi.org/10.1038/35023115

    Article  Google Scholar 

  18. S. Risi, D. Cellucci, H. Lipson, Ribosomal robots: evolved designs inspired by protein folding, in Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 263–270 (2013)

  19. K.O. Stanley, Compositional pattern producing networks: a novel abstraction of development. Genet. Program Evolvable Mach. 8(2), 131–162 (2007). https://doi.org/10.1007/s10710-007-9028-8

    Article  Google Scholar 

  20. N. Cheney, R. MacCurdy, J. Clune, H. Lipson, Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding. ACM SIGEVOlution 7(1), 11–23 (2014). https://doi.org/10.1145/2661735.2661737

    Article  Google Scholar 

  21. N. Cheney, J. Bongard, H. Lipson, Evolving soft robots in tight spaces, in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 935–942 (2015)

  22. N. Cheney, J. Bongard, V. SunSpiral, H. Lipson, Scalable co-optimization of morphology and control in embodied machines. J. R. Soc. Interface 15(143), 20170937 (2018). https://doi.org/10.1098/rsif.2017.0937

    Article  Google Scholar 

  23. J.E. Auerbach, J.C. Bongard, Environmental influence on the evolution of morphological complexity in machines. PLoS Comput. Biol. 10(1), 1003399 (2014). https://doi.org/10.1371/journal.pcbi.1003399

    Article  Google Scholar 

  24. J.E. Auerbach, J.C. Bongard, Evolving cppns to grow three-dimensional physical structures, in Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 627–634 (2010)

  25. J. Urzelai, D. Floreano, Evolutionary robotics: coping with environmental change, in Genetic and Evolutionary Computation Conference (GECCO’2000) (2000)

  26. S. Nolfi, D. Floreano, Learning and evolution. Auton. Robot. 7(1), 89–113 (1999). https://doi.org/10.1023/A:1008973931182

    Article  Google Scholar 

  27. K. Chatzilygeroudis, V. Vassiliades, J.-B. Mouret, Reset-free trial-and-error learning for robot damage recovery. Robot. Auton. Syst. 100, 236–250 (2018). https://doi.org/10.1016/j.robot.2017.11.010

    Article  Google Scholar 

  28. A. Cully, J. Clune, D. Tarapore, J.-B. Mouret, Robots that can adapt like animals. Nature 521(7553), 503–507 (2015). https://doi.org/10.1038/nature14422

    Article  Google Scholar 

  29. T. Kano, E. Sato, T. Ono, H. Aonuma, Y. Matsuzaka, A. Ishiguro, A brittle star-like robot capable of immediately adapting to unexpected physical damage. Royal Soc. Open Sci. 4(12), 171200 (2017). https://doi.org/10.1098/rsos.171200

    Article  MathSciNet  Google Scholar 

  30. E. Najarro, S. Risi, Meta-learning through hebbian plasticity in random networks. arXiv preprint arXiv:2007.02686 (2020)

  31. S. Kriegman, N. Cheney, F. Corucci, J.C. Bongard, Interoceptive robustness through environment-mediated morphological development, in Proceedings of the Genetic and Evolutionary Computation Conference, pp. 109–116 (2018)

  32. K. Walker, H. Hauser, Evolution of morphology through sculpting in a voxel based robot, in ALIFE 2021: The 2021 Conference on Artificial Life (MIT Press, 2021)

  33. D.S. Shah, J.P. Powers, L.G. Tilton, S. Kriegman, J. Bongard, R. Kramer-Bottiglio, A soft robot that adapts to environments through shape change. Nat. Mach. Intell. 3(1), 51–59 (2021). https://doi.org/10.1038/s42256-020-00263-1

    Article  Google Scholar 

  34. D. Shah, B. Yang, S. Kriegman, M. Levin, J. Bongard, R. Kramer-Bottiglio, Shape changing robots: bioinspiration, simulation, and physical realization. Adv. Mater. 33(19), 2002882 (2021). https://doi.org/10.1002/adma.202002882

    Article  Google Scholar 

  35. S. Kriegman, S. Walker, D. Shah, M. Levin, R. Kramer-Bottiglio, J. Bongard, Automated shapeshifting for function recovery in damaged robots. arXiv preprint arXiv:1905.09264 (2019)

  36. S. Kriegman, D. Blackiston, M. Levin, J. Bongard, A scalable pipeline for designing reconfigurable organisms. Proc. Natl. Acad. Sci. 117(4), 1853–1859 (2020). https://doi.org/10.1073/pnas.1910837117

    Article  Google Scholar 

  37. J. Von Neumann, in Theory of Self-Reproducing Automata, ed. by A.W. Burks (University of Illinois Press, 1966)

  38. J.F. Miller, Evolving a self-repairing self-regulating french flag organism, in Proceeding of Genetic and Evolutionary Computation Conference, pp. 129–139 (Springer-Verlag, Berlin, 2004)

  39. W. Gilpin, Cellular automata as convolutional neural networks. Phys. Rev. E 100(3), 032402 (2019). https://doi.org/10.1103/PhysRevE.100.032402

    Article  Google Scholar 

  40. J.D. Hiller, H. Lipson, Multi material topological optimization of structures and mechanisms, in Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 1521–1528 (2009)

  41. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn Res 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  42. S. Hochreiter, Long short-term memory. Neural Comput. 1780, 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  43. M. Levin, G. Pezzulo, J.M. Finkelstein, Endogenous bioelectric signaling networks: exploiting voltage gradients for control of growth and form. Annu. Rev. Biomed. Eng. 19, 353–387 (2017). https://doi.org/10.1146/annurev-bioeng-071114-040647

    Article  Google Scholar 

  44. K.A. McLaughlin, M. Levin, Bioelectric signaling in regeneration: Mechanisms of ionic controls of growth and form. Dev. Biol. 433(2), 177–189 (2018). https://doi.org/10.1016/j.ydbio.2017.08.032

    Article  Google Scholar 

  45. J.H. Holland, Genetic algorithms. Sci. Am. 267(1), 66–73 (1992). http://www.jstor.org/stable/24939139

  46. A.E. Eiben, J.E. Smith, Introduction to Evolutionary Computing. Nat. Comput. Ser. Springer, Berlin, Heidelberg (2003). https://doi.org/10.1007/978-3-662-05094-1

  47. F.P. Such, V. Madhavan, E. Conti, J. Lehman, K.O. Stanley, J. Clune, Deep Neuroevolution: Genetic Algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv (2017) arXiv:1712.06567

  48. S. Risi, K.O. Stanley, Deep neuroevolution of recurrent and discrete world models, in Proceedings of the Genetic and Evolutionary Computation Conference, pp. 456–462 (2019)

  49. K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  50. V. Nair, G.E. Hinton, Rectified linear units improve restricted Boltzmann machines, in International conference on machine learning (2010). https://icml.cc/Conferences/2010/papers/432.pdf

  51. D.P. Kingma, J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  52. B.G. Woolley, K.O. Stanley, On the deleterious effects of a priori objectives on evolution and representation, in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, pp. 957–964 (2011)

  53. S. Nichele, M.B. Ose, S. Risi, G. Tufte, CA-NEAT: evolved compositional pattern producing networks for cellular automata morphogenesis and replication. IEEE Transact. Cognitive Develop. Syst. 10(3), 687–700 (2017). https://doi.org/10.1109/TCDS.2017.2737082

    Article  Google Scholar 

  54. J.K. Pugh, L.B. Soros, K.O. Stanley, Quality diversity: A new frontier for evolutionary computation. Front. Robot. AI 3, 40 (2016). https://doi.org/10.3389/frobt.2016.00040

    Article  Google Scholar 

  55. J. Lehman, J. Clune, D. Misevic, C. Adami, L. Altenberg, J. Beaulieu, P.J. Bentley, S. Bernard, G. Beslon, D.M. Bryson et al., The surprising creativity of digital evolution: A collection of anecdotes from the evolutionary computation and artificial life research communities. Artif. Life 26(2), 274–306 (2020). https://doi.org/10.1162/artl_a_00319

    Article  Google Scholar 

  56. T. Howison, S. Hauser, J. Hughes, F. Iida, Reality-assisted evolution of soft robots through large-scale physical experimentation: a review. arXiv (2020) arXiv:2009.13960

  57. N. El-Atab, R.B. Mishra, F. Al-Modaf, L. Joharji, A.A. Alsharif, H. Alamoudi, M. Diaz, N. Qaiser, M.M. Hussain, Soft actuators for soft robotic applications: A review. Adv. Intell. Syst. 2(10), 2000128 (2020). https://doi.org/10.1002/aisy.202000128

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Tobitate! (Leap for Tomorrow) Young Ambassador Program, a DFF-Research Project1 grant (9131-00042B), and KH’s Academist supportersFootnote 1 (Takaaki Aoki, Hirohito M. Kondo, Takeshi Oura, Yusuke Kajimoto, Ryuta Aoki).

Funding

S. Risi was funded by DFF-Starting Grant (Grant number 90063-00046B).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Kazuya Horibe.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Horibe, K., Walker, K., Berg Palm, R. et al. Severe damage recovery in evolving soft robots through differentiable programming. Genet Program Evolvable Mach 23, 405–426 (2022). https://doi.org/10.1007/s10710-022-09433-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10710-022-09433-z

Keywords

Navigation