Abstract
Biological systems are very robust to morphological damage, but artificial systems (robots) are currently not. In this paper we present a system based on neural cellular automata, in which locomoting robots are evolved and then given the ability to regenerate their morphology from damage through gradient-based training. Our approach thus combines the benefits of evolution to discover a wide range of different robot morphologies, with the efficiency of supervised training for robustness through differentiable update rules. The resulting neural cellular automata are able to grow virtual robots capable of regaining more than 80% of their functionality, even after severe types of morphological damage.
Similar content being viewed by others
References
B.M. Carlson, Principles of Regenerative Biology (Elsevier/Academic Press, New York, 2011)
G.L. Wade, R.R. Westerfield, Basic Principles of Pruning Woody Plants (University of Georgia, 2009)
J.M. Davis, E.A. Estes, Spacing and pruning affect growth, yield, and economic returns of staked fresh-market tomatoes. J. Am. Soc. Hortic. Sci. 118(6), 719–725 (1993). https://doi.org/10.21273/JASHS.118.6.719
H.T. Hartmann, D.E. Kester et al., Plant Propagation: Principles and Practices (Prentice-Hall, New Jersey, 1975)
W.A. Vieira, K.M. Wells, C.D. McCusker, Advancements to the axolotl model for regeneration and aging. Gerontology 66(3), 212–222 (2020). https://doi.org/10.1159/000504294
M. Levin, J. Selberg, M. Rolandi, Endogenous bioelectrics in development, cancer, and regeneration: drugs and bioelectronic devices as electroceuticals for regenerative medicine. Iscience 22, 519–533 (2019). https://doi.org/10.1016/j.isci.2019.11.023
M.C. Vogg, B. Galliot, C.D. Tsiairis, Model systems for regeneration: Hydra. Development 146(21), 177212 (2019). https://doi.org/10.1242/dev.177212
N. Fausto, J.S. Campbell, K.J. Riehle, Liver regeneration. Hepatology 43(S1), 45–53 (2006). https://doi.org/10.1002/hep.20969
K. Horibe, K. Walker, S. Risi, Regenerating soft robots through neural cellular automata, in EuroGP, pp. 36–50 (2021)
S. Sudhakaran, D. Grbic, S. Li, A. Katona, E. Najarro, C. Glanois, S. Risi, Growing 3d artefacts and functional machines with neural cellular automata. arXiv preprint arXiv:2103.08737 (2021)
A. Mordvintsev, E. Randazzo, E. Niklasson, M. Levin, Growing neural cellular automata. Distill (2020). https://doi.org/10.23915/distill.00023.
K. Sims, Evolving 3d morphology and behavior by competition. Artif. Life 1(4), 353–372 (1994). https://doi.org/10.1162/artl.1994.1.4.353
K. Sims, Evolving virtual creaturesm, in Proceedings of the 21st Annual conference on computer graphics and interactive techniques, pp. 15–22 (1994). https://doi.org/10.1145/192161.192167
F. Dellaert, R.D. Beer, Co-evolving body and brain in autonomous agents using a developmental model. Cleveland, OH 44106 (1994)
P. Eggenberger, Evolving morphologies of simulated 3d organisms based on differential gene expression, in Proceedings of the Fourth European Conference on Artificial Life, pp. 205–213 (1997)
E.H. Ostergaard, H.H. Lund, Evolving control for modular robotic units, in Proceedings 2003 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Computational Intelligence in Robotics and Automation for The New Millennium (Cat. No. 03EX694), vol. 2, pp. 886–892 (IEEE, 2003)
H. Lipson, J.B. Pollack, Automatic design and manufacture of robotic lifeforms. Nature 406(6799), 974–978 (2000). https://doi.org/10.1038/35023115
S. Risi, D. Cellucci, H. Lipson, Ribosomal robots: evolved designs inspired by protein folding, in Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, pp. 263–270 (2013)
K.O. Stanley, Compositional pattern producing networks: a novel abstraction of development. Genet. Program Evolvable Mach. 8(2), 131–162 (2007). https://doi.org/10.1007/s10710-007-9028-8
N. Cheney, R. MacCurdy, J. Clune, H. Lipson, Unshackling evolution: evolving soft robots with multiple materials and a powerful generative encoding. ACM SIGEVOlution 7(1), 11–23 (2014). https://doi.org/10.1145/2661735.2661737
N. Cheney, J. Bongard, H. Lipson, Evolving soft robots in tight spaces, in Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation, pp. 935–942 (2015)
N. Cheney, J. Bongard, V. SunSpiral, H. Lipson, Scalable co-optimization of morphology and control in embodied machines. J. R. Soc. Interface 15(143), 20170937 (2018). https://doi.org/10.1098/rsif.2017.0937
J.E. Auerbach, J.C. Bongard, Environmental influence on the evolution of morphological complexity in machines. PLoS Comput. Biol. 10(1), 1003399 (2014). https://doi.org/10.1371/journal.pcbi.1003399
J.E. Auerbach, J.C. Bongard, Evolving cppns to grow three-dimensional physical structures, in Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 627–634 (2010)
J. Urzelai, D. Floreano, Evolutionary robotics: coping with environmental change, in Genetic and Evolutionary Computation Conference (GECCO’2000) (2000)
S. Nolfi, D. Floreano, Learning and evolution. Auton. Robot. 7(1), 89–113 (1999). https://doi.org/10.1023/A:1008973931182
K. Chatzilygeroudis, V. Vassiliades, J.-B. Mouret, Reset-free trial-and-error learning for robot damage recovery. Robot. Auton. Syst. 100, 236–250 (2018). https://doi.org/10.1016/j.robot.2017.11.010
A. Cully, J. Clune, D. Tarapore, J.-B. Mouret, Robots that can adapt like animals. Nature 521(7553), 503–507 (2015). https://doi.org/10.1038/nature14422
T. Kano, E. Sato, T. Ono, H. Aonuma, Y. Matsuzaka, A. Ishiguro, A brittle star-like robot capable of immediately adapting to unexpected physical damage. Royal Soc. Open Sci. 4(12), 171200 (2017). https://doi.org/10.1098/rsos.171200
E. Najarro, S. Risi, Meta-learning through hebbian plasticity in random networks. arXiv preprint arXiv:2007.02686 (2020)
S. Kriegman, N. Cheney, F. Corucci, J.C. Bongard, Interoceptive robustness through environment-mediated morphological development, in Proceedings of the Genetic and Evolutionary Computation Conference, pp. 109–116 (2018)
K. Walker, H. Hauser, Evolution of morphology through sculpting in a voxel based robot, in ALIFE 2021: The 2021 Conference on Artificial Life (MIT Press, 2021)
D.S. Shah, J.P. Powers, L.G. Tilton, S. Kriegman, J. Bongard, R. Kramer-Bottiglio, A soft robot that adapts to environments through shape change. Nat. Mach. Intell. 3(1), 51–59 (2021). https://doi.org/10.1038/s42256-020-00263-1
D. Shah, B. Yang, S. Kriegman, M. Levin, J. Bongard, R. Kramer-Bottiglio, Shape changing robots: bioinspiration, simulation, and physical realization. Adv. Mater. 33(19), 2002882 (2021). https://doi.org/10.1002/adma.202002882
S. Kriegman, S. Walker, D. Shah, M. Levin, R. Kramer-Bottiglio, J. Bongard, Automated shapeshifting for function recovery in damaged robots. arXiv preprint arXiv:1905.09264 (2019)
S. Kriegman, D. Blackiston, M. Levin, J. Bongard, A scalable pipeline for designing reconfigurable organisms. Proc. Natl. Acad. Sci. 117(4), 1853–1859 (2020). https://doi.org/10.1073/pnas.1910837117
J. Von Neumann, in Theory of Self-Reproducing Automata, ed. by A.W. Burks (University of Illinois Press, 1966)
J.F. Miller, Evolving a self-repairing self-regulating french flag organism, in Proceeding of Genetic and Evolutionary Computation Conference, pp. 129–139 (Springer-Verlag, Berlin, 2004)
W. Gilpin, Cellular automata as convolutional neural networks. Phys. Rev. E 100(3), 032402 (2019). https://doi.org/10.1103/PhysRevE.100.032402
J.D. Hiller, H. Lipson, Multi material topological optimization of structures and mechanisms, in Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, pp. 1521–1528 (2009)
N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, R. Salakhutdinov, Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn Res 15(1), 1929–1958 (2014)
S. Hochreiter, Long short-term memory. Neural Comput. 1780, 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
M. Levin, G. Pezzulo, J.M. Finkelstein, Endogenous bioelectric signaling networks: exploiting voltage gradients for control of growth and form. Annu. Rev. Biomed. Eng. 19, 353–387 (2017). https://doi.org/10.1146/annurev-bioeng-071114-040647
K.A. McLaughlin, M. Levin, Bioelectric signaling in regeneration: Mechanisms of ionic controls of growth and form. Dev. Biol. 433(2), 177–189 (2018). https://doi.org/10.1016/j.ydbio.2017.08.032
J.H. Holland, Genetic algorithms. Sci. Am. 267(1), 66–73 (1992). http://www.jstor.org/stable/24939139
A.E. Eiben, J.E. Smith, Introduction to Evolutionary Computing. Nat. Comput. Ser. Springer, Berlin, Heidelberg (2003). https://doi.org/10.1007/978-3-662-05094-1
F.P. Such, V. Madhavan, E. Conti, J. Lehman, K.O. Stanley, J. Clune, Deep Neuroevolution: Genetic Algorithms are a competitive alternative for training deep neural networks for reinforcement learning. arXiv (2017) arXiv:1712.06567
S. Risi, K.O. Stanley, Deep neuroevolution of recurrent and discrete world models, in Proceedings of the Genetic and Evolutionary Computation Conference, pp. 456–462 (2019)
K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
V. Nair, G.E. Hinton, Rectified linear units improve restricted Boltzmann machines, in International conference on machine learning (2010). https://icml.cc/Conferences/2010/papers/432.pdf
D.P. Kingma, J. Ba, Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)
B.G. Woolley, K.O. Stanley, On the deleterious effects of a priori objectives on evolution and representation, in Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, pp. 957–964 (2011)
S. Nichele, M.B. Ose, S. Risi, G. Tufte, CA-NEAT: evolved compositional pattern producing networks for cellular automata morphogenesis and replication. IEEE Transact. Cognitive Develop. Syst. 10(3), 687–700 (2017). https://doi.org/10.1109/TCDS.2017.2737082
J.K. Pugh, L.B. Soros, K.O. Stanley, Quality diversity: A new frontier for evolutionary computation. Front. Robot. AI 3, 40 (2016). https://doi.org/10.3389/frobt.2016.00040
J. Lehman, J. Clune, D. Misevic, C. Adami, L. Altenberg, J. Beaulieu, P.J. Bentley, S. Bernard, G. Beslon, D.M. Bryson et al., The surprising creativity of digital evolution: A collection of anecdotes from the evolutionary computation and artificial life research communities. Artif. Life 26(2), 274–306 (2020). https://doi.org/10.1162/artl_a_00319
T. Howison, S. Hauser, J. Hughes, F. Iida, Reality-assisted evolution of soft robots through large-scale physical experimentation: a review. arXiv (2020) arXiv:2009.13960
N. El-Atab, R.B. Mishra, F. Al-Modaf, L. Joharji, A.A. Alsharif, H. Alamoudi, M. Diaz, N. Qaiser, M.M. Hussain, Soft actuators for soft robotic applications: A review. Adv. Intell. Syst. 2(10), 2000128 (2020). https://doi.org/10.1002/aisy.202000128
Acknowledgements
This work was supported by the Tobitate! (Leap for Tomorrow) Young Ambassador Program, a DFF-Research Project1 grant (9131-00042B), and KH’s Academist supportersFootnote 1 (Takaaki Aoki, Hirohito M. Kondo, Takeshi Oura, Yusuke Kajimoto, Ryuta Aoki).
Funding
S. Risi was funded by DFF-Starting Grant (Grant number 90063-00046B).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Horibe, K., Walker, K., Berg Palm, R. et al. Severe damage recovery in evolving soft robots through differentiable programming. Genet Program Evolvable Mach 23, 405–426 (2022). https://doi.org/10.1007/s10710-022-09433-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10710-022-09433-z