Abstract
In this paper, we are interested in ad hoc autonomous agent team composition using cooperative co-evolutionary algorithms (CCEA). In order to accurately capture the individual contribution of team agents, we propose to limit the number of agents which are updated in-between team evaluations. However, this raises two important problems with respect to (1) the cost of accurately estimating the marginal contribution of agents with respect to the team learning speed and (2) completing tasks where improving team performance requires multiple agents to update their policies in a synchronized manner. We introduce a CCEA algorithm that is capable of learning how to update just the right amount of agents’ policies for the task at hand. We use a variation of the El Farol Bar problem, formulated as a multi-robot resource selection problem, to provide an experimental validation of the algorithms proposed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arthur, W.B.: Inductive reasoning and bounded rationality. Am. Econ. Rev. 84(2), 406–411 (1994)
Auer, P., Cesa-Bianchi, N., Freund, Y., Schapire, R.E.: The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32(1), 48–77 (2002)
Beyer, H.-G., Schwefel, H.-P.: Evolution strategies - a comprehensive introduction. Nat. Comput. 1(1), 3–52 (2002). https://doi.org/10.1023/A:1015059928466
De Jong, K.A.: Evolutionary computation: a unified approach. In: Proceedings of the 2016 on Genetic and Evolutionary Computation Conference Companion, pp. 185–199 (2016)
Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing, vol. 53. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-662-05094-1
Funes, P., Pujals, E.: Intransitivity revisited coevolutionary dynamics of numbers games. In: Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation, GECCO 2005, pp. 515–521. Association for Computing Machinery, New York (2005)
Gomes, J., Mariano, P., Christensen, A.L.: Novelty-driven cooperative coevolution. Evol. Comput. 25(2), 275–307 (2017)
Gomes, J., Mariano, P., Christensen, A.L.: Dynamic team heterogeneity in cooperative coevolutionary algorithms. IEEE Trans. Evol. Comput. 22(6), 934–948 (2018)
Gomes, J., Mariano, P., Christensen, A.L.: Challenges in cooperative coevolution of physically heterogeneous robot teams. Nat. Comput. 18(1), 29–46 (2016). https://doi.org/10.1007/s11047-016-9582-1
Ma, X., et al.: A survey on cooperative co-evolutionary algorithms. IEEE Trans. Evol. Comput. 23(3), 421–441 (2019)
Noë, R., Hammerstein, P.: Biological markets: supply and demand determine the effect of partner choice in cooperation, mutualism and mating. Behav. Ecol. Sociobiol. 35(1), 1–11 (1994). https://doi.org/10.1007/BF00167053
Panait, L.: Theoretical convergence guarantees for cooperative coevolutionary algorithms. Evol. Comput. 18(4), 581–615 (2010)
Potter, M.A., De Jong, K.A.: A cooperative coevolutionary approach to function optimization. In: Davidor, Y., Schwefel, H.-P., Männer, R. (eds.) PPSN 1994. LNCS, vol. 866, pp. 249–257. Springer, Heidelberg (1994). https://doi.org/10.1007/3-540-58484-6_269
Potter, M.A., De Jong, K.A.: Cooperative coevolution: an architecture for evolving coadapted subcomponents. Evol. Comput. 8, 1–29 (2000)
Rahmattalabi, A., Chung, J.J., Colby, M., Tumer, K.: D++: structural credit assignment in tightly coupled multiagent domains. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4424–4429. IEEE (2016)
Rockefeller, G., Khadka, S., Tumer, K.: Multi-level fitness critics for cooperative coevolution. In: Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020), pp. 1143–1151, 9–13 May 2020
Shapley, L.S.: A value for n-person games. In: Contributions to the Theory of Games II. Annals of Mathematics Studies, vol. 28, pp. 307–317 (1953)
Shoham, Y., Leyton-Brown, K.: Multiagent Systems: Algorithmic, Game-Theoretic, and Logical Foundations. Cambridge University Press, Cambridge (2008)
Stone, P., Kaminka, G.A., Kraus, S., Rosenschein, J.S.: Ad hoc autonomous agent teams: collaboration without pre-coordination. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2010, pp. 1504–1509. AAAI Press (2010)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge (2018)
Tumer, K., Agogino, A.K., Wolpert, D.H.: Learning sequences of actions in collectives of autonomous agents. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems: Part 1, pp. 378–385 (2002)
West, S.A., Griffin, A.S., Gardner, A.: Social semantics: altruism, cooperation, mutualism, strong reciprocity and group selection. J. Evol. Biol. 20(2), 415–32 (2007)
Wolpert, D.H., Tumer, K.: Optimal payoff functions for members of collectives. Adv. Complex Syst. 4(2/3), 265–279 (2001)
Wolpert, D.H., Tumer, K.: An introduction to collective intelligence. Technical report, NASA (2008)
Zerbel, N., Tumer, K.: The power of suggestion. In: Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2020), pp. 1602–1610, 9–13 May 2020
Acknowledgements
This work is funded by ANR grant ANR-18-CE33-0006.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fontbonne, N., Maudet, N., Bredeche, N. (2022). Cooperative Co-evolution and Adaptive Team Composition for a Multi-rover Resource Allocation Problem. In: Medvet, E., Pappa, G., Xue, B. (eds) Genetic Programming. EuroGP 2022. Lecture Notes in Computer Science, vol 13223. Springer, Cham. https://doi.org/10.1007/978-3-031-02056-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-031-02056-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-02055-1
Online ISBN: 978-3-031-02056-8
eBook Packages: Computer ScienceComputer Science (R0)