A competence-performance based model to develop a syntactic language for artificial agents

Abstract

The hypothesis of language use is an attractive theory in order to explain how natural languages evolve and develop in social populations. In this paper we present a model partially based on the idea of language games, so that a group of artificial agents are able to produce and share a symbolic language with syntactic structure. Grammatical structure is induced by grammatical evolution of stochastic regular grammars with learning capabilities, while language development is refined by means of language games where the agents apply on-line probabilistic reinforcement learning. Within this framework, the model adapts the concepts of competence and performance in language, as they have been proposed in some linguistic theories. The first experiments in this article have been organized around the linguistic description of visual scenes with the possibility of changing the referential situations. A second and more complicated experimental setting is also analyzed, where linguistic descriptions are enforced to keep word order constraints.

Introduction

Language is one of the fundamental cognitive skills needed for the development of advanced multi-agent or multi-robot systems, as it allows communication and cooperation among group members. This necessity has recently motivated research where computational models are applied for developing artificial languages with increasing complexity. Computational simulations, and even experiments with real robots, have also been used for understanding properties of natural languages [11], [38]. The present article belongs to the former approach: we propose a new model that combines evolutionary algorithms and reinforcement learning so that a team of agents can construct a shared artificial language, with the important property of having syntactic structure.

The initial approaches for communication within teams of agents have focused on creating a basic vocabulary or lexicon that is shared by all members of the group. The vocabulary is generally a simple mapping between symbols and meanings. This problem is not trivial because of the Symbol Grounding Problem: how to effectively connect a symbol with its meaning [7]. Humans do not seem to be limited by this issue, but artificial organisms have serious difficulties. While some works address this problem [12], [26], [28], [32], [33], [34], [37], other proposals bypass it and focus on the higher level, while the aspects related to the physical association between symbols and meanings are assumed to have been solved or are not considered [8], [17], [39].

Although a shared vocabulary is essential and it has to be the first step, syntactic competence is vital for an agent to efficiently describe reality in a symbolic way. In this work, we approach the syntactic alignment of agents in a team by means of language games applying on-line reinforcement learning algorithms and grammatical evolution. This process allows the agents to evolve and adapt their language to the current linguistic situation. Syntactic alignment refers here to the process that allows a team of agents to develop a common syntactic language, by pairwise interactions or language games, without being told by a centralized source. The concept of language games is partly inspired by the ideas of Wittgenstein [41] and it has been applied in most of Steel’s works (see [35] for a recent review) and related authors such as [3], [16]. The importance of syntactic competence is currently so relevant that other approaches based on different ideas have been proposed in [2], [4], [13], [14], [19], [24], [25], [29].

Most of the cited works, except [12], [17], [26], are focused on explaining properties of natural languages by means of computational simulations. The above three exceptions are more oriented towards artificial language development, but they do not address syntax in depth. A more recent work deals with syntax [18], by allowing team members to tune the probabilities of stochastic regular grammars by means of language games and reinforcement learning. However, the grammar production rules themselves cannot change, as they are encoded in the agents’ controller from the beginning. In this work, we follow their approach based on reinforcement learning and language games [18], but we study how the agents can evolve the grammar itself. This process allows the agents to look for the grammar that best describes the current linguistic situation which is a very important skill if the agents need to talk about different contexts. To solve this problem we propose a model that takes into account the concepts of Competence and Performance as they were initially defined by Chomsky [5]. Competence is the knowledge that a speaker has about his/her language and performance stands for the specific use of the language in a specific context. Competence is developed by means of an evolutionary algorithm in our model and it is essentially a search process in the space of possible grammars that describe the current linguistic situation. On the other hand, performance is achieved by means of a reinforcement learning mechanism, which rewards the most used structures.

The rest of the paper is organized as follows. Section 2 describes the two tasks used in the computational experiments. Section 3 studies the process of producing and evolving the symbolic artificial language by means of grammatical evolution. Section 4 analyzes how the language can be aligned by means of on-line reinforcement learning algorithms. Section 5 summarizes the whole model. Section 6 enhances the initial model to enforce syntactic order constraints. In Section 7 both the setting and the experimental results are presented and discussed. The last section draws the main conclusions.