Evolutionary construction and adaptation of intelligent systems

https://doi.org/10.1016/j.eswa.2010.04.070Get rights and content

Abstract

This paper introduces evolutionary techniques for automatically constructing intelligent self-adapting systems, capable of modifying their inner structure in order to learn from experience and self-adapt to a changing environment. These evolutionary techniques comprise an evolutionary system that is engineered by grammar-guided genetic programming, enabling the development of sub-symbolic and symbolic intelligent systems: artificial neural networks and knowledge-based systems, respectively. A context-free-grammar based codification system for artificial neural networks and rules, an initialization method and a crossover operator have been designed to properly balance the exploration and exploitation capabilities of the proposed system. This speeds up the convergence process and avoids trapping in local optima. This system has been applied to a medical domain: the detection of knee injuries from the analysis of isokinetic time series. The results of the evolved symbolic and sub-symbolic intelligent systems have been statistically compared with each other as part of a quantitative and qualitative performance analysis.

Introduction

Intelligent systems are built from knowledge extracted from a specific domain (Turbam, Aronson, & Liang, 2004). The knowledge is stored within the system in such a way that it can be processed to infer decisions that solve problems located within the domain. Many different kinds of intelligent systems from both the symbolic and sub-symbolic tracks of Artificial Intelligence (AI) have been successfully applied to knowledge areas ranging from industry and machine control to healthcare and education. Symbolic approaches intend to reproduce intelligent behaviour in a system built on a symbol-based representation of the world that is close to human reasoning. Sub-symbolic approaches try to imitate human cognition processes to reproduce intelligent behaviour without using symbols as an intermediate representation of real-world objects (Jones, 2008).

Knowledge engineering is the name given to the area of symbolic AI that acquires and formalizes specific knowledge from a domain, which is stored in a knowledge base (Gomez, Juristo, Montes, & Pazos, 1997). The use of this kind of systems is confined to the domain for which they were designed, for example, diagnosis of a particular disease. Linguistic rules are a symbolic knowledge representation able to act as an interface between human way of reasoning and computer processing. These rules use linguistic terms and have an antecedent-consequent structure, composed of easy-to-understand conditional sentences in the form “if X then Y”. These rules are also called inference rules because they are able to infer conclusions from current knowledge gathered from the environment. Applications of rule-based systems (RBS) are diverse and usually connected with solving real-world problems: medicine (Ohsaki, Yokoi, Abe, Tsumoto, & Yamaguchi, 2006), agriculture (Lee, Wu, & Wei, 2008), machine control (White & Lakany, 2008), diagnosis (Alonso, Caraça-Valente, González, & Montes, 2002), teaching and education (Canales, Peña, Peredo, Sossa, & Gutiérrez, 2007). Most of these areas use RBS to support human decision making. In knowledge areas where there is no need for human interaction, like machine control, RBS are designed to autonomously manage processes or machines. In the other areas, RBS are executed within a changing environment where systems may evolve over time. Take, for example, a RBS that is able to detect whether or not a patient has an illness considering the status of several symptoms within that patient’s organism. Those features are defined by domain experts during the system design phase according to the latest knowledge of the illness. When that knowledge is updated due to new results from medical research into the disease, the developed RBS, as well as the domain knowledge, becomes obsolete.

Artificial neural networks (ANN) are one of the most representative techniques of sub-symbolic AI. An ANN is an abstraction that simulates biological neural systems through mathematical models (Hassoun, 1995). This technique has been successfully applied to pattern recognition (Chen, Tai, Wang, Deng, & Chen, 2008) and classification problems (García-Pedrajas, Hervás-Martínez, & Ortíz-Boyer, 2005). The main disadvantage of an ANN is the process of designing of its inner structure and its training. An ANN has a purpose-specific structure, and it is trained by trial-and-error. The complexity of this process grows exponentially as the size of the network increases. As long as real-world problems require big networks, building ANNs to solve such problems will be computationally very costly.

Evolutionary computation (EC) is an area of the AI made up of a set of techniques based on the natural evolution and selection of the species proposed by Charles Darwin, as well as on Gregor Mendel’s discoveries in genetics (Holland, 1992). An EC system evolves populations of individuals starting from an initial randomly generated population that evolves through a fixed number of generations (De Jong, 2006). Generation by generation, the population is subjected to selection, crossover, mutation and replacement operators until it reaches the individual that codifies the optimal solution to the problem. EC has been successfully applied to solve searching and optimization problems (Scully & Brown, 2008), such as the generation of both symbolic (Couchet et al., 2008, Couchet et al., 2009) and sub-symbolic (Couchet et al., 2007, Manrique et al., 2006) self-adapting intelligent systems. Self-adapting systems can be applied to several different problems, evolving and adapting in tune with the problems: they are robust systems (Podgorelec, Kokol, Stiglic, Hericko, & Rozman, 2005). Another advantage of generating intelligent systems through EC is that it avoids the bottlenecks within the knowledge-based and neural systems development process. In knowledge-based systems, this drawback is due to a knowledge elicitation process that is highly dependant on the domain expert. In neural systems, it is caused by the trial-and-error design process.

Genetic Programming (GP), proposed by John R. Koza in the early 1990s (Koza, 1992) (Langdon & Poli, 2002), is one of EC’s most representative techniques. Its special feature is that it works with individuals that codify programs of non-fixed length (Michal, Ivry, Schalit-Cohen, Sipper, & Barash, 2007). Each individual in the GP population is a program that solves a given problem. A key drawback of GP is that it does not solve the closure problem (Koza, 1992): it can generate invalid individuals not belonging to the solution space during the evolutionary process. Processing these individuals increases the computational cost during the GP execution. In addition, the size of the GP individuals escapes control due to a phenomenon called code bloat (Panait & Luke, 2004).

Grammar-Guided Genetic Programming (GGGP) is a GP specialization aiming to solve the closure problem. It uses a Context-Free Grammar (CFG) to generate the language whose words are the whole set of individuals that codify a solution to a given problem (Whigham, 1995, Wong, 1995). Individuals are derivation trees of the CFG that, when the algorithm starts, are generated by a grammar-based initialization method. This method cannot generate invalid individuals because they are not contained in the language described by the CFG (O’Neil, 2003). The initialization method plays a very important role, as it influences the later convergence process of the evolutionary system (García-Arnau, Manrique, Ríos, & Rodríguez-Patón, 2007). The design of the crossover operator of a GGGP system is also responsible for solving the closure problem in such a way that the crossover of two valid individuals must generate a valid offspring. To avoid code bloat it is possible to set a parameter to control the size of the generated derivation trees in both the initialization process and the application of the crossover operator. One of the most representative operators is the Fair crossover. The Fair crossover solves this problem by controlling the size of the parts that the crossed individuals exchange (Crawford-Marks & Spector, 2002). Despite its simplicity, this control is very exhaustive and interferes with the operator’s exploration capability. Another important crossover operator is Whigham’s crossover (Whigham, 1995). This operator improves the exploration capability, as well as solving the closure problem. Even so, this operator still does not properly explore the search space, especially when working with ambiguous CFGs (Couchet et al., 2007, Hoai and McKay, 2002).

This paper presents an improved GGGP system. It includes both a grammar-based initialization method and a grammar-based crossover operator whose combined application boosts the search space exploration and exploitation capabilities. This system takes advantage of the grammar’s ambiguity, a property whereby different derivation trees represent the same sentence. This is a powerful and very efficient EC technique. The proposed system has been specifically designed to be able to construct and automatically self-adapt (robust) symbolic and sub-symbolic intelligent machines: rule- and fuzzy rule-based systems, and artificial neural networks. To do so, we have developed three different CFGs and codification systems. These automatically generated intelligent systems have been applied to a real-world problem located within the medical domain knowledge: the study of isokinetic time series for injury detection, diagnosis, rehabilitation and injury prevention (Couchet et al., 2008, Gioftsidou et al., 2008). All this is based on the analysis of isokinetic curves output by an isokinetic dynamometer on which patients exercise a muscle or joint.

Section snippets

Evolving intelligent systems

The proposed process for evolving intelligent systems is based on GGGP. Given a context-free grammar G, defined as a string-rewiring system comprising a 4-tuple G = N, ΣT, S, P)/ΣN  ΣT = Ø, where ΣN is the alphabet of non-terminal symbols, ΣT is the alphabet of terminal symbols, S represents the start symbol or axiom of the grammar, and P is the set of production rules, written in Backus-Naur Form. The individuals that are part of the genetic population codify a sentence of the language generated

Constructing intelligent systems

The evolutionary techniques explained in the previous section have been designed to automatically generate three kinds of intelligent systems: rule-based systems (RBS), fuzzy neural networks (FNN) and artificial neural networks (ANN). For this purpose, it is necessary to establish a mechanism to codify these symbolic and sub-symbolic knowledge bases as derivation trees that can be managed by the proposed CFG-based intelligent system evolution process.

The example shown throughout the previous

Applying intelligent systems to injury detection

The proposed process for evolving intelligent systems has been executed to automatically generate RBSs, ANNs and FNNs that have been applied to the real-world problem of the detection of knee injuries. The data involved in the experiments have been extracted from the analysis of isokinetic curves, obtained from knee exercises performed by real patients. Isokinetic curves are continuous time series codifying the results of performing several repetitions of the same exercise. They are subjected

Conclusions

This paper proposed an evolutionary system for generating self-adapting intelligent machines. This system uses GGGP searching capabilities to automatically extract knowledge from a data source and generate trained symbolic and sub-symbolic intelligent systems with rule and fuzzy rule knowledge bases and artificial neural networks, respectively. The proposed evolutionary system has two main advantages. First it operates in a fully expert-independent manner. Second the generated intelligent

Acknowledgements

This research is located within the framework of the project no. DEP2005-00232-C03-03 funded by the Spanish Ministry of Science, Education and Sports. Our thanks also go to Africa López from the Spanish Higher Sports Council’s Center of Sports Medicine.

References (31)

  • J. Couchet et al.

    Crossover and mutation operators for grammar-guided genetic programming

    Soft Computing

    (2007)
  • Crawford-Marks, R., & Spector, L. (2002). Size control via size fair genetic operators in the pushGP genetic...
  • K.A. De Jong

    Evolutionary computation: A unified approach

    (2006)
  • N. García-Pedrajas et al.

    Cooperative coevolution of artificial neural network ensembles for patterns classification

    IEEE Transactions on Evolutionary Computation

    (2005)
  • A. Gioftsidou et al.

    Isokinetic strength training program for muscular imbalances in professional soccer players

    Sport Sciences for Health

    (2008)
  • Cited by (14)

    • Classification of tea specimens using novel hybrid artificial intelligence methods

      2014, Sensors and Actuators, B: Chemical
      Citation Excerpt :

      In many cases one can achieve much higher performance while the complexity of the system is only slightly increased [3]. The most popular hybrid systems include the evolutionary-neural [4–9], evolutionary-fuzzy [10–14] and neuro-fuzzy systems [15–18]. Although, these systems are well known and used for many years, the subject is still actual and in particular is used for the analysis of non-linear multidimensional gas sensor signals, i.e. from electronic noses.

    • Bacterially inspired evolving system with an application to time series prediction

      2013, Applied Soft Computing Journal
      Citation Excerpt :

      Communication is done by producing and sensing acyl homoserine lactones (AHL) signal molecules, also called autoinducers. This bacterial evolutionary system is a general-purpose technique, because grammar-guided genetic programming (GGGP) assures that any intelligent system whose constraints can be codified using a context-free grammar can implement the biological processes of these bacteria [27]. The bacterial evolutionary system evolves variable-size individuals and, given that GGGP solves the closure problem [38], it is complete, because it includes initialization, selection and replacement operations – via cellular division and death –, as well as a conjugation operator that implements a grammatical crossover [39] by means of which bacteria share genetic material via conjugation [40].

    • Approximation of phenol concentration using novel hybrid computational intelligence methods

      2014, International Journal of Applied Mathematics and Computer Science
    View all citing articles on Scopus
    View full text