Evolutionary construction and adaptation of intelligent systems
Introduction
Intelligent systems are built from knowledge extracted from a specific domain (Turbam, Aronson, & Liang, 2004). The knowledge is stored within the system in such a way that it can be processed to infer decisions that solve problems located within the domain. Many different kinds of intelligent systems from both the symbolic and sub-symbolic tracks of Artificial Intelligence (AI) have been successfully applied to knowledge areas ranging from industry and machine control to healthcare and education. Symbolic approaches intend to reproduce intelligent behaviour in a system built on a symbol-based representation of the world that is close to human reasoning. Sub-symbolic approaches try to imitate human cognition processes to reproduce intelligent behaviour without using symbols as an intermediate representation of real-world objects (Jones, 2008).
Knowledge engineering is the name given to the area of symbolic AI that acquires and formalizes specific knowledge from a domain, which is stored in a knowledge base (Gomez, Juristo, Montes, & Pazos, 1997). The use of this kind of systems is confined to the domain for which they were designed, for example, diagnosis of a particular disease. Linguistic rules are a symbolic knowledge representation able to act as an interface between human way of reasoning and computer processing. These rules use linguistic terms and have an antecedent-consequent structure, composed of easy-to-understand conditional sentences in the form “if X then Y”. These rules are also called inference rules because they are able to infer conclusions from current knowledge gathered from the environment. Applications of rule-based systems (RBS) are diverse and usually connected with solving real-world problems: medicine (Ohsaki, Yokoi, Abe, Tsumoto, & Yamaguchi, 2006), agriculture (Lee, Wu, & Wei, 2008), machine control (White & Lakany, 2008), diagnosis (Alonso, Caraça-Valente, González, & Montes, 2002), teaching and education (Canales, Peña, Peredo, Sossa, & Gutiérrez, 2007). Most of these areas use RBS to support human decision making. In knowledge areas where there is no need for human interaction, like machine control, RBS are designed to autonomously manage processes or machines. In the other areas, RBS are executed within a changing environment where systems may evolve over time. Take, for example, a RBS that is able to detect whether or not a patient has an illness considering the status of several symptoms within that patient’s organism. Those features are defined by domain experts during the system design phase according to the latest knowledge of the illness. When that knowledge is updated due to new results from medical research into the disease, the developed RBS, as well as the domain knowledge, becomes obsolete.
Artificial neural networks (ANN) are one of the most representative techniques of sub-symbolic AI. An ANN is an abstraction that simulates biological neural systems through mathematical models (Hassoun, 1995). This technique has been successfully applied to pattern recognition (Chen, Tai, Wang, Deng, & Chen, 2008) and classification problems (García-Pedrajas, Hervás-Martínez, & Ortíz-Boyer, 2005). The main disadvantage of an ANN is the process of designing of its inner structure and its training. An ANN has a purpose-specific structure, and it is trained by trial-and-error. The complexity of this process grows exponentially as the size of the network increases. As long as real-world problems require big networks, building ANNs to solve such problems will be computationally very costly.
Evolutionary computation (EC) is an area of the AI made up of a set of techniques based on the natural evolution and selection of the species proposed by Charles Darwin, as well as on Gregor Mendel’s discoveries in genetics (Holland, 1992). An EC system evolves populations of individuals starting from an initial randomly generated population that evolves through a fixed number of generations (De Jong, 2006). Generation by generation, the population is subjected to selection, crossover, mutation and replacement operators until it reaches the individual that codifies the optimal solution to the problem. EC has been successfully applied to solve searching and optimization problems (Scully & Brown, 2008), such as the generation of both symbolic (Couchet et al., 2008, Couchet et al., 2009) and sub-symbolic (Couchet et al., 2007, Manrique et al., 2006) self-adapting intelligent systems. Self-adapting systems can be applied to several different problems, evolving and adapting in tune with the problems: they are robust systems (Podgorelec, Kokol, Stiglic, Hericko, & Rozman, 2005). Another advantage of generating intelligent systems through EC is that it avoids the bottlenecks within the knowledge-based and neural systems development process. In knowledge-based systems, this drawback is due to a knowledge elicitation process that is highly dependant on the domain expert. In neural systems, it is caused by the trial-and-error design process.
Genetic Programming (GP), proposed by John R. Koza in the early 1990s (Koza, 1992) (Langdon & Poli, 2002), is one of EC’s most representative techniques. Its special feature is that it works with individuals that codify programs of non-fixed length (Michal, Ivry, Schalit-Cohen, Sipper, & Barash, 2007). Each individual in the GP population is a program that solves a given problem. A key drawback of GP is that it does not solve the closure problem (Koza, 1992): it can generate invalid individuals not belonging to the solution space during the evolutionary process. Processing these individuals increases the computational cost during the GP execution. In addition, the size of the GP individuals escapes control due to a phenomenon called code bloat (Panait & Luke, 2004).
Grammar-Guided Genetic Programming (GGGP) is a GP specialization aiming to solve the closure problem. It uses a Context-Free Grammar (CFG) to generate the language whose words are the whole set of individuals that codify a solution to a given problem (Whigham, 1995, Wong, 1995). Individuals are derivation trees of the CFG that, when the algorithm starts, are generated by a grammar-based initialization method. This method cannot generate invalid individuals because they are not contained in the language described by the CFG (O’Neil, 2003). The initialization method plays a very important role, as it influences the later convergence process of the evolutionary system (García-Arnau, Manrique, Ríos, & Rodríguez-Patón, 2007). The design of the crossover operator of a GGGP system is also responsible for solving the closure problem in such a way that the crossover of two valid individuals must generate a valid offspring. To avoid code bloat it is possible to set a parameter to control the size of the generated derivation trees in both the initialization process and the application of the crossover operator. One of the most representative operators is the Fair crossover. The Fair crossover solves this problem by controlling the size of the parts that the crossed individuals exchange (Crawford-Marks & Spector, 2002). Despite its simplicity, this control is very exhaustive and interferes with the operator’s exploration capability. Another important crossover operator is Whigham’s crossover (Whigham, 1995). This operator improves the exploration capability, as well as solving the closure problem. Even so, this operator still does not properly explore the search space, especially when working with ambiguous CFGs (Couchet et al., 2007, Hoai and McKay, 2002).
This paper presents an improved GGGP system. It includes both a grammar-based initialization method and a grammar-based crossover operator whose combined application boosts the search space exploration and exploitation capabilities. This system takes advantage of the grammar’s ambiguity, a property whereby different derivation trees represent the same sentence. This is a powerful and very efficient EC technique. The proposed system has been specifically designed to be able to construct and automatically self-adapt (robust) symbolic and sub-symbolic intelligent machines: rule- and fuzzy rule-based systems, and artificial neural networks. To do so, we have developed three different CFGs and codification systems. These automatically generated intelligent systems have been applied to a real-world problem located within the medical domain knowledge: the study of isokinetic time series for injury detection, diagnosis, rehabilitation and injury prevention (Couchet et al., 2008, Gioftsidou et al., 2008). All this is based on the analysis of isokinetic curves output by an isokinetic dynamometer on which patients exercise a muscle or joint.
Section snippets
Evolving intelligent systems
The proposed process for evolving intelligent systems is based on GGGP. Given a context-free grammar G, defined as a string-rewiring system comprising a 4-tuple G = (ΣN, ΣT, S, P)/ΣN ∩ ΣT = Ø, where ΣN is the alphabet of non-terminal symbols, ΣT is the alphabet of terminal symbols, S represents the start symbol or axiom of the grammar, and P is the set of production rules, written in Backus-Naur Form. The individuals that are part of the genetic population codify a sentence of the language generated
Constructing intelligent systems
The evolutionary techniques explained in the previous section have been designed to automatically generate three kinds of intelligent systems: rule-based systems (RBS), fuzzy neural networks (FNN) and artificial neural networks (ANN). For this purpose, it is necessary to establish a mechanism to codify these symbolic and sub-symbolic knowledge bases as derivation trees that can be managed by the proposed CFG-based intelligent system evolution process.
The example shown throughout the previous
Applying intelligent systems to injury detection
The proposed process for evolving intelligent systems has been executed to automatically generate RBSs, ANNs and FNNs that have been applied to the real-world problem of the detection of knee injuries. The data involved in the experiments have been extracted from the analysis of isokinetic curves, obtained from knee exercises performed by real patients. Isokinetic curves are continuous time series codifying the results of performing several repetitions of the same exercise. They are subjected
Conclusions
This paper proposed an evolutionary system for generating self-adapting intelligent machines. This system uses GGGP searching capabilities to automatically extract knowledge from a data source and generate trained symbolic and sub-symbolic intelligent systems with rule and fuzzy rule knowledge bases and artificial neural networks, respectively. The proposed evolutionary system has two main advantages. First it operates in a fully expert-independent manner. Second the generated intelligent
Acknowledgements
This research is located within the framework of the project no. DEP2005-00232-C03-03 funded by the Spanish Ministry of Science, Education and Sports. Our thanks also go to Africa López from the Spanish Higher Sports Council’s Center of Sports Medicine.
References (31)
- et al.
Combining expert knowledge and data mining in a medical diagnosis domain
Expert Systems with Applications
(2002) - et al.
Adaptive and intelligent web based education system: Towards an integral architecture and framework
Expert Systems with Applications
(2007) - et al.
A neural network-based approach for dynamic quality prediction in a plastic injection molding process
Expert Systems with Applications
(2008) - et al.
Initialization method for grammar-guided genetic programming
Knowledge-Based Systems
(2007) - et al.
KBSLUA: A knowledge-based system applied in river land use assessment
Expert Systems with Applications
(2008) - et al.
Evolutionary system for automatically constructing and adapting radial basis function networks
Neurocomputing
(2006) - et al.
A fuzzy inference system for fault detection and isolation: Application to a fluid system
Expert Systems with Applications
(2008) - Couchet, J., Font, J. M., & Manrique, D. (2008). Using evolved fuzzy neural networks for injury detection from...
- Couchet, J., Font, J. M., & Manrique, D. (2009). Rule evolving system for knee lesion prognosis from medical isokinetic...
- Couchet, J., Manrique, D., & Porras, L. (2007). Grammar-guided neural architecture evolution. In Bio-inspired Modeling...
Crossover and mutation operators for grammar-guided genetic programming
Soft Computing
Evolutionary computation: A unified approach
Cooperative coevolution of artificial neural network ensembles for patterns classification
IEEE Transactions on Evolutionary Computation
Isokinetic strength training program for muscular imbalances in professional soccer players
Sport Sciences for Health
Cited by (14)
Knowledge transfer for labeling unknown fuzzy sets using Grammar-Guided Genetic Algorithms
2022, Applied Soft ComputingClassification of tea specimens using novel hybrid artificial intelligence methods
2014, Sensors and Actuators, B: ChemicalCitation Excerpt :In many cases one can achieve much higher performance while the complexity of the system is only slightly increased [3]. The most popular hybrid systems include the evolutionary-neural [4–9], evolutionary-fuzzy [10–14] and neuro-fuzzy systems [15–18]. Although, these systems are well known and used for many years, the subject is still actual and in particular is used for the analysis of non-linear multidimensional gas sensor signals, i.e. from electronic noses.
Bacterially inspired evolving system with an application to time series prediction
2013, Applied Soft Computing JournalCitation Excerpt :Communication is done by producing and sensing acyl homoserine lactones (AHL) signal molecules, also called autoinducers. This bacterial evolutionary system is a general-purpose technique, because grammar-guided genetic programming (GGGP) assures that any intelligent system whose constraints can be codified using a context-free grammar can implement the biological processes of these bacteria [27]. The bacterial evolutionary system evolves variable-size individuals and, given that GGGP solves the closure problem [38], it is complete, because it includes initialization, selection and replacement operations – via cellular division and death –, as well as a conjugation operator that implements a grammatical crossover [39] by means of which bacteria share genetic material via conjugation [40].
Partition based real-valued encoding scheme for evolutionary algorithms
2016, Natural ComputingApproximation of phenol concentration using novel hybrid computational intelligence methods
2014, International Journal of Applied Mathematics and Computer Science