An adaptive knowledge-acquisition system using generic genetic programming

Abstract

The knowledge-acquisition bottleneck greatly obstructs the development of knowledge-based systems. One popular approach to knowledge acquisition uses inductive concept learning to derive knowledge from examples stored in databases. However, existing learning systems cannot improve themselves automatically. This paper describes an adaptive knowledge-acquisition system that can learn first-order logical relations and improve itself automatically. The system is composed of an external interface, a biases base, a knowledge base of background knowledge, an example database, an empirical ILP learner, a meta-level learner, and a learning controller. In this system, the empirical ILP learner performs top-down search in the hypothesis space defined by the concept description language, the language bias, and the background knowledge. The search is directed by search biases which can be induced and refined by the meta-level learner based on generic genetic programming.

It has been demonstrated that the adaptive knowledge-acquisition system performs better than FOIL on inducing logical relations from perfect or noisy training examples. The result implies that the search bias evolved by evolutionary learning is better than that of FOIL which is designed by a top researcher in the field. Consequently, generic genetic programming is a promising technique for implementing a meta-level learning system. The result is very encouraging as it suggests that the process of natural selection and evolution can successfully evolve a high-performance learning system.

Introduction

The knowledge-acquisition bottleneck greatly obstructs the development of knowledge-based systems. One popular approach to knowledge acquisition uses inductive concept learning to derive knowledge from examples stored in databases. The knowledge acquired can be expressed in different knowledge representations such as first-order logical relations, decision trees, decision lists, and production rules. Existing learning systems such as CART (Breiman et al., 1984), C4.5 (Quinlan, 1992), ASSISTANT (Cestnik et al., 1987), AQ15 (Michalski et al., 1986), and CN2 (Clark and Niblett, 1989) use attribute-value language for representing the training examples and the induced knowledge and allow a finite number of objects in the universe of discourse. This representation limits them to learn only propositional descriptions in which concepts are described in terms of values of a fixed number of attributes.

Dzeroski and Lavrac show that Inductive Logic Programming (ILP) can be used to induce knowledge represented as first-order logical relations (Dzeroski and Lavrac, 1993; Dzeroski, 1996). ILP is more powerful than traditional inductive learning methods because it uses an expressive first-order logic framework and facilitates the application of background knowledge. In this formalism, domain knowledge represented in the form of relations can be used in the induced relational descriptions of concepts. Moreover, ILP has a strong theoretical foundation from logic programming and computational learning theory.

The task of inducing first-order logical relations can be formulated as a search problem (Mitchell, 1982) in a hypotheses space of logical relations. Various approaches (Quinlan, 1990; Muggleton and Feng, 1990) differ mainly in the search strategy and the heuristics used to guide the search. The search space is extremely large, so strong heuristics are required to manage the problem. Most systems are based on a greedy search strategy. They generate a sequence of logical relations from general to specific (or from specific to general) until a consistent relation is found. Each relation in the sequence is obtained by specializing (or generalizing) the previous one. For example, FOIL (Quinlan, 1990) applies a hill-climbing search strategy guided by an information-gain heuristic to search relations from general to specific. But these strategies and heuristics are not always applicable because these systems may become trapped in local maxima. In order to overcome this problem, non-greedy strategies should be adopted. Moreover, existing ILP systems cannot improve themselves automatically.

In this paper, we describe an adaptive knowledge-acquisition system that induces first-order logical relations and improves itself during learning. We formulate the definitions of inductive concept learning and adaptive knowledge acquisition in the next section. The system is based on a generic genetic programming approach that is presented in Section 3. A generic top-down first-order learning algorithm is described in Section 4. Section 5contains a description of a meta-level learner that induces search bias. The experimentation and some evaluations of the system are reported in Section 6. Finally, conclusions are presented in Section 7.