Improving the learning of Boolean queries by means of a multiobjective IQBE evolutionary algorithm

Abstract

The Inductive Query By Example (IQBE) paradigm allows a system to automatically derive queries for a specific Information Retrieval System (IRS). Classic IRSs based on this paradigm [Smith, M., & Smith, M. (1997). The use of genetic programming to build Boolean queries for text retrieval through relevance feedback. Journal of Information Science, 23(6), 423–431] generate a single solution (Boolean query) in each run, that with the best fitness value, which is usually based on a weighted combination of the basic performance criteria, precision and recall.

A desirable aspect of IRSs, especially of those based on the IQBE paradigm, is to be able to get more than one query for the same information needs, with high precision arid recall values or with different trade-offs between both.

In this contribution, a new IQBE process is proposed combining a previous basic algorithm to automatically derive Boolean queries for Boolean IRSs [Smith, M., & Smith, M. (1997). The use of genetic programming to build Boolean queries for text retrieval through relevance feedback. Journal of Information Science, 23(6), 423–431] and an advanced evolutionary multiobjective approach [Coello, C. A., Van Veldhuizen, D. A., & Lamant, G. B. (2002). Evolutionary algorithms for solving multiobjective problems. Kluwer Academic Publishers], which obtains several queries with a different precision–recall trade-off in a single run. The performance of the new proposal will be tested on the Cranfield and CACM collections and compared to the well-known Smith and Smith’s algorithm, showing how it improves the learning of queries and thus it could better assist the user in the query formulation process.

Introduction

Information retrieval (IR) may be defined, in general, as the problem of the selection of documentary information from storage in response to search questions provided by a user (Baeza-Yates and Ribeiro-Neto, 1999, Salton and McGill, 1983). Information retrieval systems (IRSs) are a kind of information systems that deal with data bases composed of information items—documents that may consist of textual, pictorial or vocal information—and process user queries trying to allow the user to access to relevant information in an appropriate time interval. Nowadays, the development of the WWW has increased the interest on the study of IRSs.

Many IRSs still consider the Boolean IR model (Van Rijsbergen, 1979), based on the use of Boolean queries where the query terms are joined by the logical operators AND and OR. This way, the user needs to have a clear knowledge on how to connect the query terms together using the Boolean operators in order to build a query defining his information needs. The difficulty found by nonexpert users to formulate these kinds of queries sometimes makes necessary the design of automatic methods for this task. The paradigm of Inductive Query by Example (IQBE) (Chen, Shankaranarayanan, She, & lyer, 1998), where a query describing the information contents of a set of documents provided by a user is automatically derived, can be useful to assist the user in the query formulation process. Focusing on the Boolean IR model, the most known existing approach is that of Smith and Smith (1997), which is based on a kind of evolutionary algorithm (EA) (Bäck, Fogel, & Michalewicz, 1997), genetic programming (GP) (Koza, 1992). As usual in the topic (Cordón, Herrera-Viedma, López-Pujalte, Luque, & Zarco, 2003), this approach is guided by a weighted fitness function combining two retrieval accuracy criteria, precision and recall. The main characteristic of this approach is that it provides a single query in each run.

Given the retrieval performance of an IRS is usually measured in terms of these two criteria, precision and recall (Van Rijsbergen, 1979), the optimization of any of its components, and concretely the automatic learning of Boolean queries, is thus a clear example of a multiobjective problem. EAs have been commonly used for IQBE purposes and their application in the area has been usually based on combining both criteria in a single scalar fitness function by means of a weighting scheme (Cordón, Herrera-Viedma, López-Pujalte, et al., 2003). However, there is a kind of EA specially designed for multiobjective problems, multiobjective evolutionary algorithms, which are able to obtain different nondominated solutions to the problem in a single run (Coello et al., 2002, Deb, 2001). In IR, specifically in the IQBE paradigm, they would allow us to derive a number of queries with a different precision–recall trade-off in a single run of the IQBE algorithm, and in such a way to improve the aid possibilities to the users in the formulation of their queries.

In this paper, we present a new evolutionary tool to learn Boolean queries that improves the Smith and Smith’s (1997) approach, called multiobjective IQBE EA. We define it by extending the Smith and Smith’s approach incorporating Pareto-based evolutionary multiobjective components into GP. To do so, we consider one of the most known and well performing Pareto-based multiobjective EAs, SPEA (Zitzler & Thiele, 1999). The main feature of this EA is the maintenance of the elitism concept in a multiobjective evolutionary algorithm. This improves the performance of our multiobjective GP algorithm. In order to represent a real-world text retrieval IQBE environment where a user provides a relatively small number of relevant and irrelevant documents, the experimental testbed will be based on two of the most known small size IR benchmarks, the Cranfield and CACM document collections (Baeza-Yates and Ribeiro-Neto, 1999, Salton and McGill, 1983). With our proposal we improve and increase the user assistance possibilities in the formulation of queries by means of evolutionary computation tools.

With this aim, this contribution is structured as follows. Section 2 is devoted to introduce the preliminaries, including the basis of Boolean IRSs, the definition of both precision and recall criteria, the main aspects of IQBE techniques, a review on EAs and on their application to IR, tasks, arid finally, the main aspects of multiobjective EAs. Section 3 is devoted to introduce the main aspects of the Smith and Smith’s proposal and to extend the latter algorithm to deal with the multiobjective problem of simultaneously optimizing both precision and recall by means of the SPEA Pareto-based approach while the experiments developed to test the new proposal and the results obtained are shown in Sections 4 Experiments developed, 5 Results and analysis of results, respectively. Finally, several concluding remarks are pointed out in Section 6.