Elsevier

Pattern Recognition Letters

Volume 23, Issue 12, October 2002, Pages 1439-1448
Pattern Recognition Letters

Character preclassification based on genetic programming

https://doi.org/10.1016/S0167-8655(02)00104-6Get rights and content

Abstract

This paper presents a learning system that uses genetic programming as a tool for automatically inferring the set of classification rules to be used during a preclassification stage by a hierarchical handwritten character recognition system. Starting from a structural description of the character shape, the aim of the learning system is that of producing a set of classification rules able to capture the similarities among those shapes, independently of whether they represent characters belonging to the same class or to different ones. In particular, the paper illustrates the structure of the classification rules, the grammar used to generate them and the genetic operators devised to manipulate the set of rules, as well as the fitness function used to drive the inference process. The experimental results obtained by using a set of 10,000 digits extracted from the NIST database show that the proposed preclassification is efficient and accurate, because it provides at most 6 classes for more than 87% of the samples, and the error rate almost equals the intrinsic confusion found in the data set.

Introduction

The recognition of handwritten characters involves identifying a correspondence between the pixels of the image representing the sample to be recognized and the abstract definitions of characters (models or prototypes). The prevalent approach to solve the problem is that of implementing a bottom–up process for extracting and combining many pieces of information (features) that are eventually used to build up the models for each class. During the classification, such models are compared with the input sample according to the adopted classification method for deciding to which class the sample belongs. Due to the extreme variability exhibited by samples produced by a large population of writers, pursuing such an approach often requires the use of a large number of prototypes for each class, in order to capture the distinctive features of different writing styles, and rather complex classification algorithms. The combination of complex classification methods with a large set of prototypes has a dramatic effect on the classifier: a larger number of prototypes requires a higher discriminating power, which, in turn, requires more sophisticated methods and algorithms to perform the classification.

For this reason, we have investigated the possibility of using a preclassification technique whose main purpose is that of reducing the number of classes to be considered when classifying an input sample. Such a reduction affects both the performance of the classifier and its computational cost. As with respect to the performance, lowering the number of classes leads to a simpler classification problem, thus reducing the risk of confusing the right class with a wrong one. The reduction of the computational cost follows immediately from the reduction of the number of prototypes, although it is counterbalanced by the cost involved by the preclassification itself. The general problem of reducing a classifier computational cost has been faced since the 70's in the framework of statistical pattern recognition (Fukunaga and Narendra, 1975) and, more recently, within shape-based methods for character recognition (Marcelli and Pavlidis, 1994; Marcelli et al., 1997). The large majority of the preclassification methods for character recognition proposed in the literature belongs to one of two categories, depending on whether they adopt a different set of features or a different classification strategy, with respect to those adopted during the classification (Mori et al., 1984).

In this paper, developed within the framework of an evolutionary approach to character recognition (De Stefano et al., 1999), we present a novel preclassification method that uses genetic programming (GP) as a tool for learning a set of classification rules (prototypes) that describe specimen belonging to one or more actual classes. To this purpose, starting from a graph-based representation of the character shape, we compute a simpler description in terms of a feature vector. Such a vector is composed of as many elements as the types of primitives and of spatial relationships among them that can be used to describe a sample, and each element of the feature vector counts the occurrence of each feature found in the sample. Consequently, a prototype is represented by a set of assertions connected by Boolean operators, each assertion specifying the constraints on the occurrence of a feature. The proposed preclassifier works in two different modes. During an off-line unsupervised training phase, an initial population of randomly generated prototypes is evolved according to the GP paradigm in order to produce the set of prototypes that achieves the maximum coverage of the training set. At the end of the training, to each prototype is associated the list of the classes covered by that prototype. In such a list are included all the classes whose samples satisfy the constraints expressed by the prototype. At run time, after the feature extraction, the feature vector is computed for a given sample and matched against the prototypes. The desired preclassification is thus obtained by assigning to the sample the list of the classes associated to the simplest matching prototype.

Let us remark that, due to the discrete nature of our feature space and to the meaning of the features, the distance between the points in this space does not necessarily reflect the shape similarity. Therefore, none of the methods based on distance evaluation, such as principal component analysis, K-means clustering and so on, can be used. An approach similar to the one followed by us has been very recently proposed (Teredesai et al., 2001). In that study, developed on the basis of a multi-level feature extraction method, GP is aimed at selecting the optimal set of features, i.e. the set of features able to maximize the separability among the classes to be recognized. Our method, on the contrary, uses GP at a coarser classification level, to capture the similarities among character shapes, independently of the classes the characters belong to.

The paper is organized as follows. Section 2 illustrates the character shape description scheme and how it is reduced to a feature vector. In Section 3 we present our approach and its implementation, while Section 4 reports the experimental results. Concluding remarks are eventually left to Section 5.

Section snippets

From character shape to feature vector

In the framework of structural methods for pattern recognition, the most common approach is based on the decomposition of an initial representation of the sample into elementary parts, each of which can be described in a simple manner. In this way a character is described in terms of a set of parts interrelated by more or less complex links. Such a structure is then described by a sentence of a language or by a relational graph. Accordingly, the classification is performed by parsing the

Learning explicit classification rules

As mentioned in the introduction, the prototypes to be used for the preclassification are given in terms of classification rules. Since classification rules may be thought of as computer programs, a natural way for introducing them into our learning system is that of adopting the GP paradigm (Koza, 1992, Koza, 1994). Such a paradigm combines genetic algorithms and programming languages in order to evolve computer programs of dynamically varying complexity (size and shape) according to a given

Experimental results

The ability of GP to generate classification rules in very complex cases, like the one at hand, and the preclassifier performance in terms of both efficiency and accuracy have been evaluated through a large set of experiments. The experiments were performed on a data set of 10,000 digits extracted from the NIST database and equally distributed among the 10 classes. This data set was randomly subdivided into a training and a test set, both including 5,000 samples. Each character was decomposed,

Conclusions

In this paper we have presented a novel preclassification method that uses GP as a tool for learning a set of prototypes able to capture the similarities among character shapes even if the characters belong to different classes. The proposed method adopts a fine-to-coarse description scheme, whose bottom level is a graph-based representation of the character shape, while the top level is a feature vector composed of as many elements as the types of arcs and of spatial relationships that can be

References (18)

  • L.B. Booker et al.

    Classifier systems and genetic algorithms

    Artificial Intell.

    (1989)
  • S.W. Mahfoud

    Populations size and genetic drift in fitness sharing

  • A. Marcelli et al.

    Structural indexing for character recognition

    Comput. Vis. Image Understanding: CVIU

    (1997)
  • J.E. Baker

    Reducing bias and inefficiency in the selection algorithm

  • A. Chianese et al.

    A structural method for handprinted character recognition

    Lecture Notes Comput. Sci.

    (1989)
  • M. Conte et al.

    Genetic programming estimates of Kolmogorov complexity

  • L.P. Cordella et al.

    Neural network classifier for OCR using structural descriptions

    Machine Vis. Appl.

    (1995)
  • K. Deb et al.

    An investigation of niche and species formation in genetic function optimization

  • De Stefano, C., Della Cioppa, A., Marcelli, A., 1999. Handwritten numerals recognition by means of evolutionary...
There are more references available in the full text version of this article.

Cited by (0)

View full text