Character preclassification based on genetic programming
Introduction
The recognition of handwritten characters involves identifying a correspondence between the pixels of the image representing the sample to be recognized and the abstract definitions of characters (models or prototypes). The prevalent approach to solve the problem is that of implementing a bottom–up process for extracting and combining many pieces of information (features) that are eventually used to build up the models for each class. During the classification, such models are compared with the input sample according to the adopted classification method for deciding to which class the sample belongs. Due to the extreme variability exhibited by samples produced by a large population of writers, pursuing such an approach often requires the use of a large number of prototypes for each class, in order to capture the distinctive features of different writing styles, and rather complex classification algorithms. The combination of complex classification methods with a large set of prototypes has a dramatic effect on the classifier: a larger number of prototypes requires a higher discriminating power, which, in turn, requires more sophisticated methods and algorithms to perform the classification.
For this reason, we have investigated the possibility of using a preclassification technique whose main purpose is that of reducing the number of classes to be considered when classifying an input sample. Such a reduction affects both the performance of the classifier and its computational cost. As with respect to the performance, lowering the number of classes leads to a simpler classification problem, thus reducing the risk of confusing the right class with a wrong one. The reduction of the computational cost follows immediately from the reduction of the number of prototypes, although it is counterbalanced by the cost involved by the preclassification itself. The general problem of reducing a classifier computational cost has been faced since the 70's in the framework of statistical pattern recognition (Fukunaga and Narendra, 1975) and, more recently, within shape-based methods for character recognition (Marcelli and Pavlidis, 1994; Marcelli et al., 1997). The large majority of the preclassification methods for character recognition proposed in the literature belongs to one of two categories, depending on whether they adopt a different set of features or a different classification strategy, with respect to those adopted during the classification (Mori et al., 1984).
In this paper, developed within the framework of an evolutionary approach to character recognition (De Stefano et al., 1999), we present a novel preclassification method that uses genetic programming (GP) as a tool for learning a set of classification rules (prototypes) that describe specimen belonging to one or more actual classes. To this purpose, starting from a graph-based representation of the character shape, we compute a simpler description in terms of a feature vector. Such a vector is composed of as many elements as the types of primitives and of spatial relationships among them that can be used to describe a sample, and each element of the feature vector counts the occurrence of each feature found in the sample. Consequently, a prototype is represented by a set of assertions connected by Boolean operators, each assertion specifying the constraints on the occurrence of a feature. The proposed preclassifier works in two different modes. During an off-line unsupervised training phase, an initial population of randomly generated prototypes is evolved according to the GP paradigm in order to produce the set of prototypes that achieves the maximum coverage of the training set. At the end of the training, to each prototype is associated the list of the classes covered by that prototype. In such a list are included all the classes whose samples satisfy the constraints expressed by the prototype. At run time, after the feature extraction, the feature vector is computed for a given sample and matched against the prototypes. The desired preclassification is thus obtained by assigning to the sample the list of the classes associated to the simplest matching prototype.
Let us remark that, due to the discrete nature of our feature space and to the meaning of the features, the distance between the points in this space does not necessarily reflect the shape similarity. Therefore, none of the methods based on distance evaluation, such as principal component analysis, K-means clustering and so on, can be used. An approach similar to the one followed by us has been very recently proposed (Teredesai et al., 2001). In that study, developed on the basis of a multi-level feature extraction method, GP is aimed at selecting the optimal set of features, i.e. the set of features able to maximize the separability among the classes to be recognized. Our method, on the contrary, uses GP at a coarser classification level, to capture the similarities among character shapes, independently of the classes the characters belong to.
The paper is organized as follows. Section 2 illustrates the character shape description scheme and how it is reduced to a feature vector. In Section 3 we present our approach and its implementation, while Section 4 reports the experimental results. Concluding remarks are eventually left to Section 5.
Section snippets
From character shape to feature vector
In the framework of structural methods for pattern recognition, the most common approach is based on the decomposition of an initial representation of the sample into elementary parts, each of which can be described in a simple manner. In this way a character is described in terms of a set of parts interrelated by more or less complex links. Such a structure is then described by a sentence of a language or by a relational graph. Accordingly, the classification is performed by parsing the
Learning explicit classification rules
As mentioned in the introduction, the prototypes to be used for the preclassification are given in terms of classification rules. Since classification rules may be thought of as computer programs, a natural way for introducing them into our learning system is that of adopting the GP paradigm (Koza, 1992, Koza, 1994). Such a paradigm combines genetic algorithms and programming languages in order to evolve computer programs of dynamically varying complexity (size and shape) according to a given
Experimental results
The ability of GP to generate classification rules in very complex cases, like the one at hand, and the preclassifier performance in terms of both efficiency and accuracy have been evaluated through a large set of experiments. The experiments were performed on a data set of 10,000 digits extracted from the NIST database and equally distributed among the 10 classes. This data set was randomly subdivided into a training and a test set, both including 5,000 samples. Each character was decomposed,
Conclusions
In this paper we have presented a novel preclassification method that uses GP as a tool for learning a set of prototypes able to capture the similarities among character shapes even if the characters belong to different classes. The proposed method adopts a fine-to-coarse description scheme, whose bottom level is a graph-based representation of the character shape, while the top level is a feature vector composed of as many elements as the types of arcs and of spatial relationships that can be
References (18)
- et al.
Classifier systems and genetic algorithms
Artificial Intell.
(1989) Populations size and genetic drift in fitness sharing
- et al.
Structural indexing for character recognition
Comput. Vis. Image Understanding: CVIU
(1997) Reducing bias and inefficiency in the selection algorithm
- et al.
A structural method for handprinted character recognition
Lecture Notes Comput. Sci.
(1989) - et al.
Genetic programming estimates of Kolmogorov complexity
- et al.
Neural network classifier for OCR using structural descriptions
Machine Vis. Appl.
(1995) - et al.
An investigation of niche and species formation in genetic function optimization
- De Stefano, C., Della Cioppa, A., Marcelli, A., 1999. Handwritten numerals recognition by means of evolutionary...