Elsevier

Pattern Recognition Letters

Volume 28, Issue 16, 1 December 2007, Pages 2385-2400
Pattern Recognition Letters

Generative learning of visual concepts using multiobjective genetic programming

https://doi.org/10.1016/j.patrec.2007.08.001Get rights and content

Abstract

This paper introduces a novel method of visual learning based on genetic programming, which evolves a population of individuals (image analysis programs) that process attributed visual primitives derived from raw raster images. The goal is to evolve an image analysis program that correctly recognizes the training concept (shape). The approach uses generative evaluation scheme: individuals are rewarded for reproducing the shape of the object being recognized using graphical primitives and elementary background knowledge encoded in predefined operators. Evolutionary run is driven by a multiobjective fitness function to prevent premature convergence and enable effective exploration of the space of solutions. We present the method in detail and verify it experimentally on the task of learning two visual concepts from examples.

Introduction

Visual learning seems to be the most promising way of building scalable and adaptive image analysis systems. Unfortunately, learning in computer vision is usually limited to parameter optimization that concerns only a particular processing step, such as preprocessing, segmentation, feature extraction, etc. Reports on methods that synthesize complete object recognition systems starting from raw image data are rare. Most algorithms are also application-specific, which makes the acquired knowledge difficult to transfer to other applications.

The most popular way of equipping a vision system with learning capability consists in introducing an off-shelf machine learning (ML) algorithm into the chain of image processing, analysis, and interpretation. Though usually straightforward, this approach implies serious simplifications in terms of representation of input data (commonly a fixed-length vector of image features) and the expected output (discrete, nominal decisions). Also, given the large number of features that can be derived from the input image, and consequently high dimensionality of the input space (when compared to non-vision ML applications), the risk of overfitting becomes grave, unless human intervention constrains the search by, e.g., pre-selecting only a handful of the most promising features.

In this paper, we hypothesize that visual learning may benefit from a novel way of assessing learner’s ability to recognize (interpret) an input image. The proposed assessment method is more thorough than in conventional ML as, in a sense, it forces the learner to prove its ‘understanding’ of the input image. Technically, learners are encoded as a genetic programming (GP) individuals (Koza, 1994), i.e., as expression trees built of elementary operators that dwell in a population maintained by an evolutionary algorithm (Goldberg, 1989, Michalewicz, 1994). Each learner processes, analyzes, and interprets information given in a form of visual primitives (VPs) that represent local salient features derived from the input raster image. When exposed to an input image, the learner produces in response a simplified sketch of that image. An evolutionary fitness function examines the sketch, using multiple objectives to assess its different aspects, and appropriately rewards the individual. In such a way, the evolutionary process promotes individuals that provide best interpretations of the input image, in the sense detailed further in the paper.

Therefore, the primary contribution of this paper is an approach to image interpretation and object recognition that (i) guides visual learning by estimating learner’s ability to reproduce the input image, (ii) engages multiple objectives for learner’s evaluation (Section 4.3), (iii) uses visual primitives as basic ‘granules’ of information (see Section 4.1), and (iv) relies on evolutionary computation (GP in particular) to effectively search the hypothesis space.

The following Sections 2 Motivations, 3 Related research in visual learning detail our motivations and summarize the related work. In Section 4, we thoroughly describe our approach. Section 5 demonstrates the performance of the approach on a visual task of acquiring two visual concepts. In Section 6, we provide summary and draw conclusions for further research.

Section snippets

Motivations

Any machine learning algorithm requires guidance when searching the space of hypotheses (identified with learners and individuals in this paper) (Michalski and Tecuci, 1994, Langley, 1996). In supervised learning, this guidance is usually driven by the quality of discrimination of decision classes, technically expressed as classification accuracy, sensitivity, selectivity, or a similar measure. This approach is characteristic for, among others, the ‘wrapper’ approach to feature selection and

Related research in visual learning

In most approaches to visual learning reported in literature, learning is limited to parameter optimization and usually concerns only a particular step, such as image preprocessing, segmentation, or feature extraction. Only a few methods close the feedback loop of the learning process at the outermost (e.g., recognition) level (Draper et al., 1993, Johnson et al., 1994, Segen, 1994, Teller and Veloso, 1997, Luke, 2002, Rizki et al., 2002, Maloof et al., 2003, Torralba et al., 2004, Krawiec and

Visual learning driven by image reproduction

The proposed approach may be shortly characterized as generative visual learning, as our evolving learners try to reproduce the input image and are rewarded according to the quality of that reproduction. In that process, learners focus on a particular aspect of visual information, which is shape in this study. Other factors, like color, texture, shading, etc., are ignored.

Image reproduction takes place on a virtual canvas spanned over of the input image. On that canvas, a learner is allowed to

Experiment objectives and training data

In this part we use the proposed approach to recognize triangles and sections. Though straightforward for humans, these tasks are nontrivial, as learner’s only input is a set P of a few dozens of VPs, each of them described by coordinates px, py and gradient orientation po. Learners have no a priori information on, e.g., spatial proximity of VPs, their collinear alignment, etc. The VPs located next to triangle vertices are not marked as special in any way; their importance has to be discovered

Conclusions

The proposed learning method successfully evolves image analysis procedures that are able to interpret compound geometrical patterns using very limited background knowledge. Generative aspect of the approach, implemented by means of drawing actions, enables in-depth evaluation of learner’s understanding of the processed pattern. As demonstrated by exemplary solution presented in Section 5.6, the method is able to autonomously decompose a complex recognition task into subtasks. This feature

References (28)

  • J. Koza

    Genetic Programming

    (1992)
  • J. Koza

    Genetic programming – 2

    (1994)
  • J. Koza

    Human-competitive applications of genetic programming

  • K. Krawiec

    Genetic programming-based construction of features for machine learning and knowledge discovery tasks

    Genet. Program. Evolvable Machine

    (2002)
  • Cited by (20)

    • A filter-based feature construction and feature selection approach for classification using Genetic Programming

      2020, Knowledge-Based Systems
      Citation Excerpt :

      Ahmed et al. (2014) [22] divided the best individual to all possible sub-trees that were transformed into multiple features. Moreover, cooperative coevolution strategy [23–26] that created multiple cocurrent populations was used to construct multiple features. To facilitate comparison, both feature selection and feature construction approaches use standard GP.

    • A hybrid multiple feature construction approach for classification using Genetic Programming

      2019, Applied Soft Computing Journal
      Citation Excerpt :

      In general, each individual constructs only one single feature [21]. However, researchers [5,17,22] try to find ways to construct multiple features to improve the classification ability of the learning algorithm. One main approach to multiple feature construction is to use cooperative coevolution strategy [22–25].

    • Interest point detection through multiobjective genetic programming

      2012, Applied Soft Computing Journal
      Citation Excerpt :

      For instance, in [30] a MO genetic algorithm is used to solve the sensor planning problem for vision metrology. Other examples include [31] where MO-GP is used to automatically learn simple visual concepts, and [32] where MO-GP is used to control program size for a feature extraction problem. More recently, in [15] we present a hybrid computer assisted design approach to develop a feature detection algorithm using genetic programming.

    • Software Testing Automation: Testability Evaluation, Refactoring, Test Data Generation and Fault Localization

      2023, Software Testing Automation: Testability Evaluation, Refactoring, Test Data Generation and Fault Localization
    View all citing articles on Scopus

    This work has been supported by grant N N519 3505 33.

    View full text