Generative learning of visual concepts using multiobjective genetic programming☆
Introduction
Visual learning seems to be the most promising way of building scalable and adaptive image analysis systems. Unfortunately, learning in computer vision is usually limited to parameter optimization that concerns only a particular processing step, such as preprocessing, segmentation, feature extraction, etc. Reports on methods that synthesize complete object recognition systems starting from raw image data are rare. Most algorithms are also application-specific, which makes the acquired knowledge difficult to transfer to other applications.
The most popular way of equipping a vision system with learning capability consists in introducing an off-shelf machine learning (ML) algorithm into the chain of image processing, analysis, and interpretation. Though usually straightforward, this approach implies serious simplifications in terms of representation of input data (commonly a fixed-length vector of image features) and the expected output (discrete, nominal decisions). Also, given the large number of features that can be derived from the input image, and consequently high dimensionality of the input space (when compared to non-vision ML applications), the risk of overfitting becomes grave, unless human intervention constrains the search by, e.g., pre-selecting only a handful of the most promising features.
In this paper, we hypothesize that visual learning may benefit from a novel way of assessing learner’s ability to recognize (interpret) an input image. The proposed assessment method is more thorough than in conventional ML as, in a sense, it forces the learner to prove its ‘understanding’ of the input image. Technically, learners are encoded as a genetic programming (GP) individuals (Koza, 1994), i.e., as expression trees built of elementary operators that dwell in a population maintained by an evolutionary algorithm (Goldberg, 1989, Michalewicz, 1994). Each learner processes, analyzes, and interprets information given in a form of visual primitives (VPs) that represent local salient features derived from the input raster image. When exposed to an input image, the learner produces in response a simplified sketch of that image. An evolutionary fitness function examines the sketch, using multiple objectives to assess its different aspects, and appropriately rewards the individual. In such a way, the evolutionary process promotes individuals that provide best interpretations of the input image, in the sense detailed further in the paper.
Therefore, the primary contribution of this paper is an approach to image interpretation and object recognition that (i) guides visual learning by estimating learner’s ability to reproduce the input image, (ii) engages multiple objectives for learner’s evaluation (Section 4.3), (iii) uses visual primitives as basic ‘granules’ of information (see Section 4.1), and (iv) relies on evolutionary computation (GP in particular) to effectively search the hypothesis space.
The following Sections 2 Motivations, 3 Related research in visual learning detail our motivations and summarize the related work. In Section 4, we thoroughly describe our approach. Section 5 demonstrates the performance of the approach on a visual task of acquiring two visual concepts. In Section 6, we provide summary and draw conclusions for further research.
Section snippets
Motivations
Any machine learning algorithm requires guidance when searching the space of hypotheses (identified with learners and individuals in this paper) (Michalski and Tecuci, 1994, Langley, 1996). In supervised learning, this guidance is usually driven by the quality of discrimination of decision classes, technically expressed as classification accuracy, sensitivity, selectivity, or a similar measure. This approach is characteristic for, among others, the ‘wrapper’ approach to feature selection and
Related research in visual learning
In most approaches to visual learning reported in literature, learning is limited to parameter optimization and usually concerns only a particular step, such as image preprocessing, segmentation, or feature extraction. Only a few methods close the feedback loop of the learning process at the outermost (e.g., recognition) level (Draper et al., 1993, Johnson et al., 1994, Segen, 1994, Teller and Veloso, 1997, Luke, 2002, Rizki et al., 2002, Maloof et al., 2003, Torralba et al., 2004, Krawiec and
Visual learning driven by image reproduction
The proposed approach may be shortly characterized as generative visual learning, as our evolving learners try to reproduce the input image and are rewarded according to the quality of that reproduction. In that process, learners focus on a particular aspect of visual information, which is shape in this study. Other factors, like color, texture, shading, etc., are ignored.
Image reproduction takes place on a virtual canvas spanned over of the input image. On that canvas, a learner is allowed to
Experiment objectives and training data
In this part we use the proposed approach to recognize triangles and sections. Though straightforward for humans, these tasks are nontrivial, as learner’s only input is a set P of a few dozens of VPs, each of them described by coordinates px, py and gradient orientation po. Learners have no a priori information on, e.g., spatial proximity of VPs, their collinear alignment, etc. The VPs located next to triangle vertices are not marked as special in any way; their importance has to be discovered
Conclusions
The proposed learning method successfully evolves image analysis procedures that are able to interpret compound geometrical patterns using very limited background knowledge. Generative aspect of the approach, implemented by means of drawing actions, enables in-depth evaluation of learner’s understanding of the processed pattern. As demonstrated by exemplary solution presented in Section 5.6, the method is able to autonomously decompose a complex recognition task into subtasks. This feature
References (28)
- et al.
Pragmatic genetic programming strategy for the problem of vehicle detection in airborne reconnaissance
Pattern Recognition Lett.
(2006) - et al.
Wrappers for feature subset selection
Artif. Intell. J.
(1997) - et al.
Evolutionary feature synthesis for facial expression recognition
Pattern Recognition Lett.
(2006) - et al.
Genetic Programming: An Introduction
(1998) - et al.
Evolutionary Synthesis of Pattern Recognition Systems
(2005) - et al.
Learning blackboard-based scheduling algorithms for computer vision
Int. J. Pattern Recognition Artif. Intell.
(1993) Genetic Algorithms in Search, Optimization and Machine Learning
(1989)- et al.
Genetic programming for cross-task knowledge sharing
- Java advanced imaging API specification, 2001. version 1.2, Tech....
- et al.
Evolving visual routines
Genetic Programming
Genetic programming – 2
Human-competitive applications of genetic programming
Genetic programming-based construction of features for machine learning and knowledge discovery tasks
Genet. Program. Evolvable Machine
Cited by (20)
Multi-generation multi-criteria feature construction using Genetic Programming
2023, Swarm and Evolutionary ComputationA filter-based feature construction and feature selection approach for classification using Genetic Programming
2020, Knowledge-Based SystemsCitation Excerpt :Ahmed et al. (2014) [22] divided the best individual to all possible sub-trees that were transformed into multiple features. Moreover, cooperative coevolution strategy [23–26] that created multiple cocurrent populations was used to construct multiple features. To facilitate comparison, both feature selection and feature construction approaches use standard GP.
A hybrid multiple feature construction approach for classification using Genetic Programming
2019, Applied Soft Computing JournalCitation Excerpt :In general, each individual constructs only one single feature [21]. However, researchers [5,17,22] try to find ways to construct multiple features to improve the classification ability of the learning algorithm. One main approach to multiple feature construction is to use cooperative coevolution strategy [22–25].
Interest point detection through multiobjective genetic programming
2012, Applied Soft Computing JournalCitation Excerpt :For instance, in [30] a MO genetic algorithm is used to solve the sensor planning problem for vision metrology. Other examples include [31] where MO-GP is used to automatically learn simple visual concepts, and [32] where MO-GP is used to control program size for a feature extraction problem. More recently, in [15] we present a hybrid computer assisted design approach to develop a feature detection algorithm using genetic programming.
Software Testing Automation: Testability Evaluation, Refactoring, Test Data Generation and Fault Localization
2023, Software Testing Automation: Testability Evaluation, Refactoring, Test Data Generation and Fault LocalizationChoosing mutation and crossover ratios for genetic algorithms-a review with a new dynamic approach
2019, Information (Switzerland)
- ☆
This work has been supported by grant N N519 3505 33.