Elsevier

Information Sciences

Volume 181, Issue 7, 1 April 2011, Pages 1249-1263
Information Sciences

Optimal depth estimation by combining focus measures using genetic programming

https://doi.org/10.1016/j.ins.2010.11.039Get rights and content

Abstract

Three-dimensional (3D) shape reconstruction is a fundamental problem in machine vision applications. Shape From Focus (SFF) is one of the passive optical methods for 3D shape recovery that uses degree of focus as a cue to estimate 3D shape. In this approach, usually a single focus measure operator is applied to measure the focus quality of each pixel in the image sequence. However, the applicability of a single focus measure is limited to estimate accurately the depth map for diverse type of real objects. To address this problem, we develop Optimal Composite Depth (OCD) function through genetic programming (GP) for accurate depth estimation. The OCD function is constructed by optimally combining the primary information extracted using one/or more focus measures. The genetically developed composite function is then used to compute the optimal depth map of objects. The performance of the developed nonlinear function is investigated using both the synthetic and the real world image sequences. Experimental results demonstrate that the proposed estimator is more useful in computing accurate depth maps as compared to the existing SFF methods. Moreover, it is found that the heterogeneous function is more effective than homogeneous function.

Introduction

Estimating three-dimensional (3D) shape of an object from its two-dimensional (2D) images is a fundamental problem in computer vision [1], [28], [39]. Broadly, 3D shape recovery algorithms based on the optical reflective model can be categorized into active and passive techniques. In active techniques, light rays are projected whereas, in passive methods, simply the reflection of light rays is captured without any projection. The Shape From Focus (SFF) is one of the passive methods, to estimate 3D structure of the object, based on the image focus analysis. It is a famous one in the paradigm of shape from X, where X denotes the cue used to infer the shape as stereo, motion, shading, de-focus, and focus. The SFF technique has been successfully utilized in many industrial applications, i.e. microelectronics [29], industrial inspection [33], medical diagnostics [5], 3D cameras [23], TFT–LCD color filter manufacturing [2], and comparison of polymers [26].

In SFF techniques, an image sequence is acquired by translating object along the optical axis. It is important to note that acquired images from lenses with limited depth of field have both the areas in and out of focus. However, it is possible to compute the well-focused image from the image sequence taken at different focus levels by computing the high frequency image contents. A criterion, usually known as focus measure, is used to compute the focus quality of each pixel in the image sequence and a focus volume is obtained. An initial depth map is extracted from the focus volume by maximizing the focus measure along the optical axis. In the literature, many focus measure operators are reported in spatial [10], [16], [36] and transform domains [18], [19], [25], [37], [42]. Once an initial depth map is computed, an approximation technique is applied to further refine these results [24], [28], [35]. Most of these techniques use a single focus measure to estimate the initial depth map. Due to the diverse nature of real images, it is difficult for a single focus measure to perform equally well under different scenarios. Moreover, it is hard to select a suitable focus measure for specific conditions. Another drawback with existing techniques is the error introduced in computing initial depth map also propagates to the approximation step. This demands to develop a new generalized optimal depth estimator that can effectively incorporate useful information from more than one focus measures.

In this work, we propose a novel idea of combining the initial depth and focus values extracted from various focus measures. Using this concept, the advantages of one focus measure can overcome the shortcomings of other focus measures. However, the problem is how to combine them in a best possible way. Under such circumstances, we introduce genetic programming (GP) based technique that optimally combines the initial information extracted from one or more focus measures. GP approach works on the principles of natural selection and recombination to search the space for possible solutions under a fitness criterion. Due to the flexibility of adjustable parameters, GP optimization technique [8], [13], [14], [17], [27] is widely used in the applications of image processing [31], pattern recognition [21], and computer vision [22]. In the proposed scheme, GP based Optimal Composite Depth (OCD) functions are developed using homogeneous and heterogeneous feature sets. In the first step, features consisting of initial focus and depth values are computed through existing focus measures. The useful feature information and the random constants are combined through arithmetic operators to develop the OCD function. The composite function is then used to compute the optimal depth map. The improved performance of the developed function is investigated using the synthetic and the real images. Experimental results demonstrate the superiority of the proposed GP based scheme over the conventional focus measures.

In the remainder of the paper, Section 2 describes the SFF scheme and presents a summary of existing focus measures and approximation techniques. Section 3 describes the effect of focus measures on the depth map based on experimental results. Section 4 describes the proposed GP based scheme in detail. Section 5 explains experimental setup and comparative analysis. Finally, Section 6 concludes this study.

Section snippets

Shape From Focus

Shape From Focus techniques retrieve the spatial information from multiple images of the same scene taken at different focus levels. In SFF, the objective is to find out the depth by measuring the distance of well-focused position of every object point from the camera lens. Once distances for all points of the object are known, the 3D shape can be recovered. Fig. 1 shows the basic geometry of image formation of focused and de-focused objects through the convex lens. Suppose the lens is

Focus measure vs depth map

In this section, we will discuss the effect of focus measures on the depth map of an object. Comparative studies of focus measure operators [9], [36], [37], [40] revealed that different focus measures provide different focus values and thus provide different depth values. Practically, it is hard to predict the suitable focus measure among a large list. Many factors including window size, mask size, noise level, illumination, contrast, affect the accuracy of the computed depth map. Some focus

Proposed GP-based scheme

Our aim is to develop GP based depth estimation function, F′: x  y, that maps the useful input information x to the target depth y-values. The proposed scheme is divided into Preprocessing module, GP module and Depth estimation module. In Preprocessing module, training data is formed by computing features of the known object. During GP process, optimal depth estimation function is developed using the training data. Depth estimation module is used to estimate the optimal depth map of the object.

Implementation details

For the development of OCD function, we prepared training data from the simulated object. A sequence of 97 images, each of size 360 × 360, of the simulated cone was generated synthetically by using the simulation software [35]. For each pixel, a homogeneous feature vector x1 is formed by applying F1 focus measure. In this way, feature vectors along with true depth values are used to construct training data of 129,600(=360 × 360) points. To avoid the over fitting problem and to reduce the training

Conclusion

In this paper, we developed an improved performance composite function for optimal depth estimation of real objects through GP based technique. The main advantage of the proposed scheme is that the useful information of individual focus measures is automatically selected and combined during GP evolution cycle. Another benefit is that this generic method does not depend on a specific focus measure. Moreover, the capability of proposed depth estimator can be enhanced by adding more useful

Acknowledgment

This work was supported by BioImaging Research Center at GIST.

References (44)

  • M.B. Ahmad et al.

    Application of three dimensional shape from image focus in LCD/TFT displays manufacturing

    IEEE Transactions on Consumer Electronics

    (2007)
  • M. Asif et al.

    Shape from focus using multilayer feedforward neural networks

    IEEE Transactions on Image Processing

    (2001)
  • J. Bernd

    Digital Image Processing

    (2005)
  • J.F. Brenner et al.

    An automated microscope for cytologic research a preliminary evaluation

    Journal of Histochemistry and Cytochemistry

    (1976)
  • T.S. Choi et al.

    Three-dimensional shape recovery from the focused-image surface

    Optical Engineering

    (2000)
  • J.A. dos Santos et al.

    A relevance feedback method based on genetic programming for classification of remote sensing images

    Information Sciences

    (2010)
  • F.C.A. Groen et al.

    A comparison of different focus functions for use in autofocus algorithms

    Cytometry

    (1985)
  • F.S. Helmli, S. Scherer, Adaptive shape from focus with an error estimation in light microscopy, in: Proceeding of the...
  • B.K. Horn

    Robot Vision

    (1986)
  • A. Khan et al.

    Combination and optimization of classifiers in gender classification using genetic programming

    International Journal of Knowledge-Based and Intelligent Engineering Systems

    (2005)
  • E. Krotkov

    Focusing

    International Journal of Computer Vision

    (1988)
  • W.B. Langdon

    Size fair and homologous tree genetic programming crossovers

    Genetic Programming and Evolvable Machines

    (2000)
  • Cited by (40)

    • Shape from focus using gradient of focus measure curve

      2023, Optics and Lasers in Engineering
    • Multiscale fusion and aggregation PCNN for 3D shape recovery

      2020, Information Sciences
      Citation Excerpt :

      For a simulated object, since the ground-truth is available, the discrepancies between the estimated depth maps and the true map is easy to calculate. We use three objective evaluation criteria: root mean squared error (RMSE), peak signal to noise ratio (PSNR) and correlation [25]. For real objects, it is not easy to evaluate a depth map quantitatively since the ground-truth map does not exist.

    • 3D shape reconstruction from multifocus image fusion using a multidirectional modified Laplacian operator

      2020, Pattern Recognition
      Citation Excerpt :

      The initial depth map can be obtained from maximizing these focus measure values. After that, some reconstruction schemes, such as the Gaussian model [14] and genetic programming [15], will make use of the focus information in the image sequence to find the accurate depth map. Optical microscopy is an important tool for high precision analysis and measurement of microscale objects due to its high magnification.

    View all citing articles on Scopus
    View full text