Deriving vegetation indices for phenology analysis using genetic programming

https://doi.org/10.1016/j.ecoinf.2015.01.003Get rights and content

Highlights

  • We extract plant color information from images and correlate with leaf phenological changes.

  • We use vegetation indices associated with plants for pattern analysis and knowledge extraction.

  • We present a novel approach for deriving appropriate vegetation indices from images.

  • Our method learns phenological patterns from plants through genetic programming.

  • We obtain composite vegetation indices that better characterize plant phenology.

Abstract

Plant phenology studies recurrent plant life cycle events and is a key component for understanding the impact of climate change. To increase accuracy of observations, new technologies have been applied for phenological observation, and one of the most successful strategies relies on the use of digital cameras, which are used as multi-channel imaging sensors to estimate color changes that are related to phenological events. We monitor leaf-changing patterns of a cerrado-savanna vegetation by taking daily digital images. We extract individual plant color information and correlate with leaf phenological changes. For that, several vegetation indices associated with plant species are exploited for both pattern analysis and knowledge extraction. In this paper, we present a novel approach for deriving appropriate vegetation indices from vegetation digital images. The proposed method is based on learning phenological patterns from plant species through a genetic programming framework. A comparative analysis of different vegetation indices is conducted and discussed. Experimental results show that our approach presents higher accuracy on characterizing plant species phenology.

Introduction

Phenology is the study of recurrent natural phenomena and its relationship to climate (Schwartz, 2013). Traditional phenology studies rely on the direct observation of plants, which is a time consuming and error-prone task. Recently, digital cameras have been applied as tools to monitor leaf changes on plants automatically (Alberton et al., 2014, Morellato et al., 2013, Schwartz, 2013).

The digital cameras can increase the accuracy of phenological observation and widen the study area, but their successful application as multi-channel imaging sensors to capture vegetation changes is reliant to the extraction of color change information out of the images (Alberton et al., 2014, Sonnentag et al., 2012). Generally, leaf color information is extracted from the red, blue, and green (RGB) color channels and the green channel is the most utilized to describe leaf changes, in combination to the red color (Richardson et al., 2007, Richardson et al., 2009). The normalized RGB chromatic coordinates are currently considered one of the most reliable indices to describe phenological changes based on image analyses (Sonnentag et al., 2012).

Considering the actual relevance of phenology as a tool for monitoring plant responses to climatic change and the need to reveal the environmental triggers of tropical phenology, here we propose a novel approach for deriving appropriate vegetation indices from vegetation digital images. The proposed method is based on learning phenological patterns from plant species through a genetic programming (GP) framework (da S. Torres et al., 2009). According to this framework, complex combinations of vegetation indices are modeled as individuals of a population. These individuals are then evolved by means of genetic operators (e.g., crossover, mutation, and reproduction) along generations. The objective is to obtain better performing complex vegetation indices that can be used to characterize the behavior of plant species or functional groups.

We performed a rigorous comparative analysis of different vegetation indices and discussed our proposed index accuracy in relation to the most utilized in the literature, the chromatic coordinates index, for near surface remote phenology studies (e.g., Ahrends et al. (2009); Alberton et al. (2014); Nagai et al. (2011); Richardson et al. (2009, 2007); Sonnentag et al. (2012)). Our experimental results show that vegetation indices may complement each other and improve indices' accuracy on characterizing plant species.

The remainder of this paper is organized as follows. Section 2 describes the methodology adopted for acquiring time series. Section 3 briefly discusses vegetation indices utilized in the phenology analysis. Section 4 presents the GP framework and shows how to apply it to model complex vegetation indices. Section 5 describes the experimental protocol adopted for evaluating vegetation indices. Section 6 reports the results of our experiments and compares our proposed index with other ones. Finally, we offer our conclusions and directions for future work in Section 7.

Section snippets

Time series acquisition

A digital hemispherical lens camera (Mobotix Q24) was set up in an 18 m tower in a Cerrado sensu stricto, a savanna vegetation located at Itirapina, São Paulo State, Brazil. We set up the camera to take a daily sequence of five JPEG images (at 1280 × 960 pixels of resolution) per hour, from 6:00 to 18:00 h (UTC-3). The present study was based on the analysis of over 2700 images, recorded at the end of the dry season, between August 29th and October 3rd 2011, day of year 241 to 278, during the main

Vegetation index-based phenology analysis

Digital images allow the detection of phenological events according to the changes of red, green, and blue (RGB) color channels along time (Richardson et al., 2007). By quantifying the RGB color channels it is possible to estimate, for instance, leaf flushing and senescence, using the green and red channels, respectively (Ahrends et al., 2009, Henneken et al., 2013, Morisette et al., 2009, Richardson et al., 2009). The quantification of RGB is performed by applying indices of color channels to

Vegetation index discovery process

This section introduces the proposed vegetation index discovery process. Section 4.1 provides a background on GP concepts, while Section 4.2 presents how GP can be used to create complex vegetation indices.

Experimental protocol

Unlike other research areas, evaluating a vegetation index in the context of phenology is not a straightforward task due to the lack of an objective ground-truth. Nowadays, some indices have been tested in the framework of near-surface remote phenology studies (e.g., Sonnentag et al. (2012)).

In this work, we adopted the evaluation method used in Almeida et al., 2013a, Almeida et al., 2013b. It relies on the identification of plant species in the image using time series extracted from pixels

Experimental results

The GP framework requires five parameters: the size of population Np, the number of generations Ngen, and the genetic operator rates (reproduction r, mutation m, and crossover c). The parameter r was set to 0.05, as suggested by da S. Torres et al. (2009). We conducted initial experiments, aiming at determining the best values for the remaining parameters, based on results reported by da S. Torres et al. (2009). First, a population of 100 GP individuals were evolved along 10 generations

Conclusions

Vegetation indices play an important role in characterizing leaf change patterns of plant species for near surface remote phenology studies. This work presented and discussed a genetic programming (GP) framework for deriving appropriate vegetation indices based on learning phenological patterns from vegetation digital images.

A rigorous comparative analysis of vegetation indices, as well as their possible combinations created by our approach, has been conducted in our experiments. Results

Acknowledgments

This research was supported by the São Paulo Research Foundation FAPESP and Microsoft Research Virtual Institute (grants #2010/52113-5, #2013/50169-1, and #2013/50155-0). BA received a master scholarship from CAPES and a doctoral fellowship from FAPESP (grant #2014/00215-0); LPCM and RST receive a Productivity Research Fellowship from CNPq (grants 306243/2010-5 and 306587/2009-2). Also, we have been benefited from funds of CAPES, CNPq (grant 449638/2014-6), FAPESP (grants #2007/52015-0, #

References (39)

  • O. Sonnentag et al.

    Digital repeat photography for phenological research in forest ecosystems

    Agric. For. Meteorol.

    (2012)
  • J. Zhao et al.

    Using digital cameras for comparative phenological monitoring in an evergreen broad-leaved forest and a seasonal rain forest

    Ecol. Inform.

    (2012)
  • L. Zhou et al.

    Modeling winter wheat phenology and carbon dioxide fluxes at the ecosystem scale based on digital photography and eddy covariance data

    Ecol. Inform.

    (2013)
  • H. Ahrends et al.

    Tree phenology and carbon dioxide fluxes: use of digital photography for process-based interpretation at the ecosystem scale

    Clim. Res.

    (2009)
  • J. Almeida et al.

    Remote phenology: applying machine learning to detect phenological patterns in a cerrado savanna

  • J. Almeida et al.

    Plant species identification with phenological visual rhythms

  • J. Almeida et al.

    Visual rhythm-based time series analysis for phenology studies

  • F.S.P. Andrade et al.

    Fusion of local and global descriptors for content-based image and video retrieval

  • R.A. Baeza-Yates et al.

    Modern Information Retrieval

    (1999)
  • Cited by (17)

    • A semi-supervised Genetic Programming method for dealing with noisy labels and hidden overfitting

      2018, Swarm and Evolutionary Computation
      Citation Excerpt :

      the machine learning community, despite its success in many real world applications [6,91]. Nevertheless, it is being used in a growing number of remote sensing applications (e.g., [92–98]), including the identification of burned areas in satellite imagery [99]. Classification trees are a non-parametric, non-linear rule based classifier that generates classification rules through an induction procedure described in [83].

    • Introducing digital cameras to monitor plant phenology in the tropics: applications for conservation

      2017, Perspectives in Ecology and Conservation
      Citation Excerpt :

      The fish-eyes lens (360°) improve the selection of crowns with more precision and less covered areas (see Alberton et al., 2014). We recommend capturing a high frequency of images (a set of 3–5 images per hour, from 6 a.m. to 6 p.m.), which provides fine-tuned information about phenology, a confident quality data collection, and also a high volume of data for light calibration, smoothing and the development of computational tools (Alberton et al., 2014; Almeida et al., 2014, 2015a; Almeida et al., 2016b). When it is not possible due to storage constraints, we recommend taking at least one image per hour during the midday hours (10 a.m. to 2 p.m., for more details see the SM).

    • Fusion of time series representations for plant recognition in phenology studies

      2016, Pattern Recognition Letters
      Citation Excerpt :

      Future work includes the use of other image descriptors to extract different visual features (e.g., shape description approaches based on contour and regions [13]). In addition, the proposed framework can be augmented to consider learning-to-rank methods (e.g., genetic programming [8]) for combining different descriptors. Another point to be explored is the use of RP for multispectral images or multisensor systems, which have more than three time series.

    • Phenological visual rhythms: Compact representations for fine-grained plant species identification

      2016, Pattern Recognition Letters
      Citation Excerpt :

      We obtained a time series associated with each ROI, raising the need of using appropriate tools for mining patterns of interest in a given digital image [7,11,32,33]. The plant species identification is a key issue for the phenological observation of tree crowns using phenocams, especially in tropical vegetations where one single image may include a high number of species [2,6,7]. This task is time consuming since first each crown in the image has to be matched to the tree in the soil and then the tree is identified at species level.

    View all citing articles on Scopus
    View full text