Forecasting with genetically programmed polynomial neural networks

https://doi.org/10.1016/j.ijforecast.2005.05.002Get rights and content

Abstract

Recent literature on nonlinear models has shown genetic programming to be a potential tool for forecasters. A special type of genetically programmed model, namely polynomial neural networks, is addressed. Their outputs are polynomials and, as such, they are open boxes that are amenable to comprehension, analysis, and interpretation.

This paper presents a polynomial neural network forecasting system, PGP, which has three innovative features: polynomial block reformulation, local ridge regression for weight estimation, and regularized weight subset selection for pruning that uses a least absolute shrinkage and selection operator. The relative performance of this system to other established forecasting procedures is the focus of this research and is illustrated by three empirical studies. Overall, the results are very promising and indicate areas for further research.

Introduction

Nonlinear models potentially track complex time series patterns that cannot be described by standard linear models. Many real-world forecasting problems have been found to be intrinsically nonlinear. Consequently, this class of models has become increasingly more attractive in forecasting (Hippert et al., 2004, Zhang et al., 1998). Among a range of nonlinear representations, such as Volterra and polynomial functions, neural networks, rational and radial basis functions, and kernel models, the present study is based on polynomial neural networks (PNN) as developed by Ivakhnenko (1971), and similar to those of Elder and Brown (2000). Polynomial neural networks are multilayer feedforward network architectures of cascaded polynomials, which can be used to produce time series and causal forecasting models.

In this paper, we report three empirical studies aimed at evaluating the relative performance of a genetically programmed polynomial neural network system, which we named PGP (polynomial genetic programming). Section 2 describes the PGP system, its theoretical background, and the current state-of-the-art in polynomial neural network forecasting, which provides the rationale for the new features implemented in the PGP. Section 3 focuses on the performance of the system, reporting on empirical studies. It starts with a brief description of the data and the alternative forecasting methods that were used for comparison. The general methodological procedure is then described and the results are summarized. In Section 4, we compare our findings with related work and assess some of its contributions, limitations, and potential improvements. Finally, we conclude and draw implications for future research.

Section snippets

Polynomial neural networks: the PGP system

In using PNN for time series forecasting, special attention needs to be given to model specification and the best way to estimate the model coefficients (weights). When the polynomial structure is known, coefficients are computed either analytically or by gradient methods. Analytical methods, as described by Schetzen (1980), assume that both the order and terms of the polynomial are either known or pre-specified. Gradient methods are implemented using neural networks with a fixed in advance

Studies

In this section, we compare the performance of PNN models obtained from the PGP system with more traditional approaches. Our main focus is on one-step-ahead forecasting performance. We used data from three different sources: the Airline Series, daily closing values of the Dow Jones index from March 4, 1997 to March 26, 2001, and hourly electric load during working days from the light utility area in Brazil over the period from 1 a.m. on January 1, 1996 to 8 a.m. on October 19, 1996. These time

The PGP versus related approaches

Recent research (Nikolaev & Iba, 2001) shows that, when evolved by genetic programming, polynomial neural networks outperform the traditional symbolic expression models that were proposed by Koza (1992). The PGP is based on binary hierarchical tree structures, which are suitable for genetic programming and can easily be extended to accommodate frequency domain components. In contrast to traditional Koza-style GP for evolving linear function models (e.g., Kaboudan, 2000, Koza, 1992), the PGP

Conclusion

This paper presented empirical evidence that genetically programmed polynomial neural network models are a promising framework for modelling nonlinear time series. Our results indicate that they may outperform other nonlinear approaches, like neural networks, and may even be applicable to the average-sized time series, which we encounter in practice.

To sum up, we can draw three main conclusions from this study. First, the PGP system can discover polynomials that capture the dynamics of the time

Acknowledgments

We are grateful to Helio F. da Silva for making the load data available to us. Thanks are also due to Hitoshi Iba, three anonymous referees, and an associate editor of the International Journal of Forecasting for their comments on previous versions of this paper.

Lilian M. de Menezes is a Senior Lecturer in Quantitative Methods at Cass Business School, City University, London. She completed her PhD at London Business School (1993) and has held posts at the London School of Economics and Goldsmiths College, University of London. Her research interests include time series forecasting, nonlinear models, and load forecasting. She has published her works in a number of journals including the European Journal of Operational Research, International Journal of

References (36)

  • S.H. Chen et al.

    Evolutionary artificial neural networks and genetic programming: A comparative study based on financial data

  • J.F. Elder et al.

    Induction and polynomial networks

  • J. Faraway et al.

    Time series forecasting with neural networks: A comparative study using the airline data

    Applied Statistics

    (1998)
  • C.F. Gerald et al.

    Applied numerical analysis

    (1994)
  • C. Granger et al.

    Modelling nonlinear economic relationships

    (1993)
  • J. Heywood et al.

    A framework for improved training of Sigma–Pi networks

    IEEE Transactions on Neural Networks

    (1996)
  • H.S. Hippert et al.

    Large neural networks for electricity load forecasting: Are they overfitted?

    International Journal of Forecasting

    (2004)
  • H.S. Hippert et al.

    Neural Networks for short term load forecasting: A review and evaluation

    IEEE Transactions on Power Systems

    (2001)
  • Cited by (28)

    • Constructing general partial differential equations using polynomial and neural networks

      2016, Neural Networks
      Citation Excerpt :

      This “combinatorial explosion” prevents from the full faultless data relation description, i.e. formation of a complete DE model. The D-PNN with more than 3-input variables must select from all the possible blocks in each layer, analogous to the GMDH algorithm (Menezes & Nikolaev, 2006) or apply aggregation sub-layers, which appear to be more efficient. In this way, each layer of the D-PNN is followed by an auxiliary sub-layer, which aggregates (sums) several block outputs to keep the previous layer number of blocks and thus prevents from an undesirable exponential combinatorial increase in the number of blocks input variables.

    • Application of evolutionary computation for rule discovery in stock algorithmic trading: A literature review

      2015, Applied Soft Computing Journal
      Citation Excerpt :

      Previous research suggested that applying EC-based models to trading rule discovery could yield promising results [32,43–45]. Moreover, EC can be employed to optimize the individual trading rule [14,26,44,46,47] and the parameters of the underlying rule discovery algorithm [44,48,49]. This study focuses on the distinctive competence of EC in rule discovery.

    • Stochastic and genetic neural network combinations in trading and hybrid time-varying leverage effects

      2014, Journal of International Financial Markets, Institutions and Money
    • Fluctuation prediction of stock market index by Legendre neural network with random time strength function

      2012, Neurocomputing
      Citation Excerpt :

      Egrioglu et al. [8] introduced a new method which is based on feed forward artificial neural network to analyze the multivariate high order fuzzy time series volatility. Menezes and Nikolaev [18] presented a polynomial neural network forecasting system, which has three features: polynomial block reformulation, local ridge regression for weight estimation and regularized weight subset selection. Further, the relative performance of this system to other established forecasting procedures is investigated and illustrated by three empirical studies.

    • Using artificial neural network models in stock market index prediction

      2011, Expert Systems with Applications
      Citation Excerpt :

      This architecture uses polynomials to build an ANN. Menezes and Nikolaev (2006) uses genetic algorithm to estimate ANN parameters such as starting polynomials, weight estimation etc. This study gives better result for some problems.

    View all citing articles on Scopus

    Lilian M. de Menezes is a Senior Lecturer in Quantitative Methods at Cass Business School, City University, London. She completed her PhD at London Business School (1993) and has held posts at the London School of Economics and Goldsmiths College, University of London. Her research interests include time series forecasting, nonlinear models, and load forecasting. She has published her works in a number of journals including the European Journal of Operational Research, International Journal of Forecasting, and Journal of the Royal Statistical Society.

    Nikolay Nikolaev is a Lecturer in the Department of Computing, Goldsmiths College, University of London. His research interests include genetic programming, neural networks, statistical learning algorithms, and time series forecasting. His papers have appeared in Neural Networks, IEEE Transactions on Evolutionary Computation, and IEEE Transactions on Neural Networks, amongst other journals.

    1

    Tel.: +44 20 7919 7854; fax: +44 20 7919 7853.

    View full text