Elsevier

Information Sciences

Volume 181, Issue 1, 1 January 2011, Pages 106-114
Information Sciences

Time-series forecasting using a system of ordinary differential equations

https://doi.org/10.1016/j.ins.2010.09.006Get rights and content

Abstract

This paper presents a hybrid evolutionary method for identifying a system of ordinary differential equations (ODEs) to predict the small-time scale traffic measurements data. We used the tree-structure based evolutionary algorithm to evolve the architecture and a particle swarm optimization (PSO) algorithm to fine tune the parameters of the additive tree models for the system of ordinary differential equations. We also illustrate some experimental comparisons with genetic programming, gene expression programming and a feedforward neural network optimized using PSO algorithm. Experimental results reveal that the proposed method is feasible and efficient for forecasting the small-scale traffic measurements data.

Introduction

Network traffic analysis and modeling play a major role in characterizing network performance and hence it has been a recent focus of many research works. Models that accurately capture the salient characteristics of the traffic is useful for analysis and simulation, and they also provide a better understanding of network dynamics. It is also useful for network design and engineering problems, e.g., the traffic balance scheme, router, switcher designing, the management of devices and its supporting software development etc.

Complexity is a key issue in network geometry and information traffic. Evidence of traffic complexity appears in many forms, such as the long-range correlations and self-similarities found in the statistical analysis of traffic measurements. There is also strong evidence of these phenomena occurring at several different time scales. The complexity revealed from the traffic measurements has led to the suggestion that the network traffic cannot be analyzed within the framework of available traffic models [9], [13], [15]. Alternative reliable traffic models and tools for quality assessment and control should be developed [7], [18], [19], [26].

Recently, communication and network technologies are developing rapidly, which prompts the traffic characteristics to change abruptly. The research emphasis of the network traffic analysis and modeling has to deal with large-time scale to smaller-time scale systems. Some recent research works have also illustrated that the traffic characteristics of the small-time scale systems were different from those of the large-time scale systems [25], [27]. So the large-time scale network traffic models cannot be suitable for the small-time scale network traffic.

Some researchers have proposed reasonable mathematical models based on the observed time series data so as to provide system analysis and prediction in various application domains [1], [4], [5], [8], [11], [12], [16], [20]. Mathematical modeling is the art of translating problems from an application area into tractable mathematical formulations, whose theoretical and numerical analysis provides insight, answers, and guidance, useful to understand the original application [12]. The system of differential equations can describe the dynamic properties of a system, which changes with time quite well and predict the future states of the system very conveniently. Cao et al. [3] used the ordinary differential equations (ODE) to predict the populations of the United States from 1790 to 1950, and the result indicate that the ODE was a powerful model in the discovery of sciential laws for dynamic data. In this research, we use the ODE model to predict the small-time scale traffic measurements data.

Various methods are proposed to infer ODEs during the last few years [3], [10], [24]. Most of these works can be classified into two groups: the first group is to identify the parameters of the ODEs and the second group is to identify the structure. The former is illustrated by genetic algorithms (GA), and the latter by the genetic programming (GP) approach. Cao et al. used GP to evolve the ODEs from the observed time series [3]. The main idea is to embed a genetic algorithm (GA) in genetic programming (GP), where GP is employed to discover and optimize the model’s structure, while the GA is employed to optimize its parameters. Authors illustrated that the GP-based approach introduce numerous advantages over other modeling methods. Tsoulos and Lagar proposed a novel method based on the grammatical evolution [24]. The method forms generations of trial solutions expressed in an analytical closed form. Iba proposed an ODE identification method by using the least mean square (LMS) along with the ordinary GP [10]. Some individuals are created by the LMS method at some intervals of generations and they replace the worst individuals in the population.

We also proposed a new representation scheme of the additive models for the system identification especially the reconstruction of polynomials and the identification of linear/nonlinear systems. This model is robust, and it is easy to analyze by traditional techniques. This is because the evolved additive tree model is simple and is very near to the traditional representation of the system to be reconstructed [6] and the computational complexity is similar to the GP.

In this paper, we proposed a hybrid evolutionary method, in which the tree-structure based evolution algorithm and particle swarm optimization (PSO) are employed to evolve the architecture and the parameters of the additive tree models for system of ordinary differential equation identification. The partitioning [2] is used in the process of identification of the structure of the system. Thus a ODE (Eq. (1)) containing some other variables can be evolved and is convenient and effective for a simple variable prediction.Y˙=f(X1,X2,,Xn).The paper is organized as follows. In Section 2, we describe the details of the proposed method. In Section 3, four examples are used to examine the effectiveness of the proposed method and finally Conclusions are drawn in Section 4.

Section snippets

Representation of additive tree model

We use the tree-structure based evolutionary algorithm to evolve the architecture of the additive tree models for the system of ordinary differential equation identification. For this purpose, we encode the right-hand side of an ODE into an additive tree individual as illustrated in Fig. 1.

Two instruction/operator sets I0 and I1 are used to generate the additive tree.I0={+2,+3,,,+N},I1=FT={,/,sin,cos,exp,rlog,x,R},where F = {*, /, sin, cos, exp, rlog} and T = {x, R} are functions and terminal sets. +N,

Structure optimization of models

Finding an optimal or near-optimal additive tree model is formulated as an evolutionary search process. We used the additive tree operators as following:

  • (1)

    Mutation. We choose three mutation operators to generate offsprings from the parents and the operation is described as follows:

    • (1)

      Change one terminal node: randomly select one terminal node in the tree and replace it with another terminal node, which is generated randomly.

    • (2)

      Grow: select a random leaf in the hidden layer of the tree and replace it

System for Prediction

After obtaining the best model, we then input the last line (feature vector) of the training data as the initial conditions of the best ODE to get the predicted time series of the system. The error is calculated by (5), (6). The process is described in Fig. 2.

Experimental results and analysis

To test the effectiveness of the proposed method, we uses the TCP traffic data, which is published by the Lawrence Berkeley Laboratory. This traffic data contain an hour’s worth of all wide-area traffic between Digital Equipment Corporation and the rest of the world. The data package used is DEC-Pkt1, and the time stamps have millisecond precision [23].The traffic data aggregated with time bin 0.1 s, that is the number of packages arrived within the 0.1 s time interval, are shown in Fig. 3.

In

Conclusions

In this paper, a hybrid evolutionary method for evolving ODEs is proposed to predict the small-time scale traffic measurements data. Tree-structure based evolutionary algorithm and particle swarm optimization (PSO) algorithm were used to evolve the architecture and the parameters of the additive tree models for identifying a System of ordinary differential equations. The experiment results clearly illustrate that the ODE model can effectively predict the traffic measurements data. The

Acknowledgments

This research was partially supported by the Natural Science Foundation of China (61070130), the Natural Science Foundation of Province (Y2007G33), the Shandong Distinguished Middle-aged and Young Scientist Encourage and Reward Foundation of China (BS2009SW003), and the Key Subject Research Foundation of Shandong Province.

References (27)

Cited by (48)

  • Sparse dynamical system identification with simultaneous structural parameters and initial condition estimation

    2022, Chaos, Solitons and Fractals
    Citation Excerpt :

    The problem of discovering governing equation from time-series observations is one of the challenges in modern dynamical system research [4]. There are a host of additional techniques for the discovery of governing equations from time-series data, including the state dependent parameter identification [2,5], symbolic regression [6–9], automated inference [10], empirical dynamic modeling [11], and deep learning [12,13]; see [14] for an incomplete review. These approaches can be viewed as special cases of the fourth paradigm: data-intensive scientific discovery [15].

  • LatinPSO: An algorithm for simultaneously inferring structure and parameters of ordinary differential equations models

    2019, BioSystems
    Citation Excerpt :

    The research in Iba (2008) evolves differential equation models by genetic programming, but this method had limitations in selecting the basis function and scalability. The research in Chen et al. (2011) presents a hybrid evolutionary method for identifying a system of ordinary differential equations (ODEs) to predict the small-time scale traffic measurements data, and it is not to search alternative structures with the same (or similar) system functionalities and behavior. In this research, we propose a novel swarm-based algorithm to infer the ODE model from biological time course data.

  • Generalized exponential autoregressive models for nonlinear time series: Stationarity, estimation and applications

    2018, Information Sciences
    Citation Excerpt :

    The above analysis indicates the GExpAR models is an effective method to model nonlinear time series and the VP method is an efficient algorithm to estimate the parameters of the GExpAR models. Time series modelling and prediction is an active research field [8,12,22,34,40], and it has wide applications [10,11,26,35]. The ExpAR model is a kind of classic nonlinear time series models.

View all citing articles on Scopus
View full text