Solving differential equations with constructed neural networks

Abstract

A novel hybrid method for the solution of ordinary and partial differential equations is presented here. The method creates trial solutions in neural network form using a scheme based on grammatical evolution. The trial solutions are enhanced periodically using a local optimization procedure. The proposed method is tested on a series of ordinary differential equations, systems of ordinary differential equations as well as on partial differential equations with Dirichlet boundary conditions and the results are reported.

Introduction

A series of problems in many scientific fields can be modelled with the use of differential equations such as problems in physics [1], [2], [3], [4], [5], chemistry [6], [7], [8], biology [9], [10], economics [11], etc. Due to the importance of differential equations many methods have been proposed in the relevant literature for their solution such as Runge Kutta methods [12], [13], [14], Predictor–Corrector [15], [16], [17], radial basis functions [18], [19], artificial neural networks [20], [21], [22], [23], [24], [25], [26], [27], models based on genetic programming [28], [29], etc. In this article a hybrid method utilizing constructed feed-forward neural networks by grammatical evolution and a local optimization procedure is used in order to solve ordinary differential equations (ODEs), systems of ordinary differential equations (SODEs) and partial differential equations (PDEs). The constructed neural networks with grammatical evolution have been recently introduced by Tsoulos et al. [30] and it utilizes the well-established grammatical evolution technique [31] to evolve the neural network topology along with the network parameters. The method has been tested with success on a series of data-fitting and classifications problems. In this article the constructed neural network methodology is applied on a series of differential equations while preserving the initial or boundary conditions using penalization. The proposed method does not require the user to enter any information regarding the topology of the network. Also, the new method can be used to solve either ODEs or PDEs and it can be easily parallelized. This idea is similar to the cascade correlation neural networks introduced by Fahlman and Lebiere [32] in which the user is not required to enter any topology information. However, the method for selecting the network topology differs since the proposed algorithm is a stochastic one. In the proposed method, the advantage of using an evolutionary algorithm is that the penalty function (used for initial or boundary conditions) can be incorporated easily into the training process.

The rest of this article is organized as follows: in Section 2 a brief description of the grammatical evolution algorithm is given followed by an analytical description of the proposed method, in Section 3 the test functions used in the experiments followed by the experimental results are outlined and in Section 4 some conclusions are derived.