Elsevier

Computers & Chemical Engineering

Volume 28, Issue 12, 15 November 2004, Pages 2779-2790
Computers & Chemical Engineering

Adaptive genetic programming for steady-state process modeling

https://doi.org/10.1016/j.compchemeng.2004.09.001Get rights and content

Abstract

Genetic programming is one of the computer algorithms in the family of evolutionary-computational methods, which have been shown to provide reliable solutions to complex optimization problems. The genetic programming under discussion in this work relies on tree-like building blocks, and thus supports process modeling with varying structure. This paper, which describes an improved GP to facilitate the generation of steady-state nonlinear empirical models for process analysis and optimization, is an evolution of several works in the field. The key feature of the method is its ability to adjust the complexity of the required model to accurately predict the true process behavior. The improved GP code incorporates a novel fitness calculation, the optimal creation of new generations, and parameter allocation. The advantages of these modifications are tested against the more commonly used approaches.

Introduction

Completeness and harmony in nature are largely the result of evolutionary forces that have adapted species to their surroundings and to each other. Examining natural phenomena, one can appreciate the potential of nature's ready-made solutions, which are often more efficient than man-made ones. For example, consider spider silk, which is more elastic than nylon and stronger than steel, and witness the abilities of the spider, essentially blind with a limited nervous system, to use six variants of silk to build a robust, complex structure in an unpredictable environment (Conniff, 2001). With nature as a motivator, recent decades have seen increasing attempts to mimic natural evolution using computers (Fogel, Owens, & Walsh, 1965; Holland, 1975, Koza, 1992). These efforts are stimulated by Darwin's notion of “the survival of the fittest”, are generally referred to as evolutionary-computation, and have been used successfully to solve particularly complex problems.

The most commonly used evolutionary-computation algorithm is the genetic algorithm (GA), which is based on solutions of fixed-length chromosomes, usually consisting of binary genes, organized into sequences, often termed schema (Holland, 1975). Mimicking nature, the algorithm starts its search from an initial population of solutions, in which the performance of each individual is evaluated using a fitness function, with the most successful (efficient) chromosomes having a higher probability to reproduce. In synthetic evolution, biological reproduction is mimicked by operators like crossover (pairing) and mutation, thus creating a generation of offspring solutions. Crossover generates new features in the solution space by combining genetic information, while mutation does this by adding random perturbations. Fitness-proportional selection, combined with these genetic operators produce generation after generation of offspring solutions. Since the more appropriate solutions are given higher probabilities to reproduce, one would expect a growing improvement of the solutions over generations.

The efficiency of genetic algorithms in a number of fields has been demonstrated (e.g. Goldberg, 1989, Lewin, 1994, Lewin, 1996, Lewin, 1998, Lewin and Parag, 2003; Lewin, Wang, & Shalev, 1998). However, they are unsuitable for generating empirical model structures, since they manipulate populations of solutions of fixed-length chromosomes, while the optimal complexity of empirical models is unknown in advance. Because of this perceived need for more intelligent construction of empirical models, a new family of evolutionary-computation methods has emerged, based on established GA ideas. These new algorithms, referred to as genetic programming (GP), rely on tree-like building blocks, and therefore support populations of model structures of varying length and complexity. Activity in genetic programming was introduced by Koza (1992), who demonstrated their applications in fields such as robotics, games, control, and symbolic regression.

One of the important applications of genetic programming is in generating input–output empirical models in process system engineering (PSE) applications (Grosman, 2001). The class of empirical models can be divided into two broad categories: (a) models with predefined structure (either linear or nonlinear), and whose parameters are determined to maximize the capacity to predict process data; or (b) black-box models with undetermined structure. An example of the first category would be a linear model relating a dependent variable, y, to a set of n independent variables, ui:y=i=1naiui,where the coefficients ai are determined to maximize the predictive power of the model. An example of a black-box model would be an artificial neural network (ANN), in which the number and identity of the relevant inputs and the number of layers are the only attributes of the structure that are determined by the user. The disadvantage of the first alternative is that the user must specify the structure of the model in advance, which is in general difficult to do. In contrast, the main disadvantage of the neural network approach is that no formal equation is obtained, and thus, the resulting model is difficult or impossible to analyze. Consequently, great care must be taken with the ANN approach to prevent over-fitting. Genetic programming, being an evolutionary method for automatically generating nonlinear input–output models, overcomes both of the disadvantages mentioned above, since structured models are obtained, whose complexity is optimized.

When applying genetic programming to automatically generate nonlinear MISO (multiple input, single output) models, the probability of a given model surviving into the next generation is proportional to how well it predicts the output data. Components of successful models are continuously recombined with those of others to form new models, similarly to GAs. The GP optimizes the model structure, with a lower level nonlinear least-squares algorithm harnessed to perform the associated parameter estimation.

Several publications describe the usage of GP for nonlinear process modeling. South, Bancroft, Willis, and Tham (1996) introduced a genetic programming called GAFFER (genetic algorithm for finding existing relationships), an algorithm that combines both logical and mathematical operators. They demonstrate the use of the code to identify the parameters of a first order discrete model, with a known structure — an application typical of what could easily be solved with a GA — and in finding the structure and parameters of a simple discrete model. The method used to identify the model parameter values is not discussed, even though, as will be shown, this issue is critical. McKay, Willis, and Barton (1997) used a GP on a synthetic nonlinear test problem (to be used as a basis for comparison in this paper), and demonstrate its application inferential estimation in a vacuum distillation column, and in reaction system modeling. Willis, Hiden, Hinchliffe, McKay, and Barton (1997) applied GP to both steady-state and dynamic process modeling, and demonstrate their algorithm on the steady-state modeling of a binary distillation column, and on discrete dynamic modeling of a twin screw cooking extruder. Gray, Murray-Smith, Li, and Sharman (1996) implement a GP for dynamic modeling, in which a GP generates continuous discrete models, by combining MATLAB-SIMULINK blocks with sets of equations. In contrast, Marenbach (1998) generates continuous dynamic models by combining first principle dynamic blocks, called SMOG (structured model generator), and apply the approach to generate bioprocess dynamic models. Their approach is demonstrated on a simple linear second order model, a system of two water tanks in series, and the dynamic modeling of a helicopter engine. Lakshminarayanan, Fujii, Grosman, Dassau, and Lewin (2000) used a composite GP-PCA approach to generate nonlinear models for product design applications. Grosman and Lewin (2002) presented an approach to automate the design of nonlinear MPC (NMPC) using a GP-driven, model-based control approach. Recently, we have demonstrated the exploitation of Grosman's NMPC methodology for the control of a complete photolithography cluster (Grosman, Lachman-Shalem, Swissa, & Lewin, in press; Lachman-Shalem, Grosman, & Lewin, 2002).

This paper describes the modifications made to the generic genetic programming algorithm to improve its performance in providing modeling support for PSE applications. As such, this work can be seen as an evolution of the previously published works in the field and especially those applied to process engineering. The main objective is to improve the efficiency and accuracy of the algorithm, through a novel approach to the fitness calculation, tighter control on the creation of new generations of solutions, and on the positioning of parameters in the empirical models.

Section snippets

How does genetic programming work?

Similar to the GA, the GP is based on simple rules that imitate biological evolution. Combining basis functions, inputs and constants creates an initial model population, whose complexity is controlled by the user. The models are structured in a tree-like fashion, with basis functions linking nodes of inputs, as in the example in Fig. 1, where the tree-structure for the model: y = (c1 × u1 + u2) × u3 is shown, where y is the dependent variable, u1, u2 and u3 are independent variables, and c1 is

Improvements to the generic GP code

A close study of the published literature on genetic programming methodologies raises questions about the key decisions made in the coding of the algorithms and in the appropriate selection of tunable parameter values (e.g., McKay et al., 1997, Willis et al., 1997). The intuitive assumptions that guided this work were that the genetic programming algorithms could be written differently, and the hope was that these changes would improve the code's performance meaningfully. The key issues

Case studies

To test the performance of the modified GP and compare it with alternative approaches in the literature, two steady-state modeling case studies have been selected. The first one is a simple example presented by McKay et al. (1997), which was used in this work as a basis for comparison. The second one involves the modeling of the kinetics expression for the hydrodemethylation of toluene, based on an example in Fogler (1998). As will be seen, the performance of the proposed GP on both case

Results

The GP parameter values in Table 1 were selected to match those used by McKay et al. (1997), to enable a direct comparison between their results and those presented here, and to highlight the impact of the improvements made in the algorithm. Since the results obtained using GPs are probabilistic in nature, it is important to study a problem by carrying out multiple runs of the algorithm, and analyze the results using descriptive statistics before arriving at conclusions. Thus, 50 distinct GP

The impact of the improved features of the GP

Next, the three principal novel features in our GP are removed and substituted with generic strategies, to enable their impact on the performance to be estimated. Table 5 summarizes the GP results obtained after removing the dependency of the fitness function on the prediction of validation data. Table 6 presents the results obtained after changing the novel survival criteria, in which new solutions are only accepted into the population if they improve on the fitness of at least one of its

Conclusions

An improved GP code has been introduced, which extends previously published implementations of GPs with the following added features: (a) A novel fitness estimator, in which prediction error is computed using both modeling and verification data, and which penalizes over-complex models with a penalty function that is continuously adjusted to account for the complexity required to adequately model the input–output data. (b) Adopting the survival of the best until a better solution is found, in

Acknowledgements

The authors gratefully acknowledge the financial support of the Mitsubishi Chemical Corporation.

References (22)

  • D.E. Goldberg

    Genetic algorithms in search optimization and machine learning

    (1989)
  • Cited by (34)

    • Prediction of tropospheric ozone concentrations: Application of a methodology based on the Darwin's theory of evolution

      2011, Expert Systems with Applications
      Citation Excerpt :

      In crossover, represented in Fig. 2, two individuals are selected, their tree structures are divided at a randomly selected crossover point, and the resulting sub-trees are recombined to form two new individuals. In mutation, a random change is performed on a selected individual by substitution (Chen, Chang, Hou, & Lin, 2008; Grosman & Lewin, 2004). Fig. 3 shows two examples of mutation operation.

    • Lyapunov-based stability analysis automated by genetic programming

      2009, Automatica
      Citation Excerpt :

      Evidently, candidates that exhibit large regions of attraction will achieve fitness values approaching unity, while those with small regions of attraction will score poorly. Furthermore, as described by Grosman and Lewin (2004), our GP penalizes candidates with excessively complex models. This paper has presented a novel approach for the automatic generation of Lyapunov functions suitable for stability analysis of nonlinear systems.

    View all citing articles on Scopus
    View full text