A comparison of genetic programming and artificial neural networks in metamodeling of discrete-event simulation models

https://doi.org/10.1016/j.cor.2011.05.004Get rights and content

Abstract

Genetic programming (GP) and artificial neural networks (ANNs) can be used in the development of surrogate models of complex systems. The purpose of this paper is to provide a comparative analysis of GP and ANNs for metamodeling of discrete-event simulation (DES) models. Three stochastic industrial systems are empirically studied: an automated material handling system (AMHS) in semiconductor manufacturing, an (s,S) inventory model and a serial production line. The results of the study show that GP provides greater accuracy in validation tests, demonstrating a better generalization capability than ANN. However, GP when compared to ANN requires more computation in metamodel development. Even given this increased computational requirement, the results presented indicate that GP is very competitive in metamodeling of DES models.

Introduction

DES models are widely used in the design and analysis of systems. In many instances model execution times of DES models can be computationally expensive. In such cases, its use in operational tasks such as design, sensitivity analysis and optimization can be significantly undermined. This shortcoming of DES has motivated development of methods that allow the creation of approximate models, i.e., metamodels of systems which sacrifice accuracy for computational gain. A metamodel refers to an approximate predictive model of system performance which is dependent on decision variables.

Wang and Shan [1] outline the techniques that can be used to build approximate response models for engineering design problems. Jin et al. [2] perform a comparative analysis of different techniques: polynomial regression (PR), Kriging (KG), multivariate adaptive regression splines (MARS), radial basis functions (RBF) on a range of test problems. The results of the study showed a dispersion of the observed performance of the techniques in terms of accuracy and problem structures, e.g., size and nonlinearity of the problem. While these articles focus on only deterministic problems, Li et al. [3] present a thorough comparison on stochastic problems considering artificial neural networks (ANNs) and support vector regression (SVR) in addition to KG, MARS and RBF. However, none of these studies considered GP [4] in metamodeling of stochastic problems to evolve symbolic expressions.

In this paper, a comparison between GP and ANNs is presented since ANNs have found frequent application in simulation metamodeling. These approaches can develop models without underlying assumptions or a priori knowledge about the relationship between the control factors and the performance. To build approximate models, they only require the information on system configuration and the corresponding performance, i.e., training data. The approximations are progressively improved using the information available from previously generated metamodels throughout the process to stimulate the search to find better metamodels. This can be a means for building highly accurate metamodels as they do not necessitate a simplification on the complexity of the systems studied.

In the evaluation of GP against ANN, DES models of three systems, which are different in the size of decision space, in the degree of variability and in the range of the performance measures, are used. Since there is neither assumptions nor information on the underlying functions of the performance of these systems, the uniform design (UD) [5] is used to sample the decision space of the problems. UDs are space-filling experimental designs which can be used to obtain training data when the underlying model is unknown [6]. Therefore, they are inherently suitable to use with metamodeling approaches, such as ANNs and GP.

In the remainder of this paper, first a literature review of simulation metamodeling is presented. Following, the GP approach is briefly introduced. Subsequently, in Section 4, the methodology of the study is given and lastly the results of the study are presented. The results focus on the accuracy and robustness of the methods across the problems both in the model building (training) and in the validation (test) stages as well as the computational requirements. The results show that GP is a very competitive metamodeling method, showing superior results in developing more generalized metamodels when compared to ANN.

Section snippets

Literature review

One of the earliest reported approaches for building approximations is the response surface methodology (RSM) [7]. In general, low order polynomials are used in conjunction with regression analysis to fit the system responses [8], [9], [10]. In polynomial regression (PR), a low order analytical function of decision variables with unknown coefficients is used. The model is estimated via regression analysis. The model coefficients are updated with respect to residual errors between the fitted

Symbolic regression with genetic programming

Genetic programming (GP) is a sub-branch of evolutionary algorithms (EAs)) which emulate the natural evolution of species. It has the capability to evolve programs of a domain via symbolic regression [4]. These programs can be interpreted as grammar rules, logic instructions, analytical functions, etc. This ability has led to attempts to solve a variety of problems such as in genetics [34], data mining [35] and chemistry [36].

Koza [4] first proposed the use of GP to find a symbolic regression

Simulation models used in the study

This section describes the problems used to compare the performance of GP and ANN for metamodeling DES. As mentioned earlier (see Section 2), the analysis considers ANNs since they are commonly applied in metamodeling due to their efficiency.

Simulation models of three common industrial systems are used in the empirical comparative study of GP and ANN: (i) an automated material handling system (AMHS) in a semiconductor manufacturing facility; (ii) a periodic review inventory problem (s,S)

Experiments and results

In this section, the results from the experiments will be provided. The configuration of the algorithms are analyzed in terms of (i) training and test performance and (ii) computational effort. The configuration of the algorithms is systematically varied to identify better algorithmic settings. Such a comparison can allow benchmarking of the competitiveness of GP against a predominant approach, i.e., ANNs, in the context of simulation-based metamodeling. To establish this, in the remainder of

Conclusion

This paper presented an empirical comparison between two methods, artificial neural networks (ANNs) and genetic programming (GP) in constructing metamodels of discrete-event simulation (DES) models. Both ANN and GP have the advantage that they do not require a priori assumptions on the form of the metamodel.

The comparative study was carried out using three different systems: an automated material handling system (AMHS) in semiconductor manufacturing, an (s,S) inventory model and a serial

Acknowledgment

This research is funded by Irish Research Council for Science, Engineering and Technology. We also like to take the opportunity to thank the anonymous reviewer for the constructive comments.

References (58)

  • F.M. Alam et al.

    A comparison of experimental designs in the development of a neural network simulation metamodel

    Simulation Modelling Practice and Theory

    (2004)
  • F. Altiparmak et al.

    Buffer allocation and performance modeling in asynchronous assembly system operations: an artificial neural network metamodeling approach

    Applied Soft Computing

    (2007)
  • M. Chambers et al.

    Process optimization via neural network metamodeling

    International Journal of Production Economics

    (2002)
  • F. Yang

    Neural network metamodeling for cycle time-throughput profiles in manufacturing

    European Journal of Operational Research

    (2010)
  • K.Y. Chan et al.

    Modeling manufacturing processes using a genetic programming-based fuzzy regression with detection of outliers

    Information Science

    (2010)
  • W.H. Cai et al.

    Heat transfer correlations by symbolic regression

    International Journal of Heat and Mass Transfer

    (2006)
  • B. McKay et al.

    Steady-state modelling of chemical process systems using genetic programming

    Computers & Chemical Engineering

    (1997)
  • H. Zhao

    A multi-objective genetic programming approach to developing Pareto optimal decision trees

    Decision Support Systems

    (2007)
  • Y. Kuo et al.

    Simulation metamodel development using uniform design and neural networks for automated material handling systems in semiconductor wafer fabrication

    Simulation Modelling Practice and Theory

    (2007)
  • T.S. Meinert et al.

    A modular simulation approach for automated material handling systems

    Simulation Practice and Theory

    (1999)
  • J.C. Tyan et al.

    Multiple response optimization in a fully automated fab: an integrated tool and vehicle dispatching strategy

    Computers & Industrial Engineering

    (2004)
  • N. Nahas et al.

    A new approach for buffer allocation in unreliable production lines

    International Journal of Production Economics

    (2006)
  • G.G. Wang et al.

    Review of metamodeling techniques in support of engineering design optimization

    Journal of Mechanical Design

    (2007)
  • R. Jin et al.

    Comparative studies of metamodelling techniques under multiple modelling criteria

    Structural and Multidisciplinary Optimization

    (2001)
  • J. Koza

    Genetic programming: on the programming of computers by means of natural selection

    (1992)
  • K.-T. Fang et al.

    Uniform design: theory and application

    Technometrics

    (2000)
  • K. Fang et al.

    Design and modeling for computer experiments

    (2006)
  • M.I.R.D. Santos et al.

    Estimating and validating nonlinear regression metamodels in simulation

    Communications in Statistics—Simulation and Computation

    (2007)
  • H. Yang et al.

    Chaos optimization svr algorithm with application in prediction of regional logistics demand

  • Cited by (0)

    View full text