Abstract
The second-order polynomial is commonly used for fitting a response surface but the low-order polynomial is not sufficient if the response surface is highly nonlinear. Based on genetic programming (GP), this paper presents a method with which high-order smooth polynomials, which can model nonlinear response surfaces, can be built. Since in many cases small samples are used to fit the response surface, it is inevitable that the high-order polynomial shows serious overfitting behaviors. Moreover, the high-order polynomial shows infamous wiggling, unwanted oscillations, and large peaks. To suppress such problematic behaviors, this paper introduces a novel method, called directional derivative-based smoothing (DDBS) that is very effective for smoothing a high-order polynomial.
The role of GP is to find appropriate terms of a polynomial through the application of genetic operators to GP trees that represent polynomials. The GP tree is transformed into the standard form of a polynomial using the translation algorithm. To estimate the coefficients of the polynomial quickly the ordinary least-square (OLS) method that incorporates the DDBS and extended data-set method is devised.
Also, by using the classical Lagrange multiplier method, the modified OLS method enabling interpolation is presented.
Four illustrative numerical examples are given to demonstrate the performance of GP with DDBS.
Similar content being viewed by others
References
Bettenhausen, K.D.; Marenbach, P.; Freyer, S.; Rettenmaier, H.; Nieken, U. 1995: Self-organizing structured modeling of a biotechnological fed-batch fermentation by means of genetic programming. In: Proc. the IEE Conf. GALESIA, Sheffield, UK, 481–486
Dennis, J.E.; Torczon Jr, V. 1997: Managing approximation models in optimization. In: Alexandrov, N.; Hussaini, M.Y. (eds.), Proc. Multidisciplinary Design Optimization: State-of-the-Art, 330–347
Goldstein, A.A.; Price, J.F. 1971: On descent from local minima. Math Comput25(115), 569–574
Gray, G.J.; Li, Y.; Murray-Smith, D.J.; Sharman, K.C. 1996: Structural system identification using genetic programming and a block diagram oriented simulation tool. Electron Lett32(15), 1422–1424
Iba, H.; Sato, T.; de Garis, H. 1994: System identification approach to genetic programming. In: Proc. the 1st IEEE Conf. on Evolutionary Computation, IEEE, Vol. 1, 401–406
Iba, H.; Nikolaev, N.Y. 2000: Genetic programming polynomial models of financial data series. In: Proc. IEEE 2000 Congress on Evolutionary Computation, IEEE, 1459–1466
Koza, J.R. 1992: Genetic programming: On the programming of computers by means of natural selection, Cambridge, MA: MIT Press
Lee, K.H.; Yeun, Y.S.; Ruy, W.S.; Yang, Y.S. 2002: Polynomial genetic programming for response surface modeling. In: Proc. the 4th Int. Workshop on Frontiers in Evolutionary Algorithms (FEA 2002), In conjunction with 6th Joint Conf. on Information Sciences, 572–577
McKay, B.; Willis, M.J.; Hiden, H.G.; Montague, G.A.; Barton, G.W. 1996: Identification of industrial processes using genetic programming. In: Proc. Identification in Engineering Systems Conf. (held in Swansea), Vol. 1, 510–519
McKay, M.D.; Beckman, R.J.; Conover, W.J. 1979: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics21(2), 239–245
Myers, R.H.; Montgomery, D.C. 1995: Response surface methodology: Process and product optimization using designed experiments, New York: Wiley
Nair, P.B.; Keane, A.J.; Shimpi, R.P. 1998: Combining approximation concepts with genetic algorithm-based structural optimization procedures. In: Proc. the 39th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conf. (held in Long Beach), 1741–1751, AIAA-98-1912
Nikolaev, N.Y.; Iba, H. 2001: Regularization approach to inductive genetic programming. IEEE Trans Evol Comput5(4), 359–376
Rodriguez, J.; Renaud, J.E.; Watson, L.T. 1998: Trust region augmented lagrangian methods for sequential response surface approximation and optimization ASME J Mech Des120(1), 58–66
Rosenbrock, H.H. 1960: An automatic method for finding the greatest or least value of a function. Comput J3(3), 175–184
Sharman, K.C.; Esparcia-Alcazar, A.I.; Li, Y. 1995: Evolving signal processing algorithms by genetic programming. In: Proc. the 1st IEE/IEEE Int. Conf. on GA in Eng. Syst.: Innovations and Appl. (held in Sheffield), 473–480
Toropov, V.V.; Alvarez, L.F. 1998a: Approximation model building for design optimization using genetic programming methodology. In: Proc. the 7th AIAA/USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization (held in St. Louis), Part1, 490–498
Toropov, V.V.; Alvarez, L.F. 1998b: Application of genetic programming and response surface methodology to optimization and inverse problems. In: Proc. Int. Symposium on Inverse Problems in Engineering Mechanics (held in Nagono City, Japan), 551–560
Yang, Y.S.; Jang, B.S.; Yeun, Y.S.; Jung, H.S. 2000: A framework for managing approximation models in place of expensive simulations in optimization. In: Proc. the 2nd ASMO UK/ISSMO Conf. on Engineering Design Optimization, 249–256
Yang, Y.S.; Jang, B.S.; Yeun, Y.S.; Ruy, W.S. 2002: Managing approximation models in multi-objective optimization. Struct Multidisc Optim24(2), 141–156
Yeun, Y.S.; Lee, K.H.; Yang, Y.S. 1999: Function approximations by coupling neural networks and genetic programming trees with oblique decision trees. Artif Intell Eng13(3), 223–239
Yeun, Y.S.; Suh, J.C.; Yang, Y.S. 2000: Function approximation by superimposing genetic programming trees: With application to engineering problems. Inf Sci122(2–4), 259–280
Yeun, Y.S.; Lee, K.H.; Han, S.M.; Yang, Y.S. 2001: Smooth fitting with a method for determining the regularization parameter under the genetic programming algorithm. Inf Sci133(3–4), 175–194
Weinbrenner, T. 1997: Genetic programming techniques applied to measurement data, Diploma Thesis, Dept. of Electronics and Electrical Engineering, Centre for Systems and Control, University of Glasgow
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yeun, Y., Yang, Y., Ruy, W. et al. Polynomial genetic programming for response surface modeling Part 1: a methodology. Struct Multidisc Optim 29, 19–34 (2005). https://doi.org/10.1007/s00158-004-0460-6
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00158-004-0460-6