Elsevier

Expert Systems with Applications

Volume 38, Issue 9, September 2011, Pages 11032-11039
Expert Systems with Applications

Modular neural network programming with genetic optimization

https://doi.org/10.1016/j.eswa.2011.02.147Get rights and content

Abstract

This study proposes a modular neural network (MNN) that is designed to accomplish both artificial intelligent prediction and programming. Each modular element adopts a high-order neural network to create a formula that considers both weights and exponents. MNN represents practical problems in mathematical terms using modular functions, weight coefficients and exponents. This paper employed genetic algorithms to optimize MNN parameters and designed a target function to avoid over-fitting. Input parameters were identified and modular function influences were addressed in manner that significantly improved previous practices. In order to compare the effectiveness of results, a reference study on high-strength concrete was adopted, which had been previously studied using a genetic programming (GP) approach. In comparison with GP, MNN calculations were more accurate, used more concise programmed formulas, and allowed the potential to conduct parameter studies. The proposed MNN is a valid alternative approach to prediction and programming using artificial neural networks.

Highlights

► A modular neural network is proposed for both predicting and programming problems. ► The programming is able to represent problems in modular functions mathematically. ► Parameter impacts and functional influences were addressed for concrete strengths. ► Good accuracies and programmed formulas were provided for high strength concrete.

Introduction

Soft-computing approaches have been refined into specialized branches that include neural networks (NNs), fuzzy logic, support vector machines, genetic algorithms (GAs) and genetic programming (GP), among others. Each has its particular merits in comparison with others. NNs are the most familiar soft-computing approach for learning tasks. The tools categorized into NNs, including back propagation networks (BPNs), radial basic functions, self-organizing mapping and learning vector quantization, are already applied in many fields for data mining and knowledge learning. BPNs are the most widely applied, accounting for over 70% of all NN applications. A neuron in the BPN model calculates net value based on associated inputs and multiplicative weights. As they lack hidden layers, BPNs are not able to address complicated problems. To enhance BPN capacity, hidden layers are employed between input-output mapping. High order neural networks (HONNs) were developed around 1990. HONN’s nonlinear combination of inputs (Zurada, 1992), allows for easy capture of high order correlations and efficient nonlinear mapping (Abdelbar and Tagliarini, 1996, Tsai, 2009). HONNs have been applied widely in various research domains, especially to problems related to pattern recognition and function approximation (Artyomov and Yadid-Pecht, 2005, Foresti and Dolso, 2004, Rovithakis et al., 2004, Wang and Lin, 1995). References show the HONN as a powerful soft-computing approach. HONN designs can further be specially adapted to represent problems in more meaningful ways.

Since it was first proposed by Koza (1992), GP has received much attention with regard to modeling nonlinear relationships in input-output mappings. GP creates solutions as programs (formulas) to solve problems with an operation tree. As some HONN models provide such programming functions as well, HONN is considered a branch of NN that, in contrast to GP, programs nonlinear relationships for input-output mappings.

This study proposes a modular neural network (MNN) able to accomplish both artificial intelligent prediction and programming. The proposed MNN comprises HONN models and modular functions, with each HONN model containing weight and exponent coefficients able to transfer inputs into meaningful polynomials.

The remaining sections of this paper include Section 2: Proposed MNN and GA optimization; Section 3: Programming high-strength concrete parameters to study MNN capacities in comparison with GP results; Section 4: Suggestions for further studies and future work and Section 5: Conclusions.

Section snippets

Modular neural network

The output (O) of the modular neural network (MNN) is a combination of several modular elements (M), with relationships wK, where w represents weight coefficients and K denotes the module index from 0 to NK (meaning that there are NK function selections and a bias term). Hence, O can be mathematically represented as (see Fig. 1):O=K=0NKwKMKwhere M0 is a bias unit with a value of 1.

Each MK is activated by a weighted summation operator and a setting function type FK. MK is a derivative of a

MNN programming for high-strength concrete parameters

Baykasoglu, Oztas, and Ozbay (2009) gathered 104 high strength concrete datasets. Six parameters were selected as input parameters, including water-binder ratio (W/B), water (W), fine aggregate (s/a), fly ash (FA), air entraining agent (AE), and super-plasticizer (SP). Output targets focused on concrete compressive strength (Strength), cost (Cost) and slump (Slump) (see Table 1). Parameter settings are listed in Table 2. The referenced RMSEs for GP were 2 MPa, 0.2 $/m3, and 20.7 cm for compressive

Future studies

Significant functions should be studied further in order for results to be applied to particular problems (like f in Eq. (6)). The NI and NJ used in this study were fixed. This may be improved by using different numbers of MNN inputs to reduce programmed formula length. Besides, the present MNN is a multi-inputs-single-output (MISO) mapping tool. To improve MNN for multi-inputs-multi-outputs (MIMO) problems is essential in order to create a set of MNN formulas with modular relationship

Conclusions

This paper proposed a modular neural network (MNN) concept applicable to both artificial intelligent prediction and programming. The resulting programmed formulas are comprehensive in terms of their mathematic functions, weights and exponents. MNN greatly improves the abilities of neural networks (even high order neural network) to predict and program. Case studies focused on parameters specific to high-strength concrete. The merits of MNN in comparison to genetic programming include:

  • 1.

    The

References (15)

There are more references available in the full text version of this article.

Cited by (11)

  • A genetically optimized neural network model for multi-class classification

    2016, Expert Systems with Applications
    Citation Excerpt :

    Koza (1992) brought a new dimension in this area, by using genetic evolution to optimize both, the weight and structure of a neural network. There have been many other works in this area(Khan, Ahmad, Khan, & Miller, 2013; Palmes, Hayasaka, & Usui, 2005; Rivero, Dorado, Rabuñal, & Pazos, 2010; Tsai & Lin, 2011; Turner, Dudek, & Ritchie, 2010). That motivates a new research dimension to model a hybrid solution i.e. evolution of artificial neural network using genetic programming.

  • Fuzzy logic in the gravitational search algorithm enhanced using fuzzy logic with dynamic alpha parameter value adaptation for the optimization of modular neural networks in echocardiogram recognition

    2015, Applied Soft Computing Journal
    Citation Excerpt :

    In this case, we are using a database of echocardiograms that contains images of disease and healthy patients. This paper focuses on the field of nature inspired computation and several approaches have been previously proposed for the optimization of modular neural networks [21,22,25,34,36,37]. A modular neural network for medical diagnosis is presented in [12,27], a MNN for pattern recognition is also shown in [16,23], preprocessing and segmentation techniques of medical images are presented in [38,41] and finally some medical applications using echocardiograms are presented in [1,9,11,30].

  • Fuzzy logic in the gravitational search algorithm for the optimization of modular neural networks in pattern recognition

    2015, Expert Systems with Applications
    Citation Excerpt :

    In this case, we are using a database of echocardiograms that contains images of disease and healthy patients. This paper focuses on the field of nature inspired computation and several approaches have been previously proposed for the optimization of modular neural networks (Martinez, Melin, & Castillo, 2005; Melin, Sánchez, & Castillo, 2012; Pandey & Garg, 2013; Tsai & Lin, 2011; Valdez, Melin, & Castillo, 2014; Valdez, Melin, & Parra, 2011), a modular neural network for medical diagnosis is presented in Das, Turkoglu, and Sengur (2009) and Rodriguez , Martín, Lafuente, Muguerza, and Pérez (1996), a MNN for pattern recognition is also shown in Gaxiola, Melin, and López (2010) and Melin, Mendoza, and Castillo (2011), preprocessing and segmentation techniques of medical images are presented in Vikram and Yongmin (1997) and Zong, Laine, and Geiser (1998) and finally some medical applications using echocardiograms are shown in Ahmed, Ungprasert, Srivali, Cheungpasitporn, and Bischof (2013), Chao, Wang, and Chan (2012), Crow, Hannan, Grandits, and Liebson (1996) and Strzelecki, Materka, Drozdz, Krzeminska, and Kasprzak (2006). The main problem in these medical applications is that normally there are many speckle noise points on the ultrasound images.

  • Breast cancer diagnosis using Genetically Optimized Neural Network model

    2015, Expert Systems with Applications
    Citation Excerpt :

    Koza and Rice (1991) brought a new dimension in this area, by using genetic evolution to optimize both, the weight and structure of a neural network. There have been many other works in this area (Palmes, Hayasaka, & Usui, 2005; Rivero, Dorado, Rabuñal, & Pazos, 2010; Tsai & Lin, 2011; Turner, Dudek, & Ritchie, 2010; Mahsal Khan, Masood Ahmad, Muhammad Khan, & Miller, 2013) however; none of the modifications is capable of delivering satisfactory performance for all problems, in general. Thus, the search for an approach to speed up its convergence and find the optimal structure still remain important.

  • An optimized instance based learning algorithm for estimation of compressive strength of concrete

    2012, Engineering Applications of Artificial Intelligence
    Citation Excerpt :

    Tesfamariam and Najjaran (2007) used an adaptive neuro fuzzy inference system to predict the compressive strength. Tsai and Lin (2011) used a modular neural network (MNN) to predict the compressive strength. The parameters of MNN were optimized by genetic algorithm.

View all citing articles on Scopus
1

Tel.: +886 2 27301073; fax: +886 2 27301074.

View full text