Elsevier

Expert Systems with Applications

Volume 36, Issue 10, December 2009, Pages 12537-12546
Expert Systems with Applications

A genetic network programming with learning approach for enhanced stock trading model

https://doi.org/10.1016/j.eswa.2009.05.054Get rights and content

Abstract

In this paper, an enhancement of stock trading model using Genetic Network Programming (GNP) with Sarsa Learning is described. There are three important points in this paper: First, we use GNP with Sarsa Learning as the basic algorithm while both Technical Indices and Candlestick Charts are introduced for efficient stock trading decision-making. In order to create more efficient judgment functions to judge the current stock price appropriately, Importance Index (IMX) has been proposed to tell GNP the timing of buying and selling stocks. Second, to improve the performance of the proposed GNP-Sarsa algorithm, we proposed a new method that can learn the appropriate function describing the relation between the value of each technical index and the value of the IMX. This is an important point that devotes to the enhancement of the GNP-Sarsa algorithm. The third point is that in order to create more efficient judgment functions, sub-nodes are introduced in each node to select appropriate stock price information depending on the situations and to determine appropriate actions (buying/selling). To confirm the effectiveness of the proposed method, we carried out the simulation and compared the results of GNP-Sarsa with other methods like GNP with Actor Critic, GNP with Candlestick Chart, GA and Buy&Hold method. The results shows that the stock trading model using GNP-Sarsa outperforms all the other methods.

Introduction

A new graph-based evolutionary algorithm named Genetic Network Programming (GNP) (Eguchi et al., 2006, Mabu et al., 2007) has been proposed as an extended method of Genetic Algorithm (GA) (Goldberg, 1989, Holland, 1975) and Genetic Programming (GP) (Koza, 1992, Koza, 1994). Since GNP represents its solutions using graph structures, which contributes to creating quite compact programs and implicitly memorizing past action sequences in the network flows, it has been clarified that GNP is an effective method mainly for dynamic problems. Moreover, we proposed an extended algorithm of GNP which combines evolution and reinforcement learning (GNP-RL) (Mabu et al., 2007, Sutton and Barto, 1998). One of the advantages of GNP-RL is online learning. Original GNP is based on evolution only, therefore the programs are evolved mainly after task execution or enough trials, i.e., offline learning. On the other hand, GNP-RL can change its programs incrementally based on rewards obtained during task execution, i.e., online learning. For example, when an agent takes a good action with a positive reward at a certain state, the action is reinforced and therefore the same action will be adopted with higher probability when visiting the same state again. Another advantage of GNP-RL is a combination of a diversified search of GNP and an intensified search of RL. The role of evolution is to make rough structures, i.e., plural paths of node transitions, through selection, crossover and mutation. The role of RL is to determine one appropriate path in a structure made by evolution. Because RL is executed based on immediate rewards obtained after taking actions, intensified search, i.e., local search, can be executed efficiently. We call the evolution as a diversified search just because it could change programs largely than RL with which the programs could escape from local minima.

We note that the research on stock price prediction and trading model using evolutionary computation and neural networks has been done (Baba et al., 2002, Hassan et al., 2007, Oh et al., 2006, Potvin et al., 2004, Zou et al., 2007) in recent years. Generally speaking, methods for predicting stock prices and determining the timing of buying or selling stocks are divided into three groups: fundamental analysis (FA), technical analysis (TA) and efficient market hypothesis (EMH). Fundamental analysis analyzes stock prices using the financial statement of each company, the economic trend and movements of the exchange rate. Technical analysis analyzes numerically the past movement of stock prices. EMH rules out TA and FA by stating that the stock market is so efficient that all information is contained in the current stock price. One can disprove EMH if one can make excess returns through TA or FA. Generally, the research on stock price prediction and trading model using soft computing methods belongs to the technical analysis, and it determines the timing of buying and selling stocks based on the Technical Indices such as Relative Strength Index, Rate of Change, and Golden Cross. The proposed method also belongs to technical analysis.

In this paper, the enhancement of stock trading rules using GNP with Sarsa Learning is described. There are three important points that devote to the enhancement: First, we use the combination of GNP and Sarsa Learning (Mabu, Hatakeyama, Thu, Hirasawa, & Hu, 2006) which is one of the reinforcement learning methods. Although GNP with Sarsa Learning was applied to evolving the robot behaviors, in this paper, it has been applied to generate the rules for buying and selling stocks, where Importance Index (IMX) and Candlestick Charts (Izumi et al., 2006, Lee and Jo, 1999, Mabu et al., 2007) are introduced for efficient stock trading decision-making. Concretely speaking, Sarsa selects appropriate actions (buying/selling) and stock price information obtained from IMX and Candlestick Charts through the experiences during the trading. IMX and Candlestick Charts tell GNP whether or not the buying or selling signals are likely to appear at the current day. Although there are so many Technical Indices in the technical analysis, GNP with Sarsa can select appropriate indices and also select Candlestick Charts to judge the buying and selling timing of stocks. In other words, GNP with Sarsa could optimize the combinations of the information obtained by Technical Indices and Candlestick Charts. In the previous works, Mabu et al. (2007) only used GNP with Importance Index and Actor Critic, while Izumi et al., 2006, Mabu et al., 2007 only used GNP with Candlestick Chart as a basic algorithm. In this paper, the stock trading rules are created based on GNP using both Candlestick Chart and Importance Index as judgment functions to get more effective stock price information from the real stock markets. Moreover, in the learning phase, we used Sarsa Learning method for GNP individuals, which is different from the Actor Critic learning method used in Mabu et al. (2007). Comparing with our previous work (Chen, Mabu, Hirasawa, & Hu, 2007), the stock trading model has been improved a lot in this paper by the evolution of Importance Index. Second, to improve the performances of the previous GNP stock trading model, we proposed a new method that can learn the appropriate function describing the relation between the value of each technical index and the value of the IMX. This is an important point that devotes to the enhancement of the GNP-Sarsa algorithm. The third point is that sub-nodes are introduced in each judgment and processing node to determine appropriate actions (buying/selling) and to select appropriate stock price information depending on the situations. The basic structure of sub-nodes is described in the paper. To confirm the effectiveness of our proposed trading system, we carried out the simulations and compared the experimental results with those traditional methods under the same conditions. The results show that we can get more efficient trading rules and obtain much more profits by the proposed method.

This paper is organized as follows: Section 2 reviews the literature on the financial field by using various approaches. Section 3 describes the proposed GNP-Sarsa approach. Section 4 presents simulation environments, conditions and results using GNP-Sarsa method. The trading profits are presented and compared with the stock trading with traditional GNP methods, GA and Buy&Hold method. Finally, Section 5 concludes this paper.

Section snippets

Literature review

Prediction in financial domains, especially in stock markets is quite difficult for a number of reasons. First, the ultimate goal of our research is not to minimize the prediction error, but to maximize the profits. It forces us to consider a large number of independent variables, thereby increasing the dimensionality of the search space. Second, the weak relationships among variables tend to be nonlinear, and may hold only in limited areas of the search space. Especially, the data in stock

Basic structure of GNP-Sarsa

Fig. 1 shows a basic structure of GNP-Sarsa and Fig. 2 shows judgment node and processing node structures. GNP-Sarsa consists of judgment nodes and processing nodes, which are connected to each other. Judgment nodes have if-then type branch decision functions. They return judgment results for assigned inputs and determine the next node. Processing nodes take actions (buying or selling stocks). While judgment nodes have conditional branches, processing nodes have no conditional branches. The

Simulation

To confirm the effectiveness of GNP-Sarsa, we carried out the trading simulations using 16 brands selected from the companies listed in the first section of Tokyo stock market in Japan. The simulation period is divided into two periods: one is used for training and the other is used for testing simulation.

  • Training: January 4, 2001–December 30, 2003 (737 days)

  • Testing: January 5, 2004–December 30, 2004 (246 days)

We suppose that the initial funds is 5,000,000 Japanese yen in both periods, and the

Conclusions

In this paper, a stock trading model using GNP-Sarsa with important index and Candlestick Charts is proposed. First, a newly defined IMX function is assigned to each technical index to tell GNP-Sarsa whether buying or selling stocks is recommended or not. Second, Sarsa learns Q values to select appropriate sub-nodes/functions, and then to judge the current stock price information and determine the buying and selling timing. To confirm the effectiveness of the proposed method, we carried out

References (27)

  • Baba, N., Inoue, N., & Yan, Y. J. (2002). Utilization of soft computing techniques for constructing reliable decision...
  • Chen, Y., Mabu, S., Hirasawa, K., & Hu, J. (2007). Genetic network programming with Sarsa Learning and its application...
  • V. Dhar

    A comparison of GLOWER and other machine learning methods for investment decision making

    (2001)
  • Cited by (48)

    • A hybrid model for predicting human physical activity status from lifelogging data

      2020, European Journal of Operational Research
      Citation Excerpt :

      As lifelogging data is usually sparse and noisy due to the fact that each individual usually has his or her own activity pattern, the MOGP algorithm seems more suitable than the SVM in activity learning. Although GP algorithms have been used to evolve probabilistic trees that search for the optimal topology in bioinformatics (Won, Hamelryck, Prügel-Bennett, & Krogh, 2007) and stock trading (Chen, Mabu, Shimada, & Hirasawa, 2009; Ghaddar, Sakr, & Asiedu, 2016), to the best of our knowledge, this is the first work that a MOGP algorithm has been used as a multi-class classifier to construct a classification-HMM hybrid model for solving sequential learning problems. Our model can be of interest and easily adapted to other relevant domains in business analytics, such as consumer choice modelling (Blanchet, Gallego, & Goyal, 2016; Sandıkci, Maillart, Schaefer, Alagoz, & Robert, 2008) and high dimensional business data classification or dimension reduction (Debaere, Coussement, & De Ruyckc, 2018; Ghaddar & Naoum-Sawaya, 2018).

    • An efficient integration of the genetic algorithm and the reinforcement learning for optimal deployment of the wireless charging electric tram system

      2019, Computers and Industrial Engineering
      Citation Excerpt :

      The simulation results showed that GNPRL can create effective graph structure and get better results in dynamic environments. Chen, Mabu, Shimada, and Hirasawa (2009) addressed enhanced stock trading model using a genetic network programming (GNP) with Sarsa Learning which is one of the well-known reinforcement learning method. Sarsa obtained the stock price information from the importance index (IMX) and candlestick charts and then selected appropriate action; buying or selling.

    • A league championship algorithm equipped with network structure and backward Q-learning for extracting stock trading rules

      2018, Applied Soft Computing Journal
      Citation Excerpt :

      Based on their results, it was clarified that their model obtains more profits than GNP-Actor Critic, GNP candlestick chart, GA, and Buy&Hold. Mabu, Hirasawa [25] created a structural change in Chen, Mabu [24]’s model and proposed genetic network programing with rule accumulation (GNP-RA) for extracting trading rules. They believed that a model with rule accumulation which extracted and saved a lot of rules can understand and interpret a greater variety of events and conditions and therefore had a higher ability in decision-making than models without rule accumulation.

    • An intelligent hybrid trading system for discovering trading rules for the futures market using rough sets and genetic algorithms

      2017, Applied Soft Computing
      Citation Excerpt :

      Dempster and Jones [8] developed a trading system for finding the optimal technical trading rules using genetic programming (GP) for the foreign exchange (FX) market. A stock trading model based on genetic network programming (GNP) has been proposed by Chen et al. [6]. Esfahanipour and Mousavi [9] implemented a GP for automatically generating risk-adjusted technical trading rules on individual stocks listed on the Tehran stock exchange (TSE).

    View all citing articles on Scopus
    View full text