Peter Lichodzijewski
Malcolm I. Heywood
ABSTRACT
Bid-based Genetic Programming (GP) provides an elegant mechanism for facilitating cooperative problem decomposition without an a priori specification of the number of team members. This is in contrast to existing teaming approaches where individuals learn a direct input-output map (e.g., from exemplars to class labels), allowing the approach to scale to problems with multiple outcomes (classes), while at the same time providing a mechanism for choosing an outcome from those suggested by team members. This paper proposes a symbiotic relationship that continues to support the cooperative bid-based process for problem decomposition while making the credit assignment process much clearer. Specifically, team membership is defined by a team population indexing combinations of GP individuals in a separate team member population. A Pareto-based competitive coevolutionary component enables the approach to scale to large problems by evolving informative test points in a third population. The ensuing Symbiotic Bid-Based (SBB) model is evaluated on three large classification problems and compared to the XCS learning classifier system (LCS) formulation and to the support vector machine (SVM) implementation LIBSVM. On two of the three problems investigated the overall accuracy of the SBB classifiers was found to be competitive with the XCS and SVM results. At the same time, on all problems, the SBB classifiers were able to detect instances of all classes whereas the XCS and SVM models often ignored exemplars of minor classes. Moreover, this was achieved with a level of model complexity significantly lower than that identified by the SVM and XCS solutions.
- M. Brameier and W. Banzhaf. Evolving teams of predictors with linear genetic programming. Genetic Programming and Evolvable Machines, 2(4):381--407, 2001.]] Google Scholar
Digital Library
- M. Brameier and W. Banzhaf. Linear Genetic Programming. Springer, Genetic and Evolutionary Computation Series, 2007.]] Google ScholarDigital Library
- A. Chandra, H. Chen, and X. Yao. Trade-off between diversity and accuracy in ensemble generation, volume 16 of Studies in Computational Intelligence, chapter 19 in Multi-Objective Machine Learning, pages 429--464. Springer-Verlag, 2006.]]Google Scholar
- C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/$\sim$cjlin/libsvm.]]Google Scholar
- E. D. de Jong. A monotonic archive for Pareto-coevolution. Evolutionary Computation, 15(1):61--93, 2007.]] Google ScholarDigital Library
- R.-E. Fan, P.-H. Chen, and C.-J. Lin. Working set selection using second order information for training support vector machines. Journal of Machine Learning Research, 6:1889--1918, 2005.]] Google ScholarDigital Library
- S. G. Ficici and J. B. Pollack. Pareto optimality in coevolutionary learning. In Proceedings of the 6th European Conference on Advances in Artificial Life, pages 316--325, 2001.]] Google ScholarDigital Library
- J. B. J. M. Garrell. Bloat control and generalization pressure using the minimum description length principle for a Pittsburgh approach learning classifier system. Learning Classifier Systems, 4399:59--79, 2007.]] Google ScholarDigital Library
- S. Hettich and S. D. Bay. The UCI KDD Archive {http://kdd/ics/uci/edu}. Irvine, CA: University of California, Dept. of Information and Comp. Science, 1999.]]Google Scholar
- K. Imamura, T. Soule, R. B. Heckendorn, and J. A. Foster. Behavioral diversity and a probabilistically optimal GP ensemble. Genetic Programming and Evolvable Machines, 4(3):235--253, 2003.]] Google ScholarDigital Library
- P. Lichodzijewski and M. I. Heywood. Coevolutionary bid-based genetic programming for problem decomposition in classification. Genetic Programming and Evolvable Machines. Submitted.]] Google ScholarDigital Library
- P. Lichodzijewski and M. I. Heywood. GP classifier problem decomposition using first-price and second-price auctions. In Proceedings of the European Conference on Genetic Programming, pages 137--147, 2007.]] Google ScholarDigital Library
- P. Lichodzijewski and M. I. Heywood. Pareto-coevolutionary genetic programming for problem decomposition in multi-class classification. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 464--471, 2007.]] Google ScholarDigital Library
- A. R. McIntyre and M. I. Heywood. MOGE: GP classification problem decomposition using multi-objective optimization. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 863--870, 2006.]] Google ScholarDigital Library
- A. R. McIntyre and M. I. Heywood. Cooperative problem decomposition in Pareto competitive classifier models of coevolution. In Proceedings of the European Conference on Genetic Programming (to appear), 2008.]] Google ScholarDigital Library
- D. E. Moriarty and R. Miikkulainen. Forming neural networks through efficient and adaptive coevolution. Evolutionary Computation, 5(4):373--399, 1998.]] Google ScholarDigital Library
- D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz. UCI Repository of Machine Learning Databases {http://www.ics.uci.edu/$\sim$mlearn/mlrepository.html}. Irvine, CA: University of California, Dept. of Information and Comp. Science, 1998.]]Google Scholar
- J. Noble and R. A. Watson. Pareto coevolution: Using performance against coevolved opponents in a game as dimensions for Pareto selection. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 493--500, 2001.]]Google Scholar
- J. Paredis. The symbiotic evolution of solutions and their representations. In Proceedings of the 6th International Conference on Genetic Algorithms, pages 359--365, 1995.]] Google ScholarDigital Library
- G. Paris, D. Robiliard, and C. Ronlupt. Applying boosting techniques to genetic programming. In Proceedings of the International Conference on Artificial Evolution, pages 267--278, 2001.]] Google ScholarDigital Library
- C. D. Rosin and R. K. Belew. New methods for competitive coevolution. Evolutionary Compuatation, 5:1--29, 1997.]] Google ScholarDigital Library
- T. Soule. Cooperative evolution on the intertwined spirals problem. In Proceedings of the European Conference on Genetic Programming, pages 434--442, 2003.]] Google ScholarDigital Library
- R. Thomason and T. Soule. Novel ways of improving cooperation and performance in ensemble classifiers. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1708--1715, 2007.]] Google ScholarDigital Library
- S. Wilson. Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149--175, 1995.]] Google ScholarDigital Library
Index Terms
- Managing team-based problem solving with symbiotic bid-based genetic programming
Recommendations
Coevolutionary bid-based genetic programming for problem decomposition in classification
In this work a cooperative, bid-based, model for problem decomposition is proposed with application to discrete action domains such as classification. This represents a significant departure from models where each individual constructs a direct input-...
Pareto-coevolutionary genetic programming for problem decomposition in multi-class classification
GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computationA bid-based approach for coevolving Genetic Programming classifiers is presented. The approach coevolves a population of learners thatdecompose the instance space by way of their aggregate bidding behaviour. To reduce computation overhead, a small, ...
Symbiotic coevolutionary genetic programming: a benchmarking study under large attribute spaces
Classification under large attribute spaces represents a dual learning problem in which attribute subspaces need to be identified at the same time as the classifier design is established. Embedded as opposed to filter or wrapper methodologies address ...
Comments