skip to main content
10.1145/1276958.1277292acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

Evolving kernels for support vector machine classification

Published:07 July 2007Publication History

ABSTRACT

While support vector machines (SVMs) have shown great promise in supervised classification problems, researchers have had to rely on expert domain knowledge when choosing the SVM's kernel function. This project seeks to replace this expert with a genetic programming (GP) system. Using strongly typed genetic programming and principled kernel closure properties, we introduce a new algorithm, called KGP, which finds near-optimal kernels. The algorithm shows wide applicability, but the combined computational overhead of GP and SVMs remains a major unresolved issue.

References

  1. G. H. Baker, L. Bottou, and J. Weston. Breaking SVM complexity with cross-training. In L. Saul, Y. Weiss, and L. Bottou, editors, Proceedings of Advances in Neural Information Processing Systems, pages 81--88, 2005.Google ScholarGoogle Scholar
  2. C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.Google ScholarGoogle Scholar
  3. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters forsupport vector machines. Machine Learning, 46:131--159, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. K. Crammer, J. Keshet, and Y. Singer. Kernel design using boosting. In S. Becker, S. Thrun, and K. Obermayer, editors, Proceedings of Advanced in Neural Information Processing Systems, pages 537--544, Vancouver, Canada, 2002. MIT Press.Google ScholarGoogle Scholar
  5. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In T. Diettrich, S. Becker, and Z. Ghahramani, editors, Proceedings of Advanced in Neural Information Processing Systems, pages 367--373, Vancouver, Canada, 2001. MIT Press.Google ScholarGoogle Scholar
  6. B. F. de Souza, A. C. de Carvalho, R. Calvo, and R. P. Ishii. Multiclass SVM model selection using particle swarm optimization. In Proceedings of the Sixth International Conference on Hybrid Intelligent Systems, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. C. B. D.J. Newman, S. Hettich and C. Merz. UCI repository of machine learning databases, 1998.Google ScholarGoogle Scholar
  8. F. Friedrichs and C. Igel. Evolutionary tuning of multiple SVM parameters. Neurocomputing, 64:107--117, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. C. Gagne, M. Schoenauer, M. Sebag, and M. Tomassini. Genetic programming for kernel based learning with co-evolving subsets selection. In Proceedings of Parallel Problem Solving in Nature, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. T. Howley and M. G. Madden. The genetic kernel support vector machine: Description and evaluation. Artificial Intelligence Review, 24(3 - 4):379--395, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. C.-W. Hsu, C.-C. Chang, and C.-J. Lin. A practical guide to support vector classification. Technical report, Department of Computer Science, National Taiwan University, 2003.Google ScholarGoogle Scholar
  12. C.-L. Huang, M.-C. Chen, and C.-J. Wang. Credit scoring with a data mining approach based on support vector machines. Expert Systems with Applications, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. C.-L. Huang and C.-J. Wang. A GA-based feature selection and parameter optimization for support vector machines. Expert Systems with Applications, 31:231 - 240, 2006.Google ScholarGoogle Scholar
  14. S. S. Keerthi. Efficient turing of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transactions on Neural Networks, 13(5):1225--1229, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. S. Luke. ECJ 13: A java-based computation and genetic programming research system. http://cs.gmu.edu/~eclab/projects/ecj/, 2005.Google ScholarGoogle Scholar
  16. S. Luke and L. Panait. Lexicographic parsimony pressure. In W. B. Langdon, editor, Proceedings of the Genetic and Evolutionary Computation Conference, pages 829--836, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. I. Mierswa. Evolutionary learning with kernels: A generic solution for large margin problems. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 1553--1560, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. D. J. Montana. Strongly typed genetic programming. Evolutionary Computation, 3(2):199--230, 11995. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. C. S. Ong, A. J. Smola, and R. C. Williamson. Hyperkernels. In S. Becker, S. Thrun, and K. Obermayer, editors, Proceedings of Advanced in Neural Information Processing Systems, pages 478--485, Vancouver, Canada, 2002. MIT Press.Google ScholarGoogle Scholar
  20. T. Phienthrakul and B. Kijsirikul. Evolutionary strategies for multi-scale radial basis function kernels in support vector machines. In Proceedings of the Genetic and Evolutionary Computation Conference, pages 905--911, Washington, DC, June 2005. ACM Press. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. T. P. Runarsson and S. Sigurdsson. Asynchronous parallel evolutionary model selection for support vector machines. Neural Information Processing - Letters and Reviews, 3(3):59--67, June 2004.Google ScholarGoogle Scholar
  22. B. Schölkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2002.Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. B. Schölkopf, A. J. Smola, R. C. Williamson, and P. L. Bartlett. New support vector algorithms. Neural Computation, 12:1207--1245, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. S. Sonnenburg, R. Gunnar, and S. Christin. A general and efficient multiple kernel learning algorithm. In Y. Weiss, B. Schölkopf, and J. Platt, editors,Proceedings of Advances in Neural Information Processing Systems, pages 1273--1280, Vancouver, Canada, 2005. MIT Press.Google ScholarGoogle Scholar
  26. I. Steinwart. Sparseness of support vector machines - some asymptotically sharp bounds. In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems, volume 16, 2004.Google ScholarGoogle Scholar

Index Terms

  1. Evolving kernels for support vector machine classification

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      GECCO '07: Proceedings of the 9th annual conference on Genetic and evolutionary computation
      July 2007
      2313 pages
      ISBN:9781595936974
      DOI:10.1145/1276958

      Copyright © 2007 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 7 July 2007

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • Article

      Acceptance Rates

      GECCO '07 Paper Acceptance Rate266of577submissions,46%Overall Acceptance Rate1,669of4,410submissions,38%

      Upcoming Conference

      GECCO '24
      Genetic and Evolutionary Computation Conference
      July 14 - 18, 2024
      Melbourne , VIC , Australia

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader