Optimising Multiple Kernels for SVM by Genetic Programming

Abstract

Kernel-based methods have shown significant performances in solving supervised classification problems. However, there is no rigorous methodology capable to learn or to evolve the kernel function together with its parameters. In fact, most of the classic kernel-based classifiers use only a single kernel, whereas the real-world applications have emphasized the need to consider a combination of kernels - also known as a multiple kernel (MK) - in order to boost the classification accuracy by adapting better to the characteristics of the data. Our aim is to propose an approach capable to automatically design a complex multiple kernel (CMK) and to optimise its parameters by evolutionary means. In order to achieve this purpose we propose a hybrid model that combines a Genetic Programming (GP) algorithm and a kernel-based Support Vector Machine (SVM) classifier. Each GP chromosome is a tree that encodes the mathematical expression of a MK function. Numerical experiments show that the SVM involving our evolved complex multiple kernel (eCMK) perform better than the classical simple kernels. Moreover, on the considered data sets, our eCMK outperform both a state of the art convex linear MK (cLMK) and an evolutionary linear MK (eLMK). These results emphasize the fact that the SVM algorithm requires a combination of kernels more complex than a linear one.