- author = "Bach Hoai Nguyen",
- title = "Evolutionary Computation for Feature Selection in Classification",
- school = "Computer Science, Victoria University of Wellington",
- year = "2018",
- address = "New Zealand",
- keywords = "genetic algorithms, genetic programming",
-
URL = "
http://hdl.handle.net/10063/7821",
-
URL = "
https://researcharchive.vuw.ac.nz/xmlui/bitstream/handle/10063/7821/thesis_access.pdf",
- size = "261 pages",
-
abstract = "Classification aims to identify a class label of an
instance according to the information from its
characteristics or features. Unfortunately, many
classification problems have a large feature set
containing irrelevant and redundant features, which
reduce the classification performance. In order to
address the above problem, feature selection is
proposed to select a small subset of relevant features.
There are three main types of feature selection
methods, i.e. wrapper, embedded and filter approaches.
Wrappers use a classification algorithm to evaluate
candidate feature subsets. In embedded approaches, the
selection process is embedded in the training process
of a classification algorithm. Different from the other
two approaches, filters do not involve any
classification algorithm during the selection process.
Feature selection is an important process but it is not
an easy task due to its large search space and complex
feature interactions. Because of the potential global
search ability, Evolutionary Computation (EC),
especially Particle Swarm Optimization (PSO), has been
widely and successfully applied to feature selection.
However, there is potential to improve the
effectiveness and efficiency of EC-based feature
selection.
The overall goal of this thesis is to investigate and improve the capability of EC for feature selection to select small feature subsets while maintaining or even improving the classification performance compared to using all features. Different aspects of feature selection are considered in this thesis such as the number of objectives (single-objective/multi-objective), the fitness function (filter/wrapper), and the searching mechanism.
This thesis introduces a new fitness function based on mutual information which is calculated by an estimation approach instead of the traditional counting approach. Results show that the estimation approach works well on both continuous and discrete data. More importantly, mutual information calculated by the estimation approach can capture feature interactions better than the traditional counting approach.
This thesis develops a novel binary PSO algorithm, which is the first work to redefine some core concepts of PSO such as velocity and momentum to suit the characteristics of binary search spaces. Experimental results show that the proposed binary PSO algorithm evolve better solutions than other binary EC algorithms when the search spaces are large and complex. Specifically, on feature selection, the proposed binary PSO algorithm can select smaller feature subsets with similar or better classification accuracies, especially when there are a large number of features.
This thesis proposes surrogate models for wrapper-based feature selection. The surrogate models use surrogate training sets which are subsets of informative instances selected from the training set. Experimental results show that the proposed surrogate models assist PSO to reduce the computational cost while maintaining or even improving the classification performance compared to using only the original training set.
The thesis develops the first wrapper-based multi-objective feature selection algorithm using MOEA/D. A new decomposition strategy using multiple reference points for MOEA/D is designed, which can deal with different characteristics of multi-objective feature selection such as highly discontinuous Pareto fronts and complex relationships between objectives. The experimental results show that the proposed algorithm can evolve more diverse non-dominated sets than other multi-objective algorithms.
This thesis introduces the first PSO-based feature selection algorithm for transfer learning. In the proposed algorithm, the fitness function uses classification performance to reduce the differences between domains while maintaining the discriminative ability on the target domain. The experimental results show that the proposed algorithm can select feature subsets which achieve better classification performance than four state-of-the-art feature-based transfer learning algorithms.",
- notes = "Supervisors: Bing Xue and Peter (Pondy) Andreae and Mengjie Zhang",
