- author = "Binh Ngan Tran",
- title = "Evolutionary Computation for Feature Manipulation in Classification on High-dimensional Data",
- school = "Computer Science, Victoria University of Wellington",
- year = "2018",
- address = "New Zealand",
- keywords = "genetic algorithms, genetic programming",
-
URL = "
https://researcharchive.vuw.ac.nz/xmlui/handle/10063/7078",
-
URL = "http://hdl.handle.net/10063/7078",
-
URL = "
https://researcharchive.vuw.ac.nz/xmlui/bitstream/handle/10063/7078/thesis_access.pdf",
- size = "275 pages",
-
abstract = "More and more high-dimensional data appears in machine
learning, especially in classification tasks. With
thousands of features, these datasets bring challenges
to learning algorithms not only because of the curse of
dimensionality but also the existence of many
irrelevant and redundant features. Therefore, feature
selection and feature construction (or feature
manipulation in short) are essential techniques in
preprocessing these datasets. While feature selection
aims to select relevant features, feature construction
constructs high-level features from the original ones
to better represent the target concept. Both methods
can decrease the dimensionality and improve the
performance of learning algorithms in terms of
classification accuracy and computation time.
Although feature manipulation has been studied for decades, the task on high-dimensional data is still challenging due to the huge search space. Existing methods usually face the problem of stagnation in local optima and/or require high computation time. Evolutionary computation techniques are well-known for their global search. Particle swarm optimisation (PSO) and genetic programming (GP) have shown promise in feature selection and feature construction, respectively. However, the use of these techniques to high-dimensional data usually requires high memory and computation time.
The overall goal of this thesis is to investigate new approaches to using PSO for feature selection and GP for feature construction on high-dimensional classification problems. This thesis focuses on incorporating a variety of strategies into the evolutionary process and developing new PSO and GP representations to improve the effectiveness and efficiency of PSO and GP for feature manipulation on high-dimensional data.
This thesis proposes a new PSO based feature selection approach to high-dimensional data by incorporating a new local search to balance global and local search of PSO. A hybrid of wrapper and filter evaluation method which can be sped up in the local search is proposed to help PSO achieve better performance, scalability and robustness on high-dimensional data. The results show that the proposed method significantly outperforms the compared methods in 80percent of the cases with an increase up to 16percent average accuracy while reduces the number of features from one to two orders of magnitude.
This thesis develops the first PSO based feature selection via discretisation method that performs both multivariate discretization and feature selection in a single stage to achieve better solutions than applying these techniques separately in two stages. Two new PSO representations are proposed to evolve cut-points for multiple features simultaneously. The results show that the proposed method selects less than 4.6percent of the features in all cases to improve the classification performance from 5percent to 23percent in most cases.
This thesis proposes the first clustering-based feature construction method to improve the performance of single-tree GP on high-dimensional data. A new feature clustering method is proposed to automatically group similar features into the same group based on a given redundancy level. The results show that compared with standard GP, the new method can select less than half of the features to construct a new high-level feature that achieves significantly better accuracy in most cases. The combination of the single constructed feature and the selected ones achieves the best performance among different feature sets created from a single tree.
This thesis develops the first class-dependent multiple feature construction method using multi-tree GP for high-dimensional data. A new GP representation and a new filter fitness function that combines two filter measures are proposed to evaluate the whole set of constructed features more effectively and efficiently. The results show that in 83percent of the cases, with less than 10 constructed features, the class-dependent method increases up to 32percent average accuracy on using all the original thousands of features and 10percent on using those constructed by the class-independent method.",
- notes = "Supervisors: Mengjie Zhang, Bing Xue",
