Two layered Genetic Programming for mixed-attribute data classification

Abstract

The important problem of data classification spans numerous real life applications. The classification problem has been tackled by using Genetic Programming in many successful ways. Most approaches focus on classification of only one type of data. However, most of the real-world data contain a mixture of categorical and continuous attributes. In this paper, we present an approach to classify mixed attribute data using Two Layered Genetic Programming (L2GP). The presented approach does not transform data into any other type and combines the properties of arithmetic expressions (using numerical data) and logical expressions (using categorical data). The outer layer contains logical functions and some nodes. These nodes contain the inner layer and are either logical or arithmetic expressions. Logical expressions give their Boolean output to the outer tree. The arithmetic expressions give a real value as their output. Positive real value is considered true and a negative value is considered false. These outputs of inner layers are used to evaluate the outer layer which determines the classification decision. The proposed classification technique has been applied on various heterogeneous data classification problems and found successful.

Introduction

Data classification is of high interest due to its applicability in several critical domains like disease diagnosis, feature recognition, fraud detection and decision making. The real-life-data is very unpredictable, which makes the classification a challenging task. This increases the need of automated classification systems with no or minimum human interference. Some properties of a good classification system are:

• Robustness: The classifier should be able to output good results over a variety of problems.

• Applicability: It should be readily applicable to data without any preprocessing.

• Accuracy: The resultant classifier should be reliable and exhibit good generalizing abilities.

• Efficient modeling: The structure should be flexible to adapt the data properties. It should be independent of data distribution.

• Comprehensibility: The classifier should be comprehensible to help in future decision making.

• Portability: The classifier should be portable to other tools for future use and efficacy.

Fortunately, one of the recent evolutionary algorithms, Genetic Programming (GP), possesses the above mentioned abilities. This important feature of GP has been recognized since its inception and GP has been widely used for the classification tasks. While being successful in various application domains [1], [2], [3], [4], GP suffers from a few limitations like code bloat, long training time and lack of convergence, etc. Researchers have been trying to overcome these limitations to make the most out of this powerful classification tool.

Broadly, GP has been applied for classification in two different ways. One evolves classifiers as logical rules applicable to categorical data (continuous attributes need discretization). Another method is the evolution of arithmetic classifier expressions which is applicable to numerical attributes (categorical attributes are encoded into numeric values). The data transformations in either case can result in loss of information or biasness, in addition to added computational effort.

We have proposed a novel GP based classification system applicable to mixed attribute data without preprocessing of data. This is a two layered approach where the outer layer is a logical expression tree with some leaf nodes. These leaf nodes form the inner layer trees. The inner layer expressions can be of two types, logical expressions for categorical attributes and arithmetic expressions for continuous attributes of the data. The logical expressions give a Boolean value as output, which can be used by the outer layer, readily. On the other hand, the real output by arithmetic expressions is considered true for positive values and false for negative values. These outputs of inner layers are used to evaluate the outer layer which determines the classification decision. This novel GP based representation has been tested on various binary datasets from UCI repository [5]. The method has been found compatible with various other GP based classification algorithms.