Record Details

Bayes Optimal Feature Selection for Supervised Learning

Electronic Theses of Indian Institute of Science

View Archive Info
 
 
Field Value
 
Title Bayes Optimal Feature Selection for Supervised Learning
 
Creator Saneem Ahmed, C G
 
Subject Data Analysis
Logarithms
Supervised Learning
Bayes Optimality
Binary Classsification
Bipartite Ranking
Multiclass Classification
Bayes Optimal Feature Selection
Optimal Feature Selection
Bayes Error
Binary Class Probability Estimation
Supervised Learning Problems
Computer Science
 
Description The problem of feature selection is critical in several areas of machine learning and data analysis such as, for example, cancer classification using gene expression data, text categorization, etc. In this work, we consider feature selection for supervised learning problems, where one wishes to select a small set of features that facilitate learning a good prediction model in the reduced feature space. Our interest is primarily in filter methods that select features independently of the learning algorithm to be used and are generally faster to implement compared to other types of feature selection algorithms. Many common filter methods for feature selection make use of information-theoretic criteria such as those based on mutual information to guide their search process. However, even in simple binary classification problems, mutual information based methods do not always select the best set of features in terms of the Bayes error.
In this thesis, we develop a general approach for selecting a set of features that directly aims to minimize the Bayes error in the reduced feature space with respect to the loss or performance measure of interest. We show that the mutual information based criterion is a special case of our setting when the loss function of interest is the logarithmic loss for class probability estimation. We give a greedy forward algorithm for approximately optimizing this criterion and demonstrate its application to several supervised learning problems including binary classification (with 0-1 error, cost-sensitive error, and F-measure), binary class probability estimation (with logarithmic loss), bipartite ranking (with pairwise disagreement loss), and multiclass classification (with multiclass 0-1 error). Our experiments suggest that the proposed approach is competitive with several state-of-the art methods.
 
Contributor Agarwal, Shivani
Veni Madhavan, C E
 
Date 2018-02-17T22:26:00Z
2018-02-17T22:26:00Z
2018-02-18
2014
 
Type Thesis
 
Identifier http://hdl.handle.net/2005/3138
http://etd.ncsi.iisc.ernet.in/abstracts/3992/G27114-Abs.pdf
 
Language en_US
 
Relation G27114