Feature weighting problem in k-Nearest neighbor classifier

Download
2017
Güleç, Nurullah
The k-Nearest Neighbor (k-NN) algorithm is one of the well-known and most common used algorithms for the classification problems. In this study, we have focused on feature weighted k-NN problems. Two different problems are studied. In the first problem, k value and the weights of each feature are optimized to maximize the classification accuracy. Objective function of the problem is nonconvex and nonsmooth. As a solution approach, Forest Optimization Algorithm (FOA), which is a newly introduced evolutionary algorithm, has been considered. Two different algorithms based on FOA are proposed. In the latter problem, class dependency on the feature weights is considered and class dependent feature weighted k-NN problem is studied where the feature weights are different for each class for maximizing the classification accuracy. A solution algorithm again based on FOA is proposed. All proposed algorithms are tested on different benchmark data sets and the numerical results are reported. Performances of the algorithms are also compared with the other algorithms from the other studies in the literature.

Suggestions

Multiobjective evolutionary feature subset selection algorithm for binary classification
Deniz Kızılöz, Firdevsi Ayça; Coşar, Ahmet; Dökeroğlu, Tansel; Department of Computer Engineering (2016)
This thesis investigates the performance of multiobjective feature subset selection (FSS) algorithms combined with the state-of-the-art machine learning techniques for binary classification problem. Recent studies try to improve the accuracy of classification by including all of the features in the dataset, neglecting to determine the best performing subset of features. However, for some problems, the number of features may reach thousands, which will cause too much computation power to be consumed during t...
Fusion of Image Segmentations under Markov Random Fields
Karadag, Ozge Oztimur; Yarman Vural, Fatoş Tunay (2014-08-28)
In this study, a fast and efficient consensus segmentation method is proposed which fuses a set of baseline segmentation maps under an unsupervised Markov Random Fields (MRF) framework. The degree of consensus among the segmentation maps are estimated as the relative frequency of co-occurrences among the adjacent segments. Then, these relative frequencies are used to construct the energy function of an unsupervised MRF model. It is well-known that MRF framework is commonly used for formulating the spatial r...
RTTES: Real-time search in dynamic environments
Undeger, Cagatay; Polat, Faruk (Springer Science and Business Media LLC, 2007-10-01)
In this paper we propose a real-time search algorithm called Real-Time Target Evaluation Search (RTTES) for the problem of searching a route in grid worlds from a starting point to a static or dynamic target point in real-time. The algorithm makes use of a new effective heuristic method which utilizes environmental information to successfully find solution paths to the target in dynamic and partially observable environments. The method requires analysis of nearby obstacles to determine closed directions and...
Nuclear Fission-Nuclear Fusion algorithm for global optimization: a modified Big Bang-Big Crunch algorithm
YALÇIN, YAĞIZER; Pekcan, Onur (Springer Science and Business Media LLC, 2020-04-01)
This study introduces a derivative of the well-known optimization algorithm, Big Bang-Big Crunch (BB-BC), named Nuclear Fission-Nuclear Fusion-based BB-BC, simply referred to as N2F. Broadly preferred in the engineering optimization community, BB-BC provides accurate solutions with reasonably fast convergence rates for many engineering problems. Regardless, the algorithm often suffers from stagnation issues. More specifically, for some problems, BB-BC either converges prematurely or exploits the promising r...
Frequent itemset minning with trie data structure and parallel execution with PVM
Guner, Levent; Karagöz, Pınar (2007-10-03)
Apriori algorithm is one of the basic algorithms introduced to solve the problem of frequent itemset mining (FIM). Since there is a new generation of affordable computers with parallel processing capability and it is easier to set up computer clusters, we can develop more efficient parallel FIM algorithms for these new systems. This paper investigates the use of trie data structure in parallel execution of Apriori algorithm, the potential problems during implementation, performance comparison of several par...
Citation Formats
N. Güleç, “Feature weighting problem in k-Nearest neighbor classifier,” M.S. - Master of Science, Middle East Technical University, 2017.