A Study of the Classification of Low-Dimensional Data with Supervised Manifold Learning

Date

2018-01-01

Author

Vural, Elif

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

73
views

0
downloads

Supervised manifold learning methods learn data representations by preserving the geometric structure of data while enhancing the separation between data samples from different classes. In this work, we propose a theoretical study of supervised manifold learning for classification. We consider nonlinear dimensionality reduction algorithms that yield linearly separable embeddings of training data and present generalization bounds for this type of algorithms. A necessary condition for satisfactory generalization performance is that the embedding allow the construction of a sufficiently regular interpolation function in relation with the separation margin of the embedding. We show that for supervised embeddings satisfying this condition, the classification error decays at an exponential rate with the number of training samples. Finally, we examine the separability of supervised nonlinear embeddings that aim to preserve the low-dimensional geometric structure of data based on graph representations. The proposed analysis is supported by experiments on several real data sets.

Subject Keywords

Manifold learning, Dimensionality reduction, Classification, Out-of-sample extensions

URI

https://hdl.handle.net/11511/52793

Journal

JOURNAL OF MACHINE LEARNING RESEARCH

Collections

Department of Electrical and Electronics Engineering, Article

Suggestions

OpenMETU
Core

Out-of-Sample Generalizations for Supervised Manifold Learning for Classification Vural, Elif (2016-03-01) Supervised manifold learning methods for data classification map high-dimensional data samples to a lower dimensional domain in a structure-preserving way while increasing the separation between different classes. Most manifold learning methods compute the embedding only of the initially available data; however, the generalization of the embedding to novel points, i.e., the out-of-sample extension problem, becomes especially important in classification applications. In this paper, we propose a semi-supervis...
A Theoretical Analysis of Multi-Modal Representation Learning with Regular Functions Vural, Elif (2021-01-07) Multi-modal data analysis methods often learn representations that align different modalities in a new common domain, while preserving the within-class compactness and within-modality geometry and enhancing the between-class separation. In this study, we present a theoretical performance analysis for multi-modal representation learning methods. We consider a quite general family of algorithms learning a nonlinear embedding of the data space into a new space via regular functions. We derive sufficient condit...
A Trie-structured Bayesian Model for Unsupervised Morphological Segmentation Kurfalı, Murathan; Ustun, Ahmet; CAN BUĞLALILAR, BURCU (2017-04-23) In this paper, we introduce a trie-structured Bayesian model for unsupervised morphological segmentation. We adopt prior information from different sources in the model. We use neural word embeddings to discover words that are morphologically derived from each other and thereby that are semantically similar. We use letter successor variety counts obtained from tries that are built by neural word embeddings. Our results show that using different information sources such as neural word embeddings and letter s...
PROGRESSIVE CLUSTERING OF MANIFOLD-MODELED DATA BASED ON TANGENT SPACE VARIATIONS Gokdogan, Gokhan; Vural, Elif (2017-09-28) An important research topic of the recent years has been to understand and analyze manifold-modeled data for clustering and classification applications. Most clustering methods developed for data of non-linear and low-dimensional structure are based on local linearity assumptions. However, clustering algorithms based on locally linear representations can tolerate difficult sampling conditions only to some extent, and may fail for scarcely sampled data manifolds or at high-curvature regions. In this paper, w...
Learning Smooth Pattern Transformation Manifolds Vural, Elif (2013-04-01) Manifold models provide low-dimensional representations that are useful for processing and analyzing data in a transformation-invariant way. In this paper, we study the problem of learning smooth pattern transformation manifolds from image sets that represent observations of geometrically transformed signals. To construct a manifold, we build a representative pattern whose transformations accurately fit various input images. We examine two objectives of the manifold-building problem, namely, approximation a...

Citation Formats

E. Vural, “A Study of the Classification of Low-Dimensional Data with Supervised Manifold Learning,” JOURNAL OF MACHINE LEARNING RESEARCH, pp. 1–55, 2018, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/52793.