Nonlinear supervised dimensionality reduction via smooth regular embeddings

Download

index.pdf

Date

2019-03-01

Author

Ornek, Cem
Vural, Elif

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

236
views

117
downloads

The recovery of the intrinsic geometric structures of data collections is an important problem in data analysis. Supervised extensions of several manifold learning approaches have been proposed in the recent years. Meanwhile, existing methods primarily focus on the embedding of the training data, and the generalization of the embedding to initially unseen test data is rather ignored. In this work, we build on recent theoretical results on the generalization performance of supervised manifold learning algorithms. Motivated by these performance bounds, we propose a supervised manifold learning method that computes a nonlinear embedding while constructing a smooth and regular interpolation function that extends the embedding to the whole data space in order to achieve satisfactory generalization. The embedding and the interpolator are jointly learnt such that the Lipschitz regularity of the interpolator is imposed while ensuring the separation between different classes. Experimental results on several image data sets show that the proposed method outperforms traditional classifiers and the supervised dimensionality reduction algorithms in comparison in terms of classification accuracy in most settings.

Subject Keywords

Manifold learning, Dimensionality reduction, Supervised learning, Out-of-sample, Nonlinear embeddings

URI

https://hdl.handle.net/11511/38133

Journal

PATTERN RECOGNITION

DOI

https://doi.org/10.1016/j.patcog.2018.10.006

Collections

Department of Electrical and Electronics Engineering, Article

Suggestions

OpenMETU
Core

Nonlinear supervised dimensionality reduction via smooth regular embeddings Örnek, Cem; Schmidt, Şenan Ece; Department of Electrical and Electronics Engineering (2018) The recovery of the intrinsic geometric structures of data collections is an important problem in data analysis. Supervised extensions of several manifold learning approaches have been proposed in the recent years. Meanwhile, existing methods primarily focus on the embedding of the training data, and the generalization of the embedding to initially unseen test data is rather ignored. In this work, we build on recent theoretical results on the generalization performance of supervised manifold learning algori...
PROGRESSIVE CLUSTERING OF MANIFOLD-MODELED DATA BASED ON TANGENT SPACE VARIATIONS Gokdogan, Gokhan; Vural, Elif (2017-09-28) An important research topic of the recent years has been to understand and analyze manifold-modeled data for clustering and classification applications. Most clustering methods developed for data of non-linear and low-dimensional structure are based on local linearity assumptions. However, clustering algorithms based on locally linear representations can tolerate difficult sampling conditions only to some extent, and may fail for scarcely sampled data manifolds or at high-curvature regions. In this paper, w...
Learning semi-supervised nonlinear embeddings for domain-adaptive pattern recognition Vural, Elif (null; 2019-05-20) We study the problem of learning nonlinear data embeddings in order to obtain representations for efficient and domain-invariant recognition of visual patterns. Given observations of a training set of patterns from different classes in two different domains, we propose a method to learn a nonlinear mapping of the data samples from different domains into a common domain. The nonlinear mapping is learnt such that the class means of different domains are mapped to nearby points in the common domain in order to...
Out-of-Sample Generalizations for Supervised Manifold Learning for Classification Vural, Elif (2016-03-01) Supervised manifold learning methods for data classification map high-dimensional data samples to a lower dimensional domain in a structure-preserving way while increasing the separation between different classes. Most manifold learning methods compute the embedding only of the initially available data; however, the generalization of the embedding to novel points, i.e., the out-of-sample extension problem, becomes especially important in classification applications. In this paper, we propose a semi-supervis...
Learning Smooth Pattern Transformation Manifolds Vural, Elif (2013-04-01) Manifold models provide low-dimensional representations that are useful for processing and analyzing data in a transformation-invariant way. In this paper, we study the problem of learning smooth pattern transformation manifolds from image sets that represent observations of geometrically transformed signals. To construct a manifold, we build a representative pattern whose transformations accurately fit various input images. We examine two objectives of the manifold-building problem, namely, approximation a...

Citation Formats

C. Ornek and E. Vural, “Nonlinear supervised dimensionality reduction via smooth regular embeddings,” PATTERN RECOGNITION, pp. 55–66, 2019, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/38133.