A Theoretical Analysis of Multi-Modal Representation Learning with Regular Functions

Date

2021-01-07

Author

Vural, Elif

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

268
views

0
downloads

Multi-modal data analysis methods often learn representations that align different modalities in a new common domain, while preserving the within-class compactness and within-modality geometry and enhancing the between-class separation. In this study, we present a theoretical performance analysis for multi-modal representation learning methods. We consider a quite general family of algorithms learning a nonlinear embedding of the data space into a new space via regular functions. We derive sufficient conditions on the properties of the embedding so that high multi-modal classification or cross-modal retrieval performance is attained. Our results show that if the Lipschitz constant of the embedding function is kept sufficiently small while increasing the between-class separation, then the probability of correct classification or retrieval approaches 1 at an exponential rate with the number of training samples.

Subject Keywords

Multi-modal learning, cross-modal retrieval, theoretical analysis, Lipschitz-continuous functions

URI

https://hdl.handle.net/11511/89407

DOI

https://doi.org/10.1109/siu49456.2020.9302458

Conference Name

2020 28th Signal Processing and Communications Applications Conference (SIU)

Collections

Department of Electrical and Electronics Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

A Study of the Classification of Low-Dimensional Data with Supervised Manifold Learning Vural, Elif (2018-01-01) Supervised manifold learning methods learn data representations by preserving the geometric structure of data while enhancing the separation between data samples from different classes. In this work, we propose a theoretical study of supervised manifold learning for classification. We consider nonlinear dimensionality reduction algorithms that yield linearly separable embeddings of training data and present generalization bounds for this type of algorithms. A necessary condition for satisfactory generalizat...
Out-of-Sample Generalizations for Supervised Manifold Learning for Classification Vural, Elif (2016-03-01) Supervised manifold learning methods for data classification map high-dimensional data samples to a lower dimensional domain in a structure-preserving way while increasing the separation between different classes. Most manifold learning methods compute the embedding only of the initially available data; however, the generalization of the embedding to novel points, i.e., the out-of-sample extension problem, becomes especially important in classification applications. In this paper, we propose a semi-supervis...
A survey on multidimensional persistence theory Karagüler, Dilan; Pamuk, Semra; Department of Mathematics (2021-8) Persistence homology is one of the commonly used theoretical methods in topological data analysis to extract information from given data using algebraic topology. Converting data to a filtered object and analyzing the topological features of each space in the filtration, we will obtain a way of representing these features called the shape of data. This will give us invariants like barcodes or persistence diagrams for the data. These invariants are stable under small perturbations. In most applications, we n...
A neuro-fuzzy MAR algorithm for temporal rule-based systems Sisman, NA; Alpaslan, Ferda Nur; Akman, V (1999-08-04) This paper introduces a new neuro-fuzzy model for constructing a knowledge base of temporal fuzzy rules obtained by the Multivariate Autoregressive (MAR) algorithm. The model described contains two main parts, one for fuzzy-rule extraction and one for the storage of extracted rules. The fuzzy rules are obtained from time series data using the MAR algorithm. Time-series analysis basically deals with tabular data. It interprets the data obtained for making inferences about future behavior of the variables. Fu...
Analysis of Face Recognition Algorithms for Online and Automatic Annotation of Personal Videos Yılmaztürk, Mehmet; Ulusoy Parnas, İlkay; Çiçekli, Fehime Nihan (Springer, Dordrecht; 2010-05-08) Different from previous automatic but offline annotation systems, this paper studies automatic and online face annotation for personal videos/episodes of TV series considering Nearest Neighbourhood, LDA and SVM classification with Local Binary Patterns, Discrete Cosine Transform and Histogram of Oriented Gradients feature extraction methods in terms of their recognition accuracies and execution times. The best performing feature extraction method and the classifier pair is found out to be SVM classification...

Citation Formats

E. Vural, “A Theoretical Analysis of Multi-Modal Representation Learning with Regular Functions,” presented at the 2020 28th Signal Processing and Communications Applications Conference (SIU), Gaziantep, Türkiye, 2021, Accessed: 00, 2021. [Online]. Available: https://hdl.handle.net/11511/89407.