A Theoretical Analysis of Multi-Modal Representation Learning with Regular Functions

2021-01-07
Multi-modal data analysis methods often learn representations that align different modalities in a new common domain, while preserving the within-class compactness and within-modality geometry and enhancing the between-class separation. In this study, we present a theoretical performance analysis for multi-modal representation learning methods. We consider a quite general family of algorithms learning a nonlinear embedding of the data space into a new space via regular functions. We derive sufficient conditions on the properties of the embedding so that high multi-modal classification or cross-modal retrieval performance is attained. Our results show that if the Lipschitz constant of the embedding function is kept sufficiently small while increasing the between-class separation, then the probability of correct classification or retrieval approaches 1 at an exponential rate with the number of training samples.
2020 28th Signal Processing and Communications Applications Conference (SIU)

Suggestions

A Study of the Classification of Low-Dimensional Data with Supervised Manifold Learning
Vural, Elif (2018-01-01)
Supervised manifold learning methods learn data representations by preserving the geometric structure of data while enhancing the separation between data samples from different classes. In this work, we propose a theoretical study of supervised manifold learning for classification. We consider nonlinear dimensionality reduction algorithms that yield linearly separable embeddings of training data and present generalization bounds for this type of algorithms. A necessary condition for satisfactory generalizat...
A survey on multidimensional persistence theory
Karagüler, Dilan; Pamuk, Semra; Department of Mathematics (2021-8)
Persistence homology is one of the commonly used theoretical methods in topological data analysis to extract information from given data using algebraic topology. Converting data to a filtered object and analyzing the topological features of each space in the filtration, we will obtain a way of representing these features called the shape of data. This will give us invariants like barcodes or persistence diagrams for the data. These invariants are stable under small perturbations. In most applications, we n...
A neuro-fuzzy MAR algorithm for temporal rule-based systems
Sisman, NA; Alpaslan, Ferda Nur; Akman, V (1999-08-04)
This paper introduces a new neuro-fuzzy model for constructing a knowledge base of temporal fuzzy rules obtained by the Multivariate Autoregressive (MAR) algorithm. The model described contains two main parts, one for fuzzy-rule extraction and one for the storage of extracted rules. The fuzzy rules are obtained from time series data using the MAR algorithm. Time-series analysis basically deals with tabular data. It interprets the data obtained for making inferences about future behavior of the variables. Fu...
Analysis of Face Recognition Algorithms for Online and Automatic Annotation of Personal Videos
Yılmaztürk, Mehmet; Ulusoy Parnas, İlkay; Çiçekli, Fehime Nihan (Springer, Dordrecht; 2010-05-08)
Different from previous automatic but offline annotation systems, this paper studies automatic and online face annotation for personal videos/episodes of TV series considering Nearest Neighbourhood, LDA and SVM classification with Local Binary Patterns, Discrete Cosine Transform and Histogram of Oriented Gradients feature extraction methods in terms of their recognition accuracies and execution times. The best performing feature extraction method and the classifier pair is found out to be SVM classification...
An approach for decentralized process modeling
Turetken, Oktay; Demirörs, Onur (2007-05-20)
This paper describes a method for organizations to perform process modeling in a decentralized and concurrent manner. The approach is based on the idea that modeling organizations' processes can be performed by individuals actually performing the processes. Instead of having a central and devoted group of people to analyze, understand, model and improve processes, real performers are held responsible to model and improve their own processes concurrently. The paper also summarizes the lessons learned by appl...
Citation Formats
E. Vural, “A Theoretical Analysis of Multi-Modal Representation Learning with Regular Functions,” presented at the 2020 28th Signal Processing and Communications Applications Conference (SIU), Gaziantep, Türkiye, 2021, Accessed: 00, 2021. [Online]. Available: https://hdl.handle.net/11511/89407.