A Theoretical Analysis of Multi-Modal Representation Learning with Regular Functions

2021-01-07
Multi-modal data analysis methods often learn representations that align different modalities in a new common domain, while preserving the within-class compactness and within-modality geometry and enhancing the between-class separation. In this study, we present a theoretical performance analysis for multi-modal representation learning methods. We consider a quite general family of algorithms learning a nonlinear embedding of the data space into a new space via regular functions. We derive sufficient conditions on the properties of the embedding so that high multi-modal classification or cross-modal retrieval performance is attained. Our results show that if the Lipschitz constant of the embedding function is kept sufficiently small while increasing the between-class separation, then the probability of correct classification or retrieval approaches 1 at an exponential rate with the number of training samples.
2020 28th Signal Processing and Communications Applications Conference (SIU)

Suggestions

A Study of the Classification of Low-Dimensional Data with Supervised Manifold Learning
Vural, Elif (2018-01-01)
Supervised manifold learning methods learn data representations by preserving the geometric structure of data while enhancing the separation between data samples from different classes. In this work, we propose a theoretical study of supervised manifold learning for classification. We consider nonlinear dimensionality reduction algorithms that yield linearly separable embeddings of training data and present generalization bounds for this type of algorithms. A necessary condition for satisfactory generalizat...
A survey on multidimensional persistence theory
Karagüler, Dilan; Pamuk, Semra; Department of Mathematics (2021-8)
Persistence homology is one of the commonly used theoretical methods in topological data analysis to extract information from given data using algebraic topology. Converting data to a filtered object and analyzing the topological features of each space in the filtration, we will obtain a way of representing these features called the shape of data. This will give us invariants like barcodes or persistence diagrams for the data. These invariants are stable under small perturbations. In most applications, we n...
Development of an incompressible navier-stokes solver with alternating cell direction implicit method on structured and unstructured quadrilateral grids
Baş, Onur; Tuncer, İsmail Hakkı; Department of Aerospace Engineering (2007)
In this research, the Alternating Cell Direction Implicit method is used in temporal discretisation of the incompressible Navier-Stokes equations and compared with the well known and widely used Point Gauss Seidel scheme on structured and quadrilateral unstructured meshes. A two dimensional, laminar and incompressible Navier-Stokes solver is developed for this purpose using the artificial compressibility formulation. The developed solver is used to obtain steady-state solutions with implicit time stepping m...
A neuro-fuzzy MAR algorithm for temporal rule-based systems
Sisman, NA; Alpaslan, Ferda Nur; Akman, V (1999-08-04)
This paper introduces a new neuro-fuzzy model for constructing a knowledge base of temporal fuzzy rules obtained by the Multivariate Autoregressive (MAR) algorithm. The model described contains two main parts, one for fuzzy-rule extraction and one for the storage of extracted rules. The fuzzy rules are obtained from time series data using the MAR algorithm. Time-series analysis basically deals with tabular data. It interprets the data obtained for making inferences about future behavior of the variables. Fu...
Analysis of Face Recognition Algorithms for Online and Automatic Annotation of Personal Videos
Yılmaztürk, Mehmet; Ulusoy Parnas, İlkay; Çiçekli, Fehime Nihan (Springer, Dordrecht; 2010-05-08)
Different from previous automatic but offline annotation systems, this paper studies automatic and online face annotation for personal videos/episodes of TV series considering Nearest Neighbourhood, LDA and SVM classification with Local Binary Patterns, Discrete Cosine Transform and Histogram of Oriented Gradients feature extraction methods in terms of their recognition accuracies and execution times. The best performing feature extraction method and the classifier pair is found out to be SVM classification...
Citation Formats
E. Vural, “A Theoretical Analysis of Multi-Modal Representation Learning with Regular Functions,” presented at the 2020 28th Signal Processing and Communications Applications Conference (SIU), Gaziantep, Türkiye, 2021, Accessed: 00, 2021. [Online]. Available: https://hdl.handle.net/11511/89407.