Learning-based methods for multi-modal and multi-spectral data

Date

2020-10-15

Author

Özkan, Savaş

Metadata

Show full item record

Item Usage Stats

7
views

0
downloads

Data-driven solutions have become essential parts of our daily lives. These solutions generally consume a large amount of supervised data. Additionally, a large-body of learnable parameters must be trained to be able to reach the level of human knowledge accurately. However, these dependencies can be overcome by making full use of domain-specific features via specialized learning structures. This thesis addresses unsupervised and multimodal data by utilizing different sensor types and application domains in particular. Our first direction is to prove that robust representations can be computed from data without supervision if data-specific features can be appropriately modeled. For this purpose, multispectral data is used, which provides rich content information. Spectral material signatures and their fractional abundances are obtained from the data blindly. The second direction is to show that multimodal data offers theoretical and practical benefits to solve complicated visual tasks. In particular, we propose a novel NN model that is able to segment biomedical data from diverse visual modalities. The superiority is that no information about data modalities is provided at both train and test phases. This feature inevitably improves the applicability of this model. Furthermore, the number of learnable pav rameters is decreased since the solution is combined within a single model. For this purpose, we present novel modifications in the NN structure, and GAN is used that aims to cluster and adapt domains for different modalities. Lastly, we apply multimodal learning for the regression optimization of multispectral data, where a large amount of data is mainly required for more accurate solutions. This limitation is eliminated by solving the regression and the classification tasks concurrently. Similarly, experimental results show that both methods significantly improve the performance compared to all baselines.

Subject Keywords

Unsupervised learning, Multimodal data, Multispectral data, Biomedical data

URI

https://hdl.handle.net/11511/69163

Collections

Graduate School of Natural and Applied Sciences, Thesis