Feature enhancement with deep generative models in deep bayesian active learning

Duymuş, Pınar Ezgi
Data-intensive models emerge as new advances in Deep Learning take place. However, access to annotated datasets with many data points is not constantly prevalent. This situation emphasizes the need for Active Learning to select the least possible amount of data without compromising the accuracy of the classifier models. Recent advancements occur in Deep Bayesian Active Learning (DBAL), which means incorporating uncertainty of model parameters into a Deep Network. In this work, we present an algorithm that improves the accuracy of a DBAL model in an image classification task. We utilize the representation power of Deep Generative Models by employing their feature extraction capabilities. We obtain improved feature space representation of input data referred to as a latent vector by training a generative model. Instead of using the entire image space in the active learning setting, we demonstrate that utilizing latent space provides better data point selection for the active learning problem, hence obtaining higher accuracy. Furthermore, this study compares different generative models in terms of the ability to capture better feature representation. The informativeness of the data points defines how well an active learning algorithm performs. Therefore, capturing the latent space representation of a data point by extracting the highest information value possible is a significant contribution. We provide comparisons and experiments on different kinds of Generative Models, namely Vanilla Variational Autoencoders (VAEs), Maximum Mean Discrepancy Variational Autoencoders (MMDVAE) and Bidirectional Generative Adversarial Networks (BiGANs). Additionally, Bayesian Active Learning suffers from the Mode- Collapse problem. In order to ease that, we propose a diversity-based query algorithm to enhance the diversity of active points and improve the accuracy of the algorithm.


Closed-form sample probing for training generative models in zero-shot learning
Çetin, Samet; Cinbiş, Ramazan Gökberk; Department of Computer Engineering (2022-2-10)
Generative modeling based approaches have led to significant advances in generalized zero-shot learning over the past few-years. These approaches typically aim to learn a conditional generator that synthesizes training samples of classes conditioned on class embeddings, such as attribute based class definitions. The final zero-shot learning model can then be obtained by training a supervised classification model over the real and/or synthesized training samples of seen and unseen classes, combined. Therefor...
Deep Learning-Based Hybrid Approach for Phase Retrieval
IŞIL, ÇAĞATAY; Öktem, Sevinç Figen; KOÇ, AYKUT (2019-06-24)
We develop a phase retrieval algorithm that utilizes the hybrid-input-output (HIO) algorithm with a deep neural network (DNN). The DNN architecture, which is trained to remove the artifacts of HIO, is used iteratively with HIO to improve the reconstructions. The results demonstrate the effectiveness of the approach with little additional cost.
Enhanced Deep Learning with Improved Feature Subspace Separation
Parlaktuna, Mustafa; Sekmen, Ali; Koku, Ahmet Buğra; Abdul Malek, Ayad (2018-09-30)
This research proposes a new deep convolutional network architecture that improves the feature subspace separation. In training, the system considers M classes of input sets {C-i}(i=1)(M) and M deep convolutional networks {DNi}(i=1)(M) whose filter and other parameters are randomly initialized. For each input class C-i, Convolutional Neural Network generates a set of features F-i. Then, a local subspace S-i is matched for each set F-i. This is followed with a full training of the deep convolutional network ...
Feature Dimensionality Reduction with Variational Autoencoders in Deep Bayesian Active Learning
Ertekin Bolelli, Şeyda (2021-06-09)
Data annotation for training of supervised learning algorithms has been a very costly procedure. The aim of deep active learning methodologies is to acquire the highest performance in supervised deep learning models by annotating as few data points as possible. As the feature space of data grows, the application of linear models in active learning settings has become insufficient. Therefore, Deep Bayesian Active Learning methodology which represents model uncertainty has been widely studied. In this paper, ...
Competing labels: a heuristic approach to pseudo-labeling in deep semi-supervised learning
Bayrak, Hamdi Burak; Ertekin Bolelli, Şeyda; Yücel, Hamdullah; Department of Scientific Computing (2022-2-10)
Semi-supervised learning is one of the dominantly utilized approaches to reduce the reliance of deep learning models on large-scale labeled data. One mostly used method of this approach is pseudo-labeling. However, pseudo-labeling, especially its originally proposed form tends to remarkably suffer from noisy training when the assigned labels are false. In order to mitigate this problem, in our work, we investigate the gradient sent to the neural network and propose a heuristic method, called competing label...
Citation Formats
P. E. Duymuş, “Feature enhancement with deep generative models in deep bayesian active learning,” M.S. - Master of Science, Middle East Technical University, 2022.