Competing labels: a heuristic approach to pseudo-labeling in deep semi-supervised learning

Download

master_thesis_burak_bayrak_final.pdf

Date

2022-2-10

Author

Bayrak, Hamdi Burak

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

553
views

192
downloads

Semi-supervised learning is one of the dominantly utilized approaches to reduce the reliance of deep learning models on large-scale labeled data. One mostly used method of this approach is pseudo-labeling. However, pseudo-labeling, especially its originally proposed form tends to remarkably suffer from noisy training when the assigned labels are false. In order to mitigate this problem, in our work, we investigate the gradient sent to the neural network and propose a heuristic method, called competing labels. In this method, we arrange the loss function and choose the pseudo-labels in a way that the gradient the model receives contains more than one negative element. We test our method on MNIST, Fashion-MNIST, and KMNIST datasets and show that our method has a better generalization performance compared to the originally proposed pseudo-labeling method.

Subject Keywords

Semi-supervised learning, Deep learning, Pseudo-labeling, Machine learning

URI

https://hdl.handle.net/11511/96293

Collections

Graduate School of Applied Mathematics, Thesis

Suggestions

OpenMETU
Core

Multi-task Deep Neural Networks in Protein Function Prediction Rifaioğlu, Ahmet Süreyya; Doğan, Tunca; Martin, Maria Jesus; Atalay, Rengül; Atalay, Mehmet Volkan (2017-05-01) In recent years, deep learning algorithms have outperformed the state-of-the art methods in several areas thanks to the efficient methods for training and for preventing overfitting, advancement in computer hardware, the availability of vast amount data. The high performance of multi-task deep neural networks in drug discovery has attracted the attention to deep learning algorithms in bioinformatics area. Here, we proposed a hierarchical multi-task deep neural network architecture based on Gene Ontology (GO...
A method for quadruplet sample selection in deep feature learning Derin Öznitelik Öǧrenme için Dördüz Örnek Seçme Yöntemi Karaman, Kaan; Gundogdu, Erhan; Koc, Aykut; Alatan, Abdullah Aydın (2018-07-05) Recently, the deep learning based feature learning methodologies have been developed to recognize the objects in fine-grained detail. In order to increase the discriminativeness and robustness of the utilized features, this paper proposes a sample selection methodology for the quadruplet based feature learning. The feature space is manipulated by using the hierarchical structure of the training set. In the training process, the quadruplets are selected by considering the distances between the samples in the...
Deep Learning-Based Hybrid Approach for Phase Retrieval IŞIL, ÇAĞATAY; Öktem, Sevinç Figen; KOÇ, AYKUT (2019-06-24) We develop a phase retrieval algorithm that utilizes the hybrid-input-output (HIO) algorithm with a deep neural network (DNN). The DNN architecture, which is trained to remove the artifacts of HIO, is used iteratively with HIO to improve the reconstructions. The results demonstrate the effectiveness of the approach with little additional cost.
Binary Classification Performance Measures/Metrics: A Comprehensive Visualized Roadmap to Gain New Insights Canbek, Gurol; SAĞIROĞLU, Şeref; Taşkaya Temizel, Tuğba; Baykal, Nazife (2017-10-08) Binary classification is one of the most frequent studies in applied machine learning problems in various domains, from medicine to biology to meteorology to malware analysis. Many researchers use some performance metrics in their classification studies to report their success. However, the literature has shown a widespread confusion about the terminology and ignorance of the fundamental aspects behind metrics. This paper clarifies the confusing terminology, suggests formal rules to distinguish between meas...
Activity Learning from Lifelogging Images Belli, Kader; Akbaş, Emre; Yazıcı, Adnan (2019-01-01) The analytics of lifelogging has generated great interest for data scientists because big and multi-dimensional data are generated as a result of lifelogging activities. In this paper, the NTCIR Lifelog dataset is used to learn activities from an image point of view. Minute definitions are classified into activity classes using images and annotations, which serve as a basis for various classification techniques, namely SVMs and convolutional neural network structures (CNN), for learning activities. The perf...

Citation Formats

H. B. Bayrak, “Competing labels: a heuristic approach to pseudo-labeling in deep semi-supervised learning,” M.S. - Master of Science, Middle East Technical University, 2022.