Competing labels: a heuristic approach to pseudo-labeling in deep semi-supervised learning

2022-2-10
Bayrak, Hamdi Burak
Semi-supervised learning is one of the dominantly utilized approaches to reduce the reliance of deep learning models on large-scale labeled data. One mostly used method of this approach is pseudo-labeling. However, pseudo-labeling, especially its originally proposed form tends to remarkably suffer from noisy training when the assigned labels are false. In order to mitigate this problem, in our work, we investigate the gradient sent to the neural network and propose a heuristic method, called competing labels. In this method, we arrange the loss function and choose the pseudo-labels in a way that the gradient the model receives contains more than one negative element. We test our method on MNIST, Fashion-MNIST, and KMNIST datasets and show that our method has a better generalization performance compared to the originally proposed pseudo-labeling method.

Suggestions

Deep Learning-Based Hybrid Approach for Phase Retrieval
IŞIL, ÇAĞATAY; Öktem, Sevinç Figen; KOÇ, AYKUT (2019-06-24)
We develop a phase retrieval algorithm that utilizes the hybrid-input-output (HIO) algorithm with a deep neural network (DNN). The DNN architecture, which is trained to remove the artifacts of HIO, is used iteratively with HIO to improve the reconstructions. The results demonstrate the effectiveness of the approach with little additional cost.
Multi-task Deep Neural Networks in Protein Function Prediction
Rifaioğlu, Ahmet Süreyya; Doğan, Tunca; Martin, Maria Jesus; Atalay, Rengül; Atalay, Mehmet Volkan (2017-05-01)
In recent years, deep learning algorithms have outperformed the state-of-the art methods in several areas thanks to the efficient methods for training and for preventing overfitting, advancement in computer hardware, the availability of vast amount data. The high performance of multi-task deep neural networks in drug discovery has attracted the attention to deep learning algorithms in bioinformatics area. Here, we proposed a hierarchical multi-task deep neural network architecture based on Gene Ontology (GO...
Binary Classification Performance Measures/Metrics: A Comprehensive Visualized Roadmap to Gain New Insights
Canbek, Gurol; SAĞIROĞLU, Şeref; Taşkaya Temizel, Tuğba; Baykal, Nazife (2017-10-08)
Binary classification is one of the most frequent studies in applied machine learning problems in various domains, from medicine to biology to meteorology to malware analysis. Many researchers use some performance metrics in their classification studies to report their success. However, the literature has shown a widespread confusion about the terminology and ignorance of the fundamental aspects behind metrics. This paper clarifies the confusing terminology, suggests formal rules to distinguish between meas...
Visual Object Tracking with Autoencoder Representations
Besbinar, Beril; Alatan, Abdullah Aydın (2016-05-19)
Deep learning is the discipline of training computational models that are composed of multiple layers and these methods have recently improved the state of the art in many areas as a virtue of large labeled datasets, increase in the computational power of current hardware and unsupervised training methods. Although such a dataset may not be available for lots of application areas, the representations obtained by the well-designed networks that have a large representation capacity and trained with enough dat...
Deep learning approach for laboratory mice grimace scaling
Eral, Mustafa; Halıcı, Uğur; Department of Electrical and Electronics Engineering (2016)
Deep learning is extremely attractive research topic in pattern recognition and machine learning areas. Applications in speech recognition, natural language processing, and machine vision fields gained huge acceleration in performance by employing deep learning. In this thesis, deep learning is used for medical purposes in order to scale pain degree of drug stimulated mice by examining facial grimace. For this purpose each frame in the videos in the training set were scaled manually by experts according to ...
Citation Formats
H. B. Bayrak, “Competing labels: a heuristic approach to pseudo-labeling in deep semi-supervised learning,” M.S. - Master of Science, Middle East Technical University, 2022.