Data programming approach for weakly supervised learning of visual relations

Date

2024-1

Author

Gürsoy, Ceren

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

209
views

288
downloads

Cite This

Classifying interactions between objects in images plays an important role in extracting meaningful information from visuals. The learning process of visual relationship classification models, employed for this purpose, typically requires labeled datasets. However, acquiring annotated datasets, especially for infrequent classes, can be challenging due to the limitations of manual labeling. Time constraints, a shortage of domain experts, and the need for extensive datasets for complex models make manual labeling less practical. The presence of inaccurately, incompletely, or imprecisely labeled datasets causes further challenges. Addressing these problems, in this thesis, a method implementing a data programming approach is proposed to reduce the cost of the labeling process of datasets, where labels are created automatically based on weakly supervised learning by defining programmable functions. As a result of the experiments observing the interactions and effects of these functions, it is evident that ground truth labels can be approximated by constructing only five functions including weak classifiers trained with features extracted from data, textual and visual knowledge bases, and off-the-shelf pre-trained models. It is observed that the performance of a visual relationship classification model, trained with a dataset automatically labeled using the proposed method, closely approaches that of supervised learning. Therefore, this study diminishes the necessity for a manually labeled dataset for the visual relationship classification task, which has a wide application area.

Subject Keywords

Weakly supervised learning, Data programming, Visual relationship classification

URI

https://hdl.handle.net/11511/108301

Collections

Graduate School of Natural and Applied Sciences, Thesis