Date

2023-1-18

Author

Demirel, Berkan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

278
views

297
downloads

Cite This

Object detection is considered one of the most challenging problems in computer vision since it requires correctly predicting both the object classes and their locations. In the literature, object detection approaches are usually trained in a fully-supervised manner, with a large amount of annotated data for all classes. Since data annotation is costly in terms of both time and labor, there are also alternative object detection methods, such as weakly supervised or mixed supervised learning to reduce these costs in the literature. In this thesis, our focus is handling object detection problem with minimum supervision. In this context, we first define a difficult scenario namely zero-shot object detection (ZSD), where no visual training data is available for some of the target object classes. Secondly, we focus on the few-shot object detection (FSOD) problem and propose the novel meta-tuning principle. In the ZSD problem, we propose an approach that uses visual class embeddings and convex combinations of semantic embeddings in the classification part of single-stage object detectors. Following the proposed method, we focus on using more informative word embeddings, background modeling, and potential applications for ZSD methods. We first analyze the use of embedding vectors in deep models since these vectors are an essential knowledge source for zero-shot learning (ZSL), and we propose a novel approach that transforms semantically meaningful word vectors into visually meaningful ones. We show that using the proposed visually meaningful word embedding vectors obtain state-of-the-art results in the zero-shot classification (ZSC) problem. Then, we propose the first attempt to handle the background modeling in ZSD using a novel textual attention mechanism. Finally, we introduce a new problem within the scope of ZSD applications, which we call zero-shot image captioning (ZSIC), where the input images may consist of unseen object instances. The proposed ZSIC method use template-based sentence generators and fills the empty visual template slots with object proposals obtained from ZSD methods. In this context, we also propose a new evaluation metric called V-METEOR to evaluate the caption qualities more accurately for the ZSIC problem. In this thesis, we also focus on the FSOD problem and propose the meta-tuning principle, which allows us to model interpretable loss functions/data augmentation magnitudes in few-shot settings. Meta-tuning allows learning inductive biases that boost FSOD as an intermediate learning step using episodic learning. With the proposed RL-based meta-tuning approach, we model the loss function parameters and augmentation magnitudes, and obtain state-of-the-art results in the FSOD problem.

Subject Keywords

Zero-shot, Few-shot, Object Detection, Image Captioning, Meta-tuning

URI

https://hdl.handle.net/11511/102034

Collections

Graduate School of Natural and Applied Sciences, Thesis