Object Augmentation for Out-of-Context Object Recognition

Download

10468316.pdf

Date

2022-5-18

Author

Eryüksel, Oğul Can

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

162
views

245
downloads

The visual context in an image contains rich information about and between foreground objects and the background. Deep learning models learn contextual information implicitly in general. However, since training datasets generally do not include all possible contexts, deep models tend to memorize contextual details. This can lead to poor recognition performance when models are deployed in real-world applications since objects may appear in unexpected contexts or places. These types of objects are called out-of-context objects. In this work, we propose an object-level augmentation framework for more robust recognition of out-of-context objects. Our proposed augmentation methodology applies object removal and object placement operations to images during the training phase. Moreover, we proposed a contrastive learning pipeline using object-level augmentations to increase performance further. Our results show that, by using object-level augmentations and contrastive learning, the out-of-context recognition performance of models can increase without losing performance on regular images. To analyze the effectiveness of the proposed method, we conducted a series of experiments for a multi-label image classification problem on the MS COCO dataset. Moreover, we provide a tool to generate images with out-of-context objects using the proposed augmentation framework.

Subject Keywords

Deep learning, Computer Vision, out-of-context objects, object removal, object placement, augmentation, multi-label classification

URI

https://hdl.handle.net/11511/97753

Collections

Graduate School of Natural and Applied Sciences, Thesis

Suggestions

OpenMETU
Core

Rescoring detections based on contextual scores in object detection Zorlu, Ersan Vural; Akbaş, Emre; Department of Computer Engineering (2019) To detect objects in an image, current state-of-the-art object detectors firstly definecandidate object locations, and then classify each of them into one of the predefinedcategories or as background. They do so by using the visual features extracted locallyfrom the candidate locations; omitting the rich contextual information embedded inthe whole image. Contextual information can be utilized to complement the informa-tion extracted locally and thereby to improve object detection accuracy. Researchershave p...
Object Detection with Convolutional Context Features Kaya, Emre Can; Alatan, Abdullah Aydın (2017-01-01) A novel extension to Huh B-ESA object detection algorithm is proposed in order to learn convolutional context features for determining boundaries of objects better. For input images, the hypothesis windows and their context around those windows are learned through convolutional layers as two parallel networks. The resulting object and context feature maps are combined in such a way that they preserve their spatial relationship. The proposed algorithm is trained and evaluated on PASCAL VOC 2007 detection ben...
OBJECT LOCALIZATION WITHOUT BOUNDING BOX INFORMATION USING GENERATIVE ADVERSARIAL REINFORCEMENT LEARNING Halici, Eren; Alatan, Abdullah Aydın (2018-10-10) Object localization can be defined as the task of finding the bounding boxes of objects in a scene. Most of the state-of-the-art approaches utilize meticulously handcrafted training datasets. In this work, we are aiming to create a generative adversarial reinforcement learning framework, which can work without having any explicit bounding box information. Instead of relying on bounding boxes, our framework uses tightly cropped object images as training data. Our image localization framework consists of two ...
Contextual Object Detection Using Set-Based Classification Cinbiş, Ramazan Gökberk (2012-01-01) We propose a new model for object detection that is based on set representations of the contextual elements. In this formulation, relative spatial locations and relative scores between pairs of detections are considered as sets of unordered items. Directly training classification models on sets of unordered items, where each set can have varying cardinality can be difficult. In order to overcome this problem, we propose SetBoost, a discriminative learning algorithm for building set classifiers. The SetBoost...
Fine-grained object recognition and zero-shot learning in multispectral imagery Sumbul, Gencer; Cinbiş, Ramazan Gökberk; AKSOY, SELİM (2018-05-05) We present a method for fine-grained object recognition problem, that aims to recognize the type of an object among a large number of sub-categories, and zero-shot learning scenario on multispectral images. In order to establish a relation between seen classes and new unseen classes, a compatibility function between image features extracted from a convolutional neural network and auxiliary information of classes is learnt. Knowledge transfer for unseen classes is carried out by maximizing this function. Per...

Citation Formats

O. C. Eryüksel, “Object Augmentation for Out-of-Context Object Recognition,” M.S. - Master of Science, Middle East Technical University, 2022.