Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery

Download

index.pdf

Date

2018-02-01

Author

Sumbul, Gencer
Cinbiş, Ramazan Gökberk
Aksoy, Selim

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

232
views

121
downloads

Fine-grained object recognition that aims to identify the type of an object among a large number of subcategories is an emerging application with the increasing resolution that exposes new details in image data. Traditional fully supervised algorithms fail to handle this problem where there is low betweenclass variance and high within-class variance for the classes of interest with small sample sizes. We study an even more extreme scenario named zero-shot learning (ZSL) in which no training example exists for some of the classes. ZSL aims to build a recognition model for new unseen categories by relating them to seen classes that were previously learned. We establish this relation by learning a compatibility function between image features extracted via a convolutional neural network and auxiliary information that describes the semantics of the classes of interest by using training samples from the seen classes. Then, we show how knowledge transfer can be performed for the unseen classes by maximizing this function during inference. We introduce a new data set that contains 40 different types of street trees in 1-ft spatial resolution aerial data, and evaluate the performance of this model with manually annotated attributes, a natural language model, and a scientific taxonomy as auxiliary information. The experiments show that the proposed model achieves 14.3% recognition accuracy for the classes with no training examples, which is significantly better than a random guess accuracy of 6.3% for 16 test classes, and three other ZSL algorithms.

Subject Keywords

Fine-grained classification, Object recognition, Zero-shot learning (ZSL)

URI

https://hdl.handle.net/11511/57907

Journal

IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING

DOI

https://doi.org/10.1109/tgrs.2017.2754648

Collections

Department of Computer Engineering, Article

Suggestions

OpenMETU
Core

Fine-grained object recognition and zero-shot learning in multispectral imagery Sumbul, Gencer; Cinbiş, Ramazan Gökberk; AKSOY, SELİM (2018-05-05) We present a method for fine-grained object recognition problem, that aims to recognize the type of an object among a large number of sub-categories, and zero-shot learning scenario on multispectral images. In order to establish a relation between seen classes and new unseen classes, a compatibility function between image features extracted from a convolutional neural network and auxiliary information of classes is learnt. Knowledge transfer for unseen classes is carried out by maximizing this function. Per...
Multisource region attention network for fine-grained object recognition in remote sensing imagery Sümbül, Gencer; Cinbiş, Ramazan Gökberk; Aksoy, Selim (Institute of Electrical and Electronics Engineers (IEEE), 2019-07) Fine-grained object recognition concerns the identification of the type of an object among a large number of closely related subcategories. Multisource data analysis that aims to leverage the complementary spectral, spatial, and structural information embedded in different sources is a promising direction toward solving the fine-grained recognition problem that involves low between-class variance, small training set sizes for rare classes, and class imbalance. However, the common assumption of coregistered ...
Weakly supervised instance attention for multisource fine-grained object recognition with an application to tree species classification Aygunes, Bulut; Cinbiş, Ramazan Gökberk; Aksoy, Selim (2021-06-01) Multisource image analysis that leverages complementary spectral, spatial, and structural information benefits fine-grained object recognition that aims to classify an object into one of many similar subcategories. However, for multisource tasks that involve relatively small objects, even the smallest registration errors can introduce high uncertainty in the classification process. We approach this problem from a weakly supervised learning perspective in which the input images correspond to larger neighborh...
Automated learning rate search using batch-level cross-validation Kabakcı, Duygu; Akbaş, Emre; Department of Computer Engineering (2019) Deep convolutional neural networks are being widely used in computer vision tasks, such as object recognition and detection, image segmentation and face recognition, with a variety of architectures. Deep learning researchers and practitioners have accumulated a significant amount of experience on training a wide variety of architectures on various datasets. However, given a specific network model and a dataset, obtaining the best model (i.e. the model giving the smallest test set error) while keeping the tr...
Rescoring detections based on contextual scores in object detection Zorlu, Ersan Vural; Akbaş, Emre; Department of Computer Engineering (2019) To detect objects in an image, current state-of-the-art object detectors firstly definecandidate object locations, and then classify each of them into one of the predefinedcategories or as background. They do so by using the visual features extracted locallyfrom the candidate locations; omitting the rich contextual information embedded inthe whole image. Contextual information can be utilized to complement the informa-tion extracted locally and thereby to improve object detection accuracy. Researchershave p...

Citation Formats

G. Sumbul, R. G. Cinbiş, and S. Aksoy, “Fine-Grained Object Recognition and Zero-Shot Learning in Remote Sensing Imagery,” IEEE TRANSACTIONS ON GEOSCIENCE AND REMOTE SENSING, pp. 770–779, 2018, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/57907.