Adversarial segmentation loss for sketch colorization

Download

index.pdf

Date

2021-01-01

Author

Hicsonmez, Samet
Samet, Nermin
Akbaş, Emre
DUYGULU ŞAHİN, PINAR

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

81
views

55
downloads

We introduce a new method for generating color images from sketches or edge maps. Current methods either require some form of additional user-guidance or are limited to the “paired” translation approach. We argue that segmentation information could provide valuable guidance for sketch colorization. To this end, we propose to leverage semantic image segmentation, as provided by a general purpose panoptic segmentation network, to create an additional adversarial loss function. Our loss function can be integrated to any baseline GAN model. Our method is not limited to datasets that contain segmentation labels, and it can be trained for “unpaired” translation tasks. We show the effectiveness of our method on four different datasets spanning scene level indoor, outdoor, and children book illustration images using qualitative, quantitative and user study analysis. Our model improves its baseline up to 35 points on the FID metric. Our code and pretrained models can be found at https://github.com/giddyyupp/AdvSegLoss.

Subject Keywords

sketch colorization, sketch to image translation, Generative Adversarial Networks (GAN), image segmentation, image to image translation, Generative adversarial networks (GAN), Image segmentation, Image to image translation, Sketch colorization, Sketch to image translation

URI

https://hdl.handle.net/11511/99684

DOI

https://doi.org/10.1109/icip42928.2021.9506637

Conference Name

2021 IEEE International Conference on Image Processing, ICIP 2021

Collections

Department of Computer Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

PROGRESSIVE COMPRESSION OF DIGITAL ELEVATION DATA USING MESHES Kose, Kivanc; Yılmaz, Erdal; ÇETİN, AHMET ENİS (2009-07-17) In this paper a new Digital Elevation Map (DEM) image compression algorithm is proposed. DEM image can be threated as a grayscale image, whose pixel values are the elevation values of the map points. The grayscale DEM image is compressed using an adaptive wavelet based image compression algorithm. The method, which is an extension of the progressive mesh compression takes advantage of the multiresolution property of the wavelets while coding the map images. This makes it possible to decode different resolut...
HANOLISTIC: A Hierarchical Automatic Image Annotation System Using Holistic Approach Karadag, Ozge Oztimur; Yarman Vural, Fatoş Tunay (2009-06-25) Automatic image annotation is the process of assigning keywords to digital images depending on the content information. In one sense, it is a mapping from the visual content information to the semantic context information. In this study, we propose a novel approach for automatic image annotation problem, where the annotation is formulated as a multivariate mapping from a set of independent descriptor spaces, representing a whole image, to a set of words, representing class labels. For this purpose, a hierar...
Image annotation with semi-supervised clustering Sayar, Ahmet; Yarman Vural, Fatoş Tunay; Department of Computer Engineering (2009) Image annotation is defined as generating a set of textual words for a given image, learning from the available training data consisting of visual image content and annotation words. Methods developed for image annotation usually make use of region clustering algorithms to quantize the visual information. Visual codebooks are generated from the region clusters of low level visual features. These codebooks are then, matched with the words of the text document related to the image, in various ways. In this th...
Change detection in aerial images Borchani, M; Cloppet, F; Atalay, Mehmet Volkan; Stamon, G (2004-01-01) This paper deals with how to characterize texture and how to get a good description of images with a minimal number of parameters. This procedure is more objective than textual data. Texture characterization has been used in a matching system to detect changes in couples of aerial images taken at two different times using different order of statistics to describe images. The results are quite encouraging.
Sea Detection on High-Resolution Panchromatic Satellite Images Using Texture and Intensity Besbinar, Beril; Alatan, Abdullah Aydın (2014-01-01) In this paper, a two-stage sea-land mask detection algorithm on high resolution panchromatic images is proposed. An initial mask is generated using texture features in the first stage and this mask is refined by using intensity values in the second stage. Image is divided into windows and the Local Binary Patterns (LBP) histograms, evaluated at each window, are modelled using the sea and land sample spaces obtained by the altitude information which has very low resolution compared to the image. These models...

Citation Formats

S. Hicsonmez, N. Samet, E. Akbaş, and P. DUYGULU ŞAHİN, “Adversarial segmentation loss for sketch colorization,” Alaska, Amerika Birleşik Devletleri, 2021, vol. 2021-September, Accessed: 00, 2022. [Online]. Available: https://hdl.handle.net/11511/99684.