How robust are discriminatively trained zero-shot learning models?

Download

index.pdf

Date

2022-3-01

Author

Yucel, Mehmet Kerim
Cinbiş, Ramazan Gökberk
DUYGULU ŞAHİN, PINAR

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

53
views

80
downloads

Data shift robustness has been primarily investigated from a fully supervised perspective, and robustness of zero shot learning (ZSL) models have been largely neglected. In this paper, we present novel analyses on the robustness of discriminative ZSL to image corruptions. We subject several ZSL models to a large set of common corruptions and defenses. In order to realize the corruption analysis, we curate and release the first ZSL corruption robustness datasets SUN-C, CUB-C and AWA2-C. We analyse our results by taking into account the dataset characteristics, class imbalance, class transitions between seen and unseen classes and the discrepancies between ZSL and GZSL performances. Our results show that discriminative ZSL suffers from corruptions and this trend is further exacerbated by the severe class imbalance and model weakness inherent in ZSL methods. We then combine our findings with those based on adversarial attacks in ZSL, and highlight the different effects of corruptions and adversarial examples, such as the pseudo-robustness effect present under adversarial attacks. We also obtain new strong baselines for both models with the defense methods. Finally, our experiments show that although existing methods to improve robustness somewhat work for ZSL models, they do not produce a tangible effect. (c) 2022 Elsevier B.V. All rights reserved.

Subject Keywords

Zero-shot learning, Robust generalization, Adversarial robustness

URI

https://hdl.handle.net/11511/102473

Journal

IMAGE AND VISION COMPUTING

DOI

https://doi.org/10.1016/j.imavis.2022.104392

Collections

Department of Computer Engineering, Article

Suggestions

OpenMETU
Core

RCMARS: Robustification of CMARS with different scenarios under polyhedral uncertainty set Ozmen, Ayse; Weber, Gerhard Wilhelm; Batmaz, İnci; Kropat, Erik (2011-12-01) Our recently developed CMARS is powerful in handling complex and heterogeneous data. We include into CMARS the existence of uncertainty about the scenarios. Indeed, data include noise in both output and input variables. Therefore, solutions of the optimization problem may reveal a remarkable sensitivity to perturbations in the parameters of the problem. The data uncertainty results in uncertain constraints and objective function. To overcome this difficulty, we refine our CMARS algorithm by a robust optimiz...
Universal adversarial perturbations using alternating loss functions Şen, Deniz; Temizel, Alptekin; Department of Modeling and Simulation (2022-8-23) Deep learning models have been the main choice for image classification, however, recently it has been shown that even the most successful models are vulnerable to adversarial attacks. Unlike image-dependent attacks, universal adversarial perturbations can generate an adversarial example when added to any image. These perturbations are usually generated to fool the whole dataset and most successful attacks can reach 100% fooling rate, however they cannot be controlled to stabilize around a desired fooling r...
Robust estimation in multiple linear regression model with non-Gaussian noise Akkaya, Ayşen (2008-02-01) The traditional least squares estimators used in multiple linear regression model are very sensitive to design anomalies. To rectify the situation we propose a reparametrization of the model. We derive modified maximum likelihood estimators and show that they are robust and considerably more efficient than the least squares estimators besides being insensitive to moderate design anomalies.
Improving classification performance of endoscopic images with generative data augmentation Çağlar, Ümit Mert; Temizel, Alptekin; Department of Modeling and Simulation (2022-2-8) The performance of a supervised deep learning model is highly dependent on the quality and variety of the images in the training dataset. In some applications, it may be impossible to obtain more images. Data augmentation methods have been proven to be successful in increasing the performance of deep learning models with limited data. Recent improvements on Generative Adversarial Networks (GAN) algorithms and structures resulted in improved image quality and diversity and made GAN training possible with lim...
Improving Perceptual Quality of Spatially Transformed Adversarial Examples Aydın, Ayberk; Temizel, Alptekin; Department of Modeling and Simulation (2022-8) Deep neural networks are known to be vulnerable to additive adversarial perturbations. The amount of these additive perturbations are generally quantified using Lp metrics over the difference between adversarial and benign examples. However, even when the measured perturbations are small, they tend to be noticeable by human observers since Lp distance metrics are not representative of human perception. Spatially transformed examples work by distorting pixel locations instead of applying an additive perturba...

Citation Formats

M. K. Yucel, R. G. Cinbiş, and P. DUYGULU ŞAHİN, “How robust are discriminatively trained zero-shot learning models?,” IMAGE AND VISION COMPUTING, vol. 119, pp. 0–0, 2022, Accessed: 00, 2023. [Online]. Available: https://hdl.handle.net/11511/102473.