Generalized Sum Pooling for Metric Learning

Date

2023-10-02

Author

GÜRBÜZ, YETİ ZİYA
ŞENER, OZAN
Alatan, Abdullah Aydın

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

27
views

0
downloads

Cite This

A common architectural choice for deep metric learning is a convolutional neural network followed by global average pooling ( GAP). Albeit simple, GAP is a highly effective way to aggregate information. One possible explanation for the effectiveness of GAP is considering each feature vector as representing a different semantic entity and GAP as a convex combination of them. Following this perspective, we generalize GAP and propose a learnable generalized sum pooling method (GSP). GSP improves GAP with two distinct abilities: i) the ability to choose a subset of semantic entities, effectively learning to ignore nuisance information, and ii) learning the weights corresponding to the importance of each entity. Formally, we propose an entropy-smoothed optimal transport problem and show that it is a strict generalization of GAP, i. e., a specific realization of the problem gives back GAP. We show that this optimization problem enjoys analytical gradients enabling us to use it as a direct learnable replacement for GAP. We further propose a zero-shot loss to ease the learning of GSP. We show the effectiveness of our method with extensive evaluations on 4 popular metric learning benchmarks. Code is available at: GSP-DML Framework

URI

https://openaccess.thecvf.com/ICCV2023?day=all
https://hdl.handle.net/11511/111851

Collections

Department of Electrical and Electronics Engineering, Conference / Seminar