Localization recall precision (LRP): A new performance metric for object detection

Download

index.pdf

Date

2018-09-14

Author

Öksüz, Kemal
Çam, Baris Can
Akbaş, Emre
Kalkan, Sinan

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

173
views

59
downloads

Average precision (AP), the area under the recall-precision (RP) curve, is the standard performance measure for object detection. Despite its wide acceptance, it has a number of shortcomings, the most important of which are (i) the inability to distinguish very different RP curves, and (ii) the lack of directly measuring bounding box localization accuracy. In this paper, we propose “Localization Recall Precision (LRP) Error”, a new metric specifically designed for object detection. LRP Error is composed of three components related to localization, false negative (FN) rate and false positive (FP) rate. Based on LRP, we introduce the “Optimal LRP” (oLRP), the minimum achievable LRP error representing the best achievable configuration of the detector in terms of recall-precision and the tightness of the boxes. In contrast to AP, which considers precisions over the entire recall domain, oLRP determines the “best” confidence score threshold for a class, which balances the trade-off between localization and recall-precision. In our experiments, we show that oLRP provides richer and more discriminative information than AP. We also demonstrate that the best confidence score thresholds vary significantly among classes and detectors. Moreover, we present LRP results of a simple online video object detector and show that the class-specific optimized thresholds increase the accuracy against the common approach of using a general threshold for all classes. Our experiments demonstrate that LRP is more competent than AP in capturing the performance of detectors. Our source code for PASCAL VOC AND MSCOCO datasets are provided at https://github.com/cancam/LRP.

Subject Keywords

Average precision, Object detection, Performance metric, Optimal threshold, Recall-precision

URI

https://hdl.handle.net/11511/41609

DOI

https://doi.org/10.1007/978-3-030-01234-2_31

Collections

Department of Computer Engineering, Conference / Seminar

Suggestions

OpenMETU
Core

One Metric to Measure them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks Oksuz, Kemal; Cam, Baris Can; Kalkan, Sinan; Akbaş, Emre (2021-1-01) Despite being widely used as a performance measure for visual detection tasks, Average Precision (AP) is limited in (i) reflecting localisation quality, (ii) interpretability and (iii) robustness to the design choices regarding its computation, and its applicability to outputs without confidence scores. Panoptic Quality (PQ), a measure proposed for evaluating panoptic segmentation (Kirillov et al., 2019), does not suffer from these limitations but is limited to panoptic segmentation. In this paper, we propo...
Applications of simulated annealing for the design of special digital filters - Comments Çiloğlu, Tolga; Ünver, Zafer (1996-04-01) The way of measuring the performance of a discrete coefficient filter which is designed by scaling optimization is discussed.
Covariance Matrix Estimation of Texture Correlated Compound-Gaussian Vectors for Adaptive Radar Detection Candan, Çağatay; Pascal, Frederic (2022-01-01) Covariance matrix estimation of compound-Gaussian vectors with texture-correlation (spatial correlation for the adaptive radar detectors) is examined. The texture parameters are treated as hidden random parameters whose statistical description is given by a Markov chain. States of the chain represent the value of texture coefficient and the transition probabilities establish the correlation in the texture sequence. An Expectation-Maximization (EM) method based covariance matrix estimation solution is given ...
Damage detection in beam-like structures via combined genetic algorithm and non-linear optimisation Aktaşoğlu, Seyfullah; Şahin, Melih; Department of Aerospace Engineering (2012) In this study, a combined genetic algorithm and non-linear optimisation system is designed and used in the identification of structural damage of a cantilever isotropic beam regarding its location and severity. The vibration-based features, both natural frequencies (i.e. eigenvalues) and displacement mode shapes (i.e. eigenvectors) of the structure in the first two out of plane bending modes, are selected as damage features for various types of damage comprising saw-cut and impact. For this purpose, commerc...
Multitarget tracking performance metric: deficiency aware subpattern assignment Oksuz, Kemal; CEMGİL, ALİ TAYLAN (2018-03-01) Multitarget tracking is a sequential estimation problem where conditioned on noisy sensor measurements, state variables of several targets need to be estimated recursively. In this study, the authors propose a novel performance measure for multitarget tracking named as Deficiency Aware Subpattern Assignment (DASA), that can be used to consistently compare algorithms in a broad spectrum of formulations ranging from conventional data association methods to random finite set based multitarget tracking algorith...

Citation Formats

K. Öksüz, B. C. Çam, E. Akbaş, and S. Kalkan, “Localization recall precision (LRP): A new performance metric for object detection,” 2018, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/41609.