Show/Hide Menu
Hide/Show Apps
Logout
Türkçe
Türkçe
Search
Search
Login
Login
OpenMETU
OpenMETU
About
About
Open Science Policy
Open Science Policy
Communities & Collections
Communities & Collections
Help
Help
Frequently Asked Questions
Frequently Asked Questions
Guides
Guides
Thesis submission
Thesis submission
MS without thesis term project submission
MS without thesis term project submission
Publication submission with DOI
Publication submission with DOI
Publication submission
Publication submission
Supporting Information
Supporting Information
General Information
General Information
Copyright, Embargo and License
Copyright, Embargo and License
Contact us
Contact us
Localization recall precision (LRP): A new performance metric for object detection
Download
index.pdf
Date
2018-09-14
Author
Öksüz, Kemal
Çam, Baris Can
Akbaş, Emre
Kalkan, Sinan
Metadata
Show full item record
This work is licensed under a
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
.
Item Usage Stats
104
views
35
downloads
Cite This
Average precision (AP), the area under the recall-precision (RP) curve, is the standard performance measure for object detection. Despite its wide acceptance, it has a number of shortcomings, the most important of which are (i) the inability to distinguish very different RP curves, and (ii) the lack of directly measuring bounding box localization accuracy. In this paper, we propose “Localization Recall Precision (LRP) Error”, a new metric specifically designed for object detection. LRP Error is composed of three components related to localization, false negative (FN) rate and false positive (FP) rate. Based on LRP, we introduce the “Optimal LRP” (oLRP), the minimum achievable LRP error representing the best achievable configuration of the detector in terms of recall-precision and the tightness of the boxes. In contrast to AP, which considers precisions over the entire recall domain, oLRP determines the “best” confidence score threshold for a class, which balances the trade-off between localization and recall-precision. In our experiments, we show that oLRP provides richer and more discriminative information than AP. We also demonstrate that the best confidence score thresholds vary significantly among classes and detectors. Moreover, we present LRP results of a simple online video object detector and show that the class-specific optimized thresholds increase the accuracy against the common approach of using a general threshold for all classes. Our experiments demonstrate that LRP is more competent than AP in capturing the performance of detectors. Our source code for PASCAL VOC AND MSCOCO datasets are provided at https://github.com/cancam/LRP.
Subject Keywords
Average precision
,
Object detection
,
Performance metric
,
Optimal threshold
,
Recall-precision
URI
https://hdl.handle.net/11511/41609
DOI
https://doi.org/10.1007/978-3-030-01234-2_31
Collections
Department of Computer Engineering, Conference / Seminar
Suggestions
OpenMETU
Core
One Metric to Measure them All: Localisation Recall Precision (LRP) for Evaluating Visual Detection Tasks
Oksuz, Kemal; Cam, Baris Can; Kalkan, Sinan; Akbaş, Emre (2021-1-01)
Despite being widely used as a performance measure for visual detection tasks, Average Precision (AP) is limited in (i) reflecting localisation quality, (ii) interpretability and (iii) robustness to the design choices regarding its computation, and its applicability to outputs without confidence scores. Panoptic Quality (PQ), a measure proposed for evaluating panoptic segmentation (Kirillov et al., 2019), does not suffer from these limitations but is limited to panoptic segmentation. In this paper, we propo...
Applications of simulated annealing for the design of special digital filters - Comments
Çiloğlu, Tolga; Ünver, Zafer (1996-04-01)
The way of measuring the performance of a discrete coefficient filter which is designed by scaling optimization is discussed.
Region of Interest Detection Based Fast and Robust Geo-Spatial Object Recognition
Gürbüz, Yeti Ziya; Alatan, Abdullah Aydın (2013-01-01)
In this paper a novel computationally efficient algorithm to detect objects automatically from high definition satellite imagery with high performance is presented. The proposed algorithm has three main steps supporting each other: Filtering, shape based and appearance based object detection. A region of interest indicating the possible regions that may have the objects to be detected is determined in a very short time via filtering step. In the remaining steps, the objects are extracted from that region an...
Multitarget tracking performance metric: deficiency aware subpattern assignment
Oksuz, Kemal; CEMGİL, ALİ TAYLAN (2018-03-01)
Multitarget tracking is a sequential estimation problem where conditioned on noisy sensor measurements, state variables of several targets need to be estimated recursively. In this study, the authors propose a novel performance measure for multitarget tracking named as Deficiency Aware Subpattern Assignment (DASA), that can be used to consistently compare algorithms in a broad spectrum of formulations ranging from conventional data association methods to random finite set based multitarget tracking algorith...
Multi-perspective analysis and systematic benchmarking for binary-classification performance evaluation instruments
Canbek, Gürol; Taşkaya Temizel, Tuğba; Department of Information Systems (2019)
This thesis proposes novel methods to analyze and benchmark binary-classification performance evaluation instruments. It addresses critical problems found in the literature, clarifies terminology and distinguishes instruments as measure, metric, and as a new category indicator for the first time. The multi-perspective analysis introduces novel concepts such as canonical form, geometry, duality, complementation, dependency, and leveling with formal definitions as well as two new basic instruments. An indicat...
Citation Formats
IEEE
ACM
APA
CHICAGO
MLA
BibTeX
K. Öksüz, B. C. Çam, E. Akbaş, and S. Kalkan, “Localization recall precision (LRP): A new performance metric for object detection,” 2018, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/41609.