Show/Hide Menu
Hide/Show Apps
Logout
Türkçe
Türkçe
Search
Search
Login
Login
OpenMETU
OpenMETU
About
About
Open Science Policy
Open Science Policy
Open Access Guideline
Open Access Guideline
Postgraduate Thesis Guideline
Postgraduate Thesis Guideline
Communities & Collections
Communities & Collections
Help
Help
Frequently Asked Questions
Frequently Asked Questions
Guides
Guides
Thesis submission
Thesis submission
MS without thesis term project submission
MS without thesis term project submission
Publication submission with DOI
Publication submission with DOI
Publication submission
Publication submission
Supporting Information
Supporting Information
General Information
General Information
Copyright, Embargo and License
Copyright, Embargo and License
Contact us
Contact us
Accuracy Barrier (ACCBAR): A novel performance indicator for binary classification
Date
2022-01-01
Author
Canbek, Gurol
Taşkaya Temizel, Tuğba
SAĞIROĞLU, ŞEREF
Metadata
Show full item record
This work is licensed under a
Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License
.
Item Usage Stats
110
views
0
downloads
Cite This
Although several binary classification performance metrics have been defined, a few of them are used for performance evaluation of classifiers and performance comparison/reporting in the literature. Specifically, F1 and Accuracy (ACC) are the most known and conventionally used metrics. Despite their popularity and easy-to-understand characteristics, those metrics exhibit critical robustness issues. This paper suggests a new instrument category named 'performance indicators' and proposes a novel indicator named accuracy barrier (ACCBAR for short) that works to uncover confounding problems in performance reporting of ACC metric. The given case study in mobile malware classification, which is a domain of cyber security, has shown that the indicator gives an accurate interpretation of the results presented in terms of ACC. This study also recommends that researchers should use ACCBAR to eliminate potential publication or confirmation bias in classification performance evaluation.
Subject Keywords
confirmation bias
,
performance evaluation
,
performance indicators
,
performance measures
,
publication bias
URI
https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85142796888&origin=inward
https://hdl.handle.net/11511/101495
DOI
https://doi.org/10.1109/iscturkey56345.2022.9931888
Conference Name
15th International Conference on Information Security and Cryptography, ISCTURKEY 2022
Collections
Graduate School of Informatics, Conference / Seminar
Suggestions
OpenMETU
Core
PToPI: A Comprehensive Review, Analysis, and Knowledge Representation of Binary Classification Performance Measures/Metrics
Canbek, Gürol; Taşkaya Temizel, Tuğba; SAĞIROĞLU, ŞEREF (2023-1-01)
Although few performance evaluation instruments have been used conventionally in different machine learning-based classification problem domains, there are numerous ones defined in the literature. This study reviews and describes performance instruments via formally defined novel concepts and clarifies the terminology. The study first highlights the issues in performance evaluation via a survey of 78 mobile-malware classification studies and reviews terminology. Based on three research questions, it propose...
Multi-perspective analysis and systematic benchmarking for binary-classification performance evaluation instruments
Canbek, Gürol; Taşkaya Temizel, Tuğba; Department of Information Systems (2019)
This thesis proposes novel methods to analyze and benchmark binary-classification performance evaluation instruments. It addresses critical problems found in the literature, clarifies terminology and distinguishes instruments as measure, metric, and as a new category indicator for the first time. The multi-perspective analysis introduces novel concepts such as canonical form, geometry, duality, complementation, dependency, and leveling with formal definitions as well as two new basic instruments. An indicat...
Power-delay optimized VLSI threshold detection circuits and their use in parallel integer multiplication
Ercan, Furkan; Muhtaroğlu, Ali; Sustainable Environment and Energy Systems (2015-6)
Threshold detection is a fundamental logic function that has broad use in arithmetic processors, and other digital applications. Thus, any improvement in threshold detection in terms of power and/or delay contributes significantly to the field of digital circuit design. A recently reported parallel integer multiplier architecture, ABACUS, uses column compression networks to compress partial products through the final addition network. Architecture of column compression network of ABACUS is suitable for thre...
Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling
Gupta, Hoshin V.; Kling, Harald; Yılmaz, Koray Kamil; Martinez, Guillermo F. (2009-10-20)
The mean squared error (MSE) and the related normalization, the Nash-Sutcliffe efficiency (NSE), are the two criteria most widely used for calibration and evaluation of hydrological models with observed data. Here, we present a diagnostically interesting decomposition of NSE (and hence MSE), which facilitates analysis of the relative importance of its different components in the context of hydrological modelling, and show how model calibration problems can arise due to interactions among these components. T...
Identification and prioritization of stage-level KPIs for BOT projects - evidence from Turkey
BUDAYAN, Cenk; OKUDAN, Ozan; Dikmen Toker, İrem (2020-09-01)
Purpose The purpose of this paper is to identify and prioritize key performance indicators (KPIs) that can be used for stage-based performance assessment of build-operate-transfer (BOT) projects. Design/methodology/approach This research was conducted through focus group discussions and face-to-face questionnaires. Firstly, stage-level KPIs for BOT projects were identified by conducting a literature survey. The list of KPIs that can be used for measuring performance at different stages of a BOT project was ...
Citation Formats
IEEE
ACM
APA
CHICAGO
MLA
BibTeX
G. Canbek, T. Taşkaya Temizel, and Ş. SAĞIROĞLU, “Accuracy Barrier (ACCBAR): A novel performance indicator for binary classification,” presented at the 15th International Conference on Information Security and Cryptography, ISCTURKEY 2022, Ankara, Türkiye, 2022, Accessed: 00, 2023. [Online]. Available: https://www.scopus.com/inward/record.uri?partnerID=HzOxMe3b&scp=85142796888&origin=inward.