Comparison of Subjective and Objective Evaluation Methods for Audio Source Separation

Download

index.pdf

Date

2008-01-01

Author

Josef, Kornycky
Günel Kılıç, Banu
Ahmet, Kondoz

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

184
views

0
downloads

The evaluation of audio separation algorithms can either be performed objectively by calculation of numerical measures, or subjectively through listening tests. Although objective evaluation is inherently more straightforward, subjective listening tests are still essential in determining the perceived quality of separation. This paper aims to ﬁnd relationships between objective and subjective results so that numerical values can be translated into perceptual criteria. A generic audio source separation system was modelled which provided varying levels of interference, noise and artifacts. This enabled a full spread of objective measurement values to be obtained. Extensive tests were performed utilising the output synthesised by this separation model. The relationships found were presented and the factors of prime importance were determined.

Subject Keywords

Blind source separation, Listening test, Performance measures, BSS modelling system

URI

https://hdl.handle.net/11511/32362

Journal

Proceedings of Meetings on Acoustics

DOI

https://doi.org/10.1121/1.2972136

Collections

Graduate School of Informatics, Article

Suggestions

OpenMETU
Core

Multi-perspective analysis and systematic benchmarking for binary-classification performance evaluation instruments Canbek, Gürol; Taşkaya Temizel, Tuğba; Department of Information Systems (2019) This thesis proposes novel methods to analyze and benchmark binary-classification performance evaluation instruments. It addresses critical problems found in the literature, clarifies terminology and distinguishes instruments as measure, metric, and as a new category indicator for the first time. The multi-perspective analysis introduces novel concepts such as canonical form, geometry, duality, complementation, dependency, and leveling with formal definitions as well as two new basic instruments. An indicat...
Alternative Approach to Maurer's Universal Statistical Test Tezcan, Cihangir; Doğanaksoy, Ali (null; 2008-12-01) Statistical tests for randomness play an important role in cryptography since many cryptographic applications require random or pseudorandom numbers. In this study, we introduce an alternative approach to Maurer’s Universal Test. This approach allows us to test short binary sequences as small as 66 bits and to choose slightly larger block sizes. Moreover, it does not have an initialization part and requires less time to test a binary sequence.
Quantitative measure of observability for linear stochastic systems Subasi, Yuksel; Demirekler, Mübeccel (Elsevier BV, 2014-06-01) In this study we define a new observability measure for stochastic systems: the mutual information between the state sequence and the corresponding measurement sequence for a given time horizon. Although the definition is given for a general system representation, the paper focuses on the linear time invariant Gaussian case. Some basic analytical results are derived for this special case. The measure is extended to the observability of a subspace of the state space, specifically an individual state and/or t...
Implementation of an enhanced fixed point Variable Bit-Rate MELP vocoder on TMS320C549 Ertan, AE; Aksu, EB; Ilk, HG; Karci, H; Karpat, O; Kolcak, T; Sendur, L; Demirekler, Mübeccel; Cetin, AE (1999-03-19) In this paper, a fixed point Variable Bit-Rate (VBR) Mixed Excitation Linear Predictive Coding (MELP(TM)) vocoder is presented. The VBR-MELP vocoder is also implemented on a TMS320C54x and it achieves virtually indistinguishable federal standard MELP quality at bit-rates between 1.0 to 1.6 kb/s. The backbone of VBR-MELP vocoder is similar to that of federal standard MELP. It utilizes a novel sub-band based voice activity detector in the back-end of encoder to discriminate background noise from speech activi...
Evaluation of visual quality metrics Ölgün, Ramazan Ferhat; Akar, Gözde; Department of Electrical and Electronics Engineering (2011) The aim of this study is to work on the visual quality metrics that are widely accepted in literature, to evaluate them on different distortion types and to give a comparison of overall performances in terms of prediction accuracy, monotonicity, consistency and complexity. The algorithms behind the quality metrics in literature and parameters used for quality metric performance evaluations are studied. This thesis also includes the explanation of Human Visual System, classification of visual quality metrics...

Citation Formats

K. Josef, B. Günel Kılıç, and K. Ahmet, “Comparison of Subjective and Objective Evaluation Methods for Audio Source Separation,” Proceedings of Meetings on Acoustics, pp. 0–0, 2008, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/32362.