Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling

Download

index.pdf

Date

2009-10-20

Author

Gupta, Hoshin V.
Kling, Harald
Yılmaz, Koray Kamil
Martinez, Guillermo F.

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

373
views

2600
downloads

The mean squared error (MSE) and the related normalization, the Nash-Sutcliffe efficiency (NSE), are the two criteria most widely used for calibration and evaluation of hydrological models with observed data. Here, we present a diagnostically interesting decomposition of NSE (and hence MSE), which facilitates analysis of the relative importance of its different components in the context of hydrological modelling, and show how model calibration problems can arise due to interactions among these components. The analysis is illustrated by calibrating a simple conceptual precipitation-runoff model to daily data for a number of Austrian basins having a broad range of hydro-meteorological characteristics. Evaluation of the results clearly demonstrates the problems that can be associated with any calibration based on the NSE (or MSE) criterion. While we propose and test an alternative criterion that can help to reduce model calibration problems, the primary purpose of this Study is not to present an improved measure of model performance. Instead, we seek to show that there are systematic problems inherent with any optimization based on formulations related to the MSE. The analysis and results have implications to the manner in which we calibrate and evaluate environmental models, we discuss these and suggest possible ways forward that may move us towards an improved and diagnostically meaningful approach to model performance evaluation and identification.

Subject Keywords

Criteria decomposition, Multiple criteria, Calibration, Model performance evaluation, Nash-Sutcliffe efficiency, Mean squared error

URI

https://hdl.handle.net/11511/42626

Journal

JOURNAL OF HYDROLOGY

DOI

https://doi.org/10.1016/j.jhydrol.2009.08.003

Collections

Department of Geological Engineering, Article

Suggestions

OpenMETU
Core

Generalizing the Sampling Property of the Q-function for Error Rate Analysis of Cooperative Communication in Fading Channels- Aktas, Tugcan; Yılmaz, Ali Özgür; Aktas, Emre (2013-07-12) This paper extends some approximation methods that are used to identify closed form Bit Error Rate (BER) expressions which are frequently utilized in investigation and comparison of performance for wireless communication systems in the literature. By using this group of approximation methods, some expectation integrals, whose exact analyses are intractable and whose Monte Carlo simulation computations have high complexity, can be computed. For these integrals, by using the sampling property of the integrand...
Modeling the effect of key cathode design parameters on the electrochemical performance of a lithium-sulfur battery Erisen, Nisa; Emerce, Nur Ber; Erensoy, Sevgi Can; Eroglu, Damla (2018-06-25) A 1D model is developed for the Li-S cell to predict the effect of critical cathode design parameterscarbon-to-sulfur (C/S) and electrolyte-to-sulfur (E/S) ratios in the cathodeon the electrochemical performance. Cell voltage at 60% depth of discharge corresponding to the lower voltage plateau is used as a metric for calculating the cell performance. The cathode kinetics in the lower voltage plateau is defined with a single electrochemical reaction; thus, the model has a single apparent kinetic model parame...
Robust estimation in multivariate heteroscedastic regression models with autoregressive covariance structures using EM algorithm GÜNEY, YEŞİM; ARSLAN, OLÇAY; Gökalp Yavuz, Fulya (2022-09-01) © 2022 Elsevier Inc.In the analysis of repeated or clustered measurements, it is crucial to determine the dynamics that affect the mean, variance, and correlations of the data, which will be possible using appropriate models. One of these models is the joint mean–covariance model, which is a multivariate heteroscedastic regression model with autoregressive covariance structures. In these models, parameter estimation is usually carried on under normality assumption, but the resulting estimators will be very ...
Some Inequalities Between Pairs of Marginal and Joint Bayesian Lower Bounds Bacharach, Lucien; Chaumette, Eric; Fritsche, Carsten; Orguner, Umut (2019-01-01) In this paper, tightness relations (or inequalities) between Bayesian lower bounds (BLBs) on the mean-squared-error are derived which result from the marginalization of a joint probability density function (pdf) depending on both parameters of interest and extraneous or nuisance parameters. In particular, it is shown that for a large class of BLBs, the BLB derived from the marginal pdf is at least as tight as the corresponding BLB derived from the joint pdf. A Bayesian linear regression example is used to i...
Evaluation of Assumptions in Soil Moisture Triple Collocation Analysis Yılmaz, Mustafa Tuğrul (American Meteorological Society, 2014-06-01) Triple collocation analysis (TCA) enables estimation of error variances for three or more products that retrieve or estimate the same geophysical variable using mutually independent methods. Several statistical assumptions regarding the statistical nature of errors (e.g., mutual independence and orthogonality with respect to the truth) are required for TCA estimates to be unbiased. Even though soil moisture studies commonly acknowledge that these assumptions are required for an unbiased TCA, no study has sp...

Citation Formats

H. V. Gupta, H. Kling, K. K. Yılmaz, and G. F. Martinez, “Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling,” JOURNAL OF HYDROLOGY, pp. 80–91, 2009, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/42626.