Statistical Indicator Definitions

This document details the formulation of the statistical and complementary indicators used in the derivation of the MQI.

Statistical Performance Indicators

Temporal Indicators

Averaged measured value

$$\bar{O} = \frac{\sum_{t=1}^{N_t} O(t)}{N_{t}}$$

Averaged modelled value

$$\bar{M} = \frac{\sum_{t=1}^{N_t} M(t)}{N_{t}}$$

Bias (vec_bias())

$$BIAS = \bar{M} - \bar{O}$$

Root Mean Square Error (vec_rmse())

$$RMSE = \sqrt{\frac{1}{N_t}\sum_{t=1}^{N_t}{(O(t)-M(t))^2}}$$

Measurement Root Mean Square Uncertainty (vec_rmsu())

$$RMSU_O = \sqrt{\frac{1}{N_t}\sum_{t=1}^{N_t}U^2(O(t))}$$

Correlation coefficient (vec_cor())

$$R = \frac{\sum_{t=1}^{N_t}(M(t)-\bar{M})(O(t)-\bar{O})}{\sqrt{\sum_{t=1}^{N_t}(M(t)-\bar{M})^2} \sqrt{\sum_{t=1}^{N_t}(O(t)-\bar{O})^2}}$$

Measurement standard deviation (vec_sd())

$$\sigma_{O} = \sqrt{\frac{1}{N_t}\sum_{t=1}^{N_t}{(O(t)-\bar{O})^2}}$$

Modelling standard deviation (vec_sd())

$$\sigma_{M} = \sqrt{\frac{1}{N_t}\sum_{t=1}^{N_t}{(M(t)-\bar{M})^2}}$$

Centred Root Mean Square Error (vec_crmse())

$$\sigma_{M} = \sqrt{\frac{1}{N_t}\sum_{t=1}^{N_t}[(M(t)-\bar{M}) - (O(t)-\bar{O})]^2}$$

Model Quality Indicator (vec_mqi())

$$MQI_{short} = \frac{RMSE}{\sqrt{1+\beta^2_{short}}\cdot RMSU_{O}}$$

Spatial Indicators

Averaged measured value (all sampling points)

$$\bar{O}^{sp} = \frac{\sum_{s=1}^{N_s} \bar{O}(t)}{N_{s}}$$

Averaged modelled value (all sampling points)

$$\bar{M}^{sp} = \frac{\sum_{s=1}^{N_s} \bar{M}(t)}{N_{s}}$$

Bias (vec_bias())

$$BIAS^{sp} = \bar{M}^{sp} - \bar{O}^{sp}$$

Root Mean Square Error (vec_rmse())

$$RMSE^{sp} = \sqrt{\frac{1}{N_s}\sum_{s=1}^{N_s}{(\bar{O}(s)-\bar{M}(s))^2}}$$

Measurement Root Mean Square Uncertainty (vec_rmsu())

$$RMSU_\bar{O}^{sp} = \sqrt{\frac{1}{N_s}\sum_{s=1}^{N_s}U_O^2(\bar{O}(t))}$$

Correlation coefficient (vec_cor())

$$R^{sp} = \frac{\sum_{s=1}^{N_s}(\bar{M}(s)-\bar{M}^{sp})(\bar{O}(s)-\bar{O}^{sp})}{\sqrt{\sum_{s=1}^{N_s}(\bar{M}(s)-\bar{M}^{sp})^2} \sqrt{\sum_{s=1}^{N_s}(\bar{O}(s)-\bar{O}^{sp})^2}}$$

Measurement standard deviation (vec_sd())

$$\sigma_{\bar{O}}^{sp} = \sqrt{\frac{1}{N_s-1}\sum_{s=1}^{N_s}{(\bar{O}(s)-\bar{O}^{sp})^2}}$$

Modelling standard deviation (vec_sd())

$$\sigma_{\bar{M}}^{sp} = \sqrt{\frac{1}{N_s-1}\sum_{s=1}^{N_s}{(\bar{M}(s)-\bar{M}^{sp})^2}}$$

Model Quality Indicator (vec_mqi())

$$MQI_{long} = \frac{|BIAS|}{\sqrt{1+\beta^2_{long}}\cdot U_O(\bar{O})}$$

Complementary Performance Indicators

Temporal Indicators

Bias Indicator (objective: $|TI_B|_{90th} \leq 1$) (vec_pi_bias())

$$TI_B = \frac{BIAS}{\sqrt{1+\beta^2_{short}} RMSU_O}$$

Correlation Indicator (objective: $|TI_R|_{90th} \leq 1$) (vec_pi_cor())

$$TI_R = \frac{\sqrt{(1-R)2\sigma_O\sigma_M}}{\sqrt{1+\beta^2_{short}} RMSU_O}$$

Standard Deviation Indicator (objective: $|TI_\sigma|_{90th} \leq 1$) (vec_pi_sd())

$$TI_\sigma = \frac{\sigma_M-\sigma_O}{\sqrt{1+\beta^2_{short}} RMSU_O}$$ Centred Root Mean Square Indicator (vec_ti_crmse())

$$TI_{CRMSE} = \frac{CRMSE}{\sqrt{1+\beta^2_{short}} RMSU_O} = \sqrt{TI_R^2 + TI_\sigma^2} = \sqrt{MQI_{short}^2-TI_B^2}$$

Spatial Indicators

Bias Indicator (objective: $|SI_B| \leq 1$) (vec_pi_bias())

$$SI_B = \frac{BIAS^{sp}}{\sqrt{1+\beta^2_{long} RMSU_\bar{O}^{sp}}}$$

Correlation Indicator (objective: $SI_R \leq 1$) (vec_pi_cor())

$$SI_R = \frac{\sqrt{(1-R^{sp})2\sigma_{\bar{O}^{sp}}\sigma_{\bar{M}^{sp}}}}{\sqrt{1+\beta^2_{long}} RMSU_\bar{O}^{sp}}$$

Standard Deviation Indicator (objective: $|SI_\sigma| \leq 1$) (vec_pi_sd())

$$SI_\sigma = \frac{\sigma_\bar{M}^{sp}-\sigma_\bar{O}^{sp}}{\sqrt{1+\beta^2_{long}} RMSU_\bar{O}^{sp}}$$

Root Mean Square Error Indicator (objective: $SI_{RMSE} \leq 1$) (vec_si_rmse())

$$SI_{RMSE} = \frac{RMSE^{sp}}{\sqrt{1+\beta^2_{long}} RMSU_\bar{O}^{sp}}$$

Statistical Methods

Percentile by Linear Interpolation

In mqo_percentile(), the 90th percentile value is inferred by linear interpolation:

$$MQI_{90th} = MQI(N_{90th}) + (MQI(N_{90th}+1)-MQI(N_{90th})) \cdot D$$

where $N_{90th}$ is the calculated 90th percentile of the number of sampling points defined as the integer part of a given real number (e.g., $floor(3.9)=3$):

$$N_{90th}=floor(0.9N_{s})$$

with $N_s$ the total number of available sampling points and $D$ the non-integer distance defined as:

$$D= 0.9N_{s}-N_{90th}$$

Note: While mqo_percentile() default to the 90th percentile (quantile = 0.9), any quantile between 0 and 1 can be calculated in this way.