Statistics underlying summary diagrams

Computes the statistics used as coordinates in Taylor, solar, and target diagrams without constructing a plot. This is useful for inspecting the numerical quantities represented by the diagrams or for constructing custom visualizations.

Usage

diagram_stats(mods, obs, na.rm = TRUE)

Arguments

mods: Numeric vector, list of numeric vectors, or numeric matrix/data frame with one model per column. Rows match obs in order. Supplied model names must be unique; missing names are generated.
obs: A numeric observation vector.
na.rm: Logical; remove incomplete observation-prediction pairs separately for each model? If FALSE, missing pairs cause an error because diagram coordinates cannot be calculated. The default is TRUE.

Value

A data frame with one row per model and the following columns:

model: Model identifier.
n: Number of complete observation-prediction pairs.
r: Pearson correlation coefficient.
sd_ratio: Ratio of prediction to observation standard deviation.
mean_error: Mean error, calculated as observation minus prediction, in the original response units.
nME: Normalized mean error (ME*), obtained by dividing mean error by the population-moment standard deviation of the observations.
sde: Standardized standard deviation of the error (SDE*), obtained by dividing the population-moment standard deviation of the errors by the population-moment standard deviation of the observations.
signed_sde: SDE* multiplied by the sign of the difference between prediction and observation standard deviations.

Details

Inputs are paired by position. At least two complete pairs and non-zero observation standard deviation are required for every model.

For the diagram geometry, standard deviations are calculated as population moments, using divisor $n$, rather than the $n - 1$ sample standard deviation returned by stats::sd(). This convention makes the normalized error decomposition exact for finite samples.

Let

$$e_i = obs_i - pred_i$$

denote the prediction error, and let $\sigma_p$ and $\sigma_o$ denote the population-moment standard deviations of predictions and observations, respectively.

The standard-deviation ratio is

$$ \sigma^* = \frac{\sigma_p}{\sigma_o}. $$

The normalized mean error is

$$ \mathrm{ME}^* = \frac{\bar{e}}{\sigma_o}. $$

Positive ME* indicates underprediction and negative ME* indicates overprediction under the package convention obs - pred.

Let $\sigma_e$ denote the population-moment standard deviation of the errors. The standardized error standard deviation is

$$ \mathrm{SDE}^* = \frac{\sigma_e}{\sigma_o}. $$

Using the relationship between the variances of observations, predictions, and their errors, this is equivalently

$$ \mathrm{SDE}^* = \sqrt{ 1 + \sigma^{*2} - 2\sigma^*r }. $$

The sign used for signed_sde indicates whether the prediction standard deviation is smaller or larger than the observation standard deviation: negative when $\sigma_p < \sigma_o$ and positive when $\sigma_p \geq \sigma_o$. Equal standard deviations therefore receive a positive sign, following the original diagram implementation.

Constant predictions have undefined Pearson correlation and are returned with Pearson correlation set to NA, an SD ratio of zero, and SDE equal to one. Their diagram geometry remains defined even though their correlation is not.

With this common population-moment normalization, the exact finite-sample relationship is

$$ \frac{\mathrm{RMSE}^2}{\sigma_o^2} = \mathrm{ME}^{*2} + \mathrm{SDE}^{*2}. $$

Thus Euclidean distance from the origin in the solar and target diagrams is exactly RMSE normalized by the population-moment observation standard deviation.

References

Wadoux, A. M. J.-C., Walvoort, D. J. J., and Brus, D. J. (2022). An integrated approach for the evaluation of quantitative soil maps through Taylor and solar diagrams. Geoderma, 405, 115332. doi:10.1016/j.geoderma.2021.115332

Examples

obs <- c(1, 2, 3, 4, 5)

mods <- list(
  perfect = obs,
  biased = obs + 1,
  noisy = c(1, 3, 2, 5, 4)
)

diagram_stats(mods, obs)
#>     model n   r sd_ratio mean_error        nME          sde   signed_sde
#> 1 perfect 5 1.0        1          0  0.0000000 0.000000e+00 0.000000e+00
#> 2  biased 5 1.0        1         -1 -0.7071068 1.824283e-16 1.824283e-16
#> 3   noisy 5 0.8        1          0  0.0000000 6.324555e-01 6.324555e-01