The interval score is upper - lower + 2 / alpha * (lower - obs) below the
interval and upper - lower + 2 / alpha * (obs - upper) above it, where
alpha = 1 - level; there is no penalty inside the interval. Lower scores
indicate sharper, well-calibrated intervals.
Usage
interval_score(
obs,
lower = NULL,
upper = NULL,
level = 0.95,
na.rm = TRUE,
pred = NULL,
predictive_sd = NULL,
distribution = NULL
)Arguments
- obs
Numeric observation vector.
- lower, upper
Numeric bounds of the central prediction interval. For the usual proper-score interpretation, these must be the equal-tailed predictive quantiles corresponding to
level.- level
Nominal central interval coverage, strictly between zero and one; determines the required predictive quantiles for
lowerandupper.- na.rm
Logical; remove incomplete triplets?
- pred, predictive_sd
Optional numeric vectors of predictive means and predictive standard deviations, supplied together and of the same length as
obs. Assumes normal predictive distributions. Non-missing predictive standard deviations must be finite and strictly positive.- distribution
Optional numeric matrix or data frame of equally weighted predictive samples: one row per observation and one column per predictive draw. Supply this instead of bounds or predictive means and standard deviations. At least one draw is required; infinite values are not allowed.
Details
For the usual proper-scoring interpretation, lower and upper must be the
central (equal-tailed) predictive quantiles corresponding to level: the
predictive quantiles at probabilities (1 - level) / 2 and
(1 + level) / 2, respectively. Supplying arbitrary interval bounds together
with a nominal level still evaluates the formula below, but does not in
general give the same proper interval-score interpretation.
$$\mathrm{IS}_\tau = \frac{1}{n}\sum_{i=1}^{n}[upper_i-lower_i+ \frac{2}{1-\tau}\max(lower_i-obs_i,0)+ \frac{2}{1-\tau}\max(obs_i-upper_i,0)]$$
The score has response units and lower values are better. It rewards narrow intervals but penalizes observations outside them by their distance from the nearest bound.
Input representations
Supply exactly one of explicit lower and upper bounds, predictive mean
and standard deviation (pred and predictive_sd), or predictive samples
(distribution). Normal inputs generate central intervals using normal
quantiles. Samples generate equal-tailed intervals using stats::quantile()
with type = 7, at probabilities (1 - level) / 2 and (1 + level) / 2.
With na.rm = TRUE, a case is removed if its observation or any supplied
predictive value is missing; individual missing draws are not discarded
within a case. With na.rm = FALSE, incomplete inputs give NA. No complete
cases also gives NA. Standalone interval_width() ignores missing obs.