Model Decay

From Open Risk Manual

Definition

Model Decay (also Model Failure) is an informal characterization of pathologies of models already deployed (in operation), whereby the model performance may deteriorate to the point of the model not being any longer fit for purpose.

Context

Model decay and failure applies to both risk models and valuation/pricing models. Both classes of models fail when they provide poor information about future financial world states. (The assumption is that at the time the model was developed / approved, it did meet fitness for purpose criteria).

The first class (risk models) fails when calculating a risk metric in a manner that can be statistically demonstrated to be unreliable when applied to new empirical dataset.

The second class (valuation models) fails by producing a valuation at a future date (once markets have changed) that is at demonstrable variance with observed future market valuations. This is considered a less severe (or even normal) effect when it can be remedied by a model recalibration.

Ultimately the cause of model failure is inadequate capture of the underlying economic reality, which is most easily observed when new data demonstrably fall outside the "reasonable" range of the model

Examples

  • drifting populations for credit risk models that require re-fitting the model or even develop completely new model
  • discontinuous changes to market volatility and/or correlations that require model recalibration
  • impact of counterparty credit risk that invalides risk-free model assumptions
  • liquidity, regulatory and other structural changes

Issues and Challenges

  • Fundamental difficulty to predict useful lifetime of a model