Model Risk

From Open Risk Manual

Definition

Model Risk refers to the potential for error in the development and implementation of a financial model and/or the application or interpretation of model results. In turn model errors can lead to a variety of financial and / or reputational loss events.

Context

A financial model is a mathematical representation of future states the world along with the description of those states in financial / economic terms. All financial models are essentially forecasting tools that project explicit or implicit scenarios about the future evolution of economic or financial variables.

Whereas financial risk is generated by business activity (for example the decision of traders to pursue financial transactions with uncertain outcome, of loan officers to extend credit or managers to pursue investment projects), model risk is explicitly and solely due to modelers, "quants", statisticians and other technically trained individuals with the mandate to develop and implement models.

Such individuals maybe either in the employment of the firm or external (e.g., consultants, with rating agencies etc.). Modelers are tasked to create a mathematical representation of future states of the world on the basis of past and current information ("the model"). Models are then used to assist with pricing, risk management and a variety of other business activities.

Causes

Model risk enters all operational aspects of the model development process in the form of

  • poor or missing data (which can refer to both historical data and data relevant for capturing the current state)
  • flawed reasoning (in the form of invalid assumptions or incoherent logical structure)
  • erroneous application of mathematics

The most challenging forms of model risk originate from the deepe difficulties of capturing complex financial or economic activity using relatively simple recipes.

Operationally model risk is generated by the limited ability and/or willingness of modelers to create the best possible, error free, model.

Modeling ability may be limited by

  • resources availability to modellers (data, knowledge, time, computational facilities etc.) and
  • the "modelability" of future financial states

The willingness of the modeler may also be affected by the specific financial, reputational and other elements of his/her role, which may provide adverse incentives and generate biases.

Comparison with other risk types

Model risk is analogous to other risk types in the sense that it can lead to real and unanticipated financial and reputational losses. For example:

  • a faulty credit scorecard can lead to business being accepted that will lead to unexpectedly large credit losses (which may have not been priced-in)
  • a faulty derivatives pricing model can lead to pricing and risk management decisions that lead to significant losses over the life of the product
  • a faulty economic capital model may lead to the firm being unable to meet unexpected losses with risk capital and lead to bankruptcy

How does model risk manifest itself?

Model risk is a “second order” risk (it is always linked to other financial and operational risks) so it can be difficult to identify as a separate risk type. For example, an excess of credit losses by a lending institution can be meaningfully linked to a realization of model risk only if all the following apply:

  • there were models operating and used for as a key input in decision making
  • these models attributed systematically lower probabilities to large loss events
  • the mis-attribution of the models was not known to model users / decision makers

Mitigation

As a risk type, model risk can be mitigated in part with increased scrutiny and attention, typically offered by a competent and independent Model Validation function.

Model risk can potentially be managed with

  • in-depth model validation
    • self assessment and documentation: The developer is directly asked to evaluate his/her model and document the analysis
    • peer review: Other developers/experts are asked to opine on a model
    • independent validation: Specialized and independent internal or external validation experts are asked to opine
  • use of model based limits
    • various conditions for accepting a model that are set in policy
    • amount of business that is relying on a new model
    • amount of monitoring required for a new model
    • setting limits on how long a model can be used before being re-examined
  • use of model risk reserves
    • linked to the sensitivity of model outputs (e.g. valuation) to uncertain / unobservable parameters
  • use economic capital
    • linked to worst case loss analysis

Incentives of the organization to pro-actively manage model risk

Model risk is subject to risk-reward calculations where the cost of developing and validating improved models must be set against the risk posed by poorer models. Factors that may enter such an analysis:

  • Cost of funding and the degree to which model risk may be a factor in investor / market participant views of the firm
  • Regulatory view of model risk and minimum capital requirements.
  • Capitalization of model risk in internal capital assessments

Issues and Challenges

  • Modelers (quants) are paid to produce a model, hence have a basic incentive to produce a standard accepted model and downplay its limitations
  • Model users may have limited technical skills to evaluate model credibility and may rely on third party assurances
  • Implemented models are typically proprietary

Contributors to this article

» Wiki admin