IFRS 9 Model Validation
As part of the specification of an internal control framework around IFRS 9 these papers establish the necessary steps to ensure that the credit risk assessment and measurement models are able to generate accurate, consistent and unbiased predictive estimates, on an ongoing basis. This includes establishing policies and procedures which set out the accountability and reporting structure of the model validation process, internal rules for assessing and approving changes to the models, and reporting of the outcome of the model validation
Credit institutions may use in the ECL assessment and measurement process models and assumption-based estimates for risk identification and measurement, at both the individual lending exposure and overall portfolio levels, including credit grading, credit risk identification, measurement of ECL allowances for accounting purposes, stress testing and capital allocation.
Models used in the ECL assessment and measurement process should consider the impact of changes to borrower and credit risk-related variables such as changes in PDs, LGDs, exposure amounts, collateral values, migration of default probabilities and internal borrower credit risk grades based on historical, current, and reasonable and supportable forward-looking information, including macroeconomic factors.
The development and use of ECL assessment and measurement models involves extensive judgment hence effective model validation policies and procedures are crucial. Credit institutions should have robust policies and procedures in place to appropriately validate the accuracy and consistency of the models used to assess the credit risk and measure ECL, including their model-based credit risk rating systems and processes and the estimation of all relevant risk components, at the outset of model usage and on an ongoing basis. Such policies and procedures should appropriately include the role of professional judgement.
Model validation should be conducted when
- the ECL models are initially developed (Initial Validation) and
- when significant changes are made to the models and
- should ensure that the models are suitable for their proposed usage on an ongoing basis (Periodic Validation)
Model Validation Framework Elements
- Clear roles and responsibilities for model validation with adequate independence and competence.
- Model validation should be performed independently of the model development process and by staff with the necessary experience and expertise.
- The findings and outcomes of model validation should be reported in a prompt and timely manner to the appropriate level of authority.
- Where a credit institution has outsourced its validation function to an external party, the credit institution remains responsible for the effectiveness of all model validation work and should ensure that the work done by the external party meets the elements of a sound model validation framework on an ongoing basis
- An appropriate model validation scope and methodology should include a systematic process of evaluating the model’s robustness, consistency and accuracy as well as its continued relevance to the underlying individual lending exposure or portfolio. An effective model validation process should also enable potential limitations of a model to be identified and addressed on a timely basis. The scope for validation should include a review of model inputs, model design and model outputs/performance.
- Model inputs: Credit institutions should have internally established quality and reliability standards on data (historical, current and forward-looking information) used as model inputs. Data used to estimate ECL allowances should be relevant to the credit institutions’ portfolios and, as far as possible, accurate, reliable and complete (i.e. without exclusions that could bias ECL estimates). Validation should ensure that the data used meet these standards.
- Model design: For model design, validation should assess that the underlying theory of the model is conceptually sound, recognised and generally accepted for its intended purpose. From a forward-looking perspective, validation should also assess the extent to which the model, at the overall model and individual risk factor level, can take into consideration changes in the economic or credit environment, as well as changes to portfolio business profile or strategy, without significantly reducing model robustness
- Model output/performance: Credit institutions should have internally established standards for acceptable model performance. Where performance thresholds are significantly breached, remedial actions up to the extent of model re-calibration or re-development should be taken.
- Comprehensive documentation of the model validation framework and process. This should include documenting the validation procedures performed, any changes in validation methodology and tools, the range of data used, validation results and any remedial actions taken where necessary. Credit institutions should ensure that the documentation is regularly reviewed and updated.
- A review of the model validation process by independent parties (e.g. internal or external parties) to evaluate the overall effectiveness of the model validation process and the independence of the model validation process from the development process. The findings of the review should be reported in a prompt and timely manner to the appropriate level of authority (e.g. senior management, audit committee).
Issues and Challenges
- Given the newly introduced standard and the non-prescriptive nature of the ECL specification, there is a wide range of possible approaches and no widespread best practice in model development and validation
- BIS-D350, Dec 2015