How to Build a Credit Scorecard
- 1 How to Build a Credit Scorecard
- 2 The Six Stages of the Credit Scorecard Lifecycle
- 3 Stage 1: Preliminary considerations
- 4 Stage 2: Scorecard Development
- 5 Stage 3: Model Validation
- 6 Stage 4: Model Deployment
- 7 Stage 5: Model Monitoring
- 8 Stage 6: Adjustment / Decommissioning
- 9 See Also
- 10 Issues and Challenges
How to Build a Credit Scorecard
To understand the development process for a scorecard we place it in the context of the overall lifecycle of a Risk Model
The Six Stages of the Credit Scorecard Lifecycle
- Stage 1: Preparation or Preliminary Considerations: This stage defines the scope and objectives of the credit scorecard. The outcome of this stage may include a formal Model Origination or Requirements Document, a Project Plan, and/or other detailed documentation depending on the organizational governance of the entity introducing the credit scorecard
- Stage 2: Development: This stage captures the main technical activities (Data Collection, Data Review, Data Cleansing, Model Development, Expert Analysis, Model Documentation etc.) that produce a complete credit scorecard specification
- Stage 3: Model Validation: This stage (sometimes bundled with development) provides a more or less formal review of the Stage 2 developed scorecard. When scorecards are used in regulated / audited context this stage may reject the scorecard specification produced in Stage 2, offering concrete reasons
- Stage 4: Deployment: This stage includes production implementation, acceptance testing, user training etc. The outcome is an operating scorecard that is processing actual client data, is fully embedded in the Credit Rating System and any related risk / management processes
- Stage 5: Monitoring: Scorecard performance is monitored throughout its active life to identify pathologies such as Model Decay. Typically there is a Model Monitoring Report that captures the essential performance indicators (including also historical development)
- Stage 6: Adjustment or Decommissioning. If the monitoring report or other current information suggests so, the scorecard might need to be re-estimated using additional data or decommissioned. In case of re-estimation / redevelopment the six stages are repeated but now on the basis of the pre-existing implementation.
Stage 1: Preliminary considerations
Selecting the type of scorecard
There is a very large variety of possible credit scorecards. Selecting the right type requires identifying the concrete needs of the project in terms of abilities and functionality but also the practicalities of implementation (availability of data, computer systems, human expertise, degree of automation)
Stage 2: Scorecard Development
The specifics of the scorecard development process depend on the type of scorecard. The following is a list of activities that will generally be required for most common types
Practical Development Steps
- Data Collection. Establishing links with existing databases / files. Writing and testing queries and filters. Importing data
- Data Cleaning. Reviewing and establishing Data Quality
- Missing Data. Where appropriate impute missing data
- Creating a master data table. This table of characteristics and outcomes is the basic input to the quantitative estimation
- Setting up a machine learning estimation framework (where applicable). Using a commercial or open source toolkit.
- Performing and reviewing model estimation outcomes ( model accuracy, out-of-sample performance etc.)
Conceptual Development Steps
- Historical Sample Selection.
- Credit Event Definition.
- Identification of Characteristics. There is an enormous variety of possible characteristics depending of the type of credit risk being evaluated.
There may be legal, regulatory or business (cost) limitations
- Population Segmentation.
- Characteristic Selection. Narrowing down the list of characteristics, e.g. using Backward Selection.
- Transformation Methodologies. Investigating the application of non-linear transformations to characteristics
- Selection of model family (e.g logistic regression)
- Selection of operational parameters (like cutoffs) where applicable
Stage 3: Model Validation
The model validation stage will include the following steps, depending on the rigour / independence required
- Review of conceptual methodology
- Review of practical development steps
- Independent replication of the model
Stage 4: Model Deployment
Depending on the systems of the entity using the scorecard, the following will be typical steps
- Implementation of the developed model as a scorecard inference system in production systems. (Production systems typically do not require ability to re-estimate models on the fly)
- Acceptance testing of the implementation by the operating unit
- User training
Stage 5: Model Monitoring
Monitoring is done at the appropriate timescale (e.g., from daily to quarterly). Various levels of monitoring might be used. Monitoring typically produces updated metrics for the set of metrics that was already used in the selection / validation of the model. This includes primarily
- Portfolio evolution statistics
- Model performance statistics
Stage 6: Adjustment / Decommissioning
When a monitoring report or other insight suggests the scorecard in production is no longer fit-for-purpose then depending on the context the scorecard must be adjusted or replaced. Model adjustment can be
- minimal (e.g. re-estimation using an adjusted dataset)
- substantial (e.g. introduction of a new characteristic, changing segmentation)
- significant intervention (e.g. changing the model family)
Depending on the context (e.g. regulation) any significant change may be classified as a new model and triggers a full validation / implementation cycle.
- A collection of distinct types of quantitative models is given in the Credit Scoring Models page.
Issues and Challenges
- Developing and using quantitative risk models such as credit scorecards if full of pitfalls. Check the The Zen of Modeling for a high level list.