Open Risk Data
Open Risk Data is any dataset that has direct or indirect applications in Risk Management and is released under an Open Data license. Broadly speaking, Open Data are freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control.
Risk Data Classification
By Risk Type Category
Open Data relevant for risk management can be of various types, broadly aligned with the legal entities and risk types involved:
- Government Data and Statistics collected and published by National Statistical Agencies or Supranational Organizations. Such data can be directly used in connection with Sovereign Risk but typically includes also aggregate economic data that are relevant also for corporate / consumer analyses
- Market Data from organized markets / exchanges can be used directly for Market Risk analysis but also for more general asset market information
- Consumer Data can be used directly for assessing individual Credit Risk. Consumer data might be made available by either
- Governement agencies granting credit
- Peer-to-peer platforms that offer visibility into their credit portfolios
- Corporate Data can be used in the context of assessing business and credit risk
- Regulatory Disclosures provide information on Bank balance sheets, thereby indirect measures of credit risk
- There are currently no public datasets for Operational Risk
Technically open data might be available as
- feeds (regularly updated) that are provided online via an API
- snapshots, one-of instances of datasets that are not repeated
By Open Data License
Formally a dataset is open if it is complies with a recognized open data definition. The two main categories of open data are either:
- Data in the Public Domain (possibly without explicit license)
- Data provided explicitly under an open data license
In addition, there may be data sets relevant for risk management that are free to use for some purposes, but are not actually open as per the above definition.
By Data Formats
There are currently no public risk data formats or standards for the transmission of risk related data. Generally, risk data are transmitted using ad-hoc formats, or formats developed in domains other than risk management. A discussion of the current status is provided in Risk Data Standards. It is worth mentioning that statistical data exchange among statistics institutions is to some extend standardized around SDMX.
Open Data Provider List
The table below organizes available Open Risk Data providers.
Criteria for inclusion
The objective of the list is to identify as far as possible direct sources of open data sets. Yet there might be significant overlap among the various data repositories as various institutions share and republish open data.
At present the list does not attempt to evaluate the degree of "openness" of the various data sources.
The first entry per record is a general identifier of the data provider. The second entry is the general type of data provided. Large data repositories may include multiple types of data. The URL is a web link where actual data can be accessed, potentially requiring registration. Format/API is an indication how the data can be retrieved, keeping in mind that large repositories may be providing data in multiple formats / channels.
|Supra-national statistics and macro data|
|United Nations Data||General||http://data.un.org/||SDMX|
|European Union Open Data Portal||General||https://open-data.europa.eu/en/data/||Various|
|European Data Portal||General||http://www.europeandataportal.eu/||Various|
|African Development Bank Group||General||http://dataportal.afdb.org/default.aspx||Various|
|ECB Statistical Data Warehouse||Macroeconomic||http://sdw.ecb.europa.eu/||SDMX/REST|
|Select National Statistics / Macro Data Warehouses|
|FRED Economic Data||Macroeconomic||http://research.stlouisfed.org/fred2/||FRED® API|
|Euro Area Business Cycle Network||Macroeconomic||http://www.eabcn.org/page/european-data-sources|
|Bank of England Statistics||Macroeconomic||https://www.bankofengland.co.uk/boeapps/database/||Online Browser / CSV Download|
|Bank of Canada Sovereign Default Data||Credit Events||https://www.bankofcanada.ca/wp-content/uploads/2018/07/crag-database-update-13-07-18.xlsx||XLSX Download|
|BIS Statistics||Banking System Statistics||http://stats.bis.org/bis-stats-tool/org.bis.stats.ui.StatsApplication/StatsApplication.html||Various|
|European Banking Authority Risk Dashboard||European Banking System Statistics||http://www.c-ebs.org/web/guest/risk-analysis-and-data/risk-dashboard||XLS|
|SEC Market Data||Market Structure Data||https://www.sec.gov/data||CSV|
|Bloomberg SDR||Swap Data||http://www.bloombergsdr.com/slicefiles||CSV|
|Depository Trust & Clearing Corp. (DTCC) SDR||Swap Data||https://rtdata.dtcc.com/gtr/dashboard.do||ZIP/CSV|
|Open Corporates||Corporate Web Addresses||https://opencorporates.com/||JSON/XML/REST|
|Fannie Mae||Single Family Loan Performance Data||https://loanperformancedata.fanniemae.com/lppub/index.html#Portfolio||ZIP/CSV||Account Required|
|Freddie Mac||Single Family Loan-Level Dataset||http://www.freddiemac.com/news/finance/sf_loanlevel_dataset.html||ZIP/CSV||Account Required|
|Lending Club||Portfolio Performance Data||https://www.lendingclub.com/info/download-data.action||ZIP/CSV|
|UCI Machine Learning Repository||German Credit Data||https://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29||CSV||1000 records (Donated in 1994)|
|UCI Machine Learning Repository||Taiwan Credit Card Data||https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients||CSV||30000 records|
Issues and Challenges
- Sometimes data providers do not explicitly identify the applicable type of license
- Accessibility of data (formats, API's) and overall Data Quality may vary greatly