Open Risk Data

From Open Risk Manual

Definition

Open Risk Data is any dataset that has direct or indirect applications in Risk Management and is released under an Open Data license. Broadly speaking, Open Data are freely available to everyone to use and republish as they wish, without restrictions from copyright, patents or other mechanisms of control.

Risk Data Classification

By Risk Type Category

Open Data relevant for risk management can be of various types, broadly aligned with the legal entities and risk types involved:

  • Government Data and Statistics collected and published by National Statistical Agencies or Supranational Organizations. Such data can be directly used in connection with Sovereign Risk but typically includes also aggregate economic data that are relevant also for corporate / consumer analyses
  • Market Data from organized markets / exchanges can be used directly for Market Risk analysis but also for more general asset market information
  • Consumer Data can be used directly for assessing individual Credit Risk. Consumer data might be made available by either
    • Governement agencies granting credit
    • Peer-to-peer platforms that offer visibility into their credit portfolios
  • Corporate Data can be used in the context of assessing business and credit risk
  • Regulatory Disclosures provide information on Bank balance sheets, thereby indirect measures of credit risk
  • There are currently no public datasets for Operational Risk

By Availability

Technically open data might be available as

  • feeds (regularly updated) that are provided online via an API
  • snapshots, one-of instances of datasets that are not repeated


By Open Data License

Formally a dataset is open if it is complies with a recognized open data definition. The two main categories of open data are either:

  • Data in the Public Domain (possibly without explicit license)
  • Data provided explicitly under an open data license


In addition, there may be data sets relevant for risk management that are free to use for some purposes, but are not actually open as per the above definition.

By Data Formats

There are currently no public risk data formats or standards for the transmission of risk related data. Generally, risk data are transmitted using ad-hoc formats, or formats developed in domains other than risk management. A discussion of the current status is provided in Risk Data Standards. It is worth mentioning that statistical data exchange among statistics institutions is to some extend standardized around SDMX.

Open Data Provider List

The table below organizes available Open Risk Data providers.

Criteria for inclusion

The objective of the list is to identify as far as possible direct sources of open data sets. Yet there might be significant overlap among the various data repositories as various institutions share and republish open data.

At present the list does not attempt to evaluate the degree of "openness" of the various data sources.

Legend

The first entry per record is a general identifier of the data provider. The second entry is the general type of data provided. Large data repositories may include multiple types of data. The URL is a web link where actual data can be accessed, potentially requiring registration. Format/API is an indication how the data can be retrieved, keeping in mind that large repositories may be providing data in multiple formats / channels.

Table

Data Provider Type URL Format/API Comments
Supra-national statistics and macro data
United Nations Data General http://data.un.org/ SDMX
European Union Open Data Portal General https://open-data.europa.eu/en/data/ Various
European Data Portal General http://www.europeandataportal.eu/ Various
openAfrica General https://africaopendata.org/ Various
African Development Bank Group General http://dataportal.afdb.org/default.aspx Various
World Bank General http://data.worldbank.org/ Various
IMF Data General http://data.imf.org Various
Eurostat General http://ec.europa.eu/eurostat SDMX
OECD Data General https://data.oecd.org/ Various
ECB Statistical Data Warehouse Macroeconomic http://sdw.ecb.europa.eu/ SDMX/REST
Select National Statistics / Macro Data Warehouses
FRED Economic Data Macroeconomic http://research.stlouisfed.org/fred2/ FRED® API
Euro Area Business Cycle Network Macroeconomic http://www.eabcn.org/page/european-data-sources
Bank of England Statistics Macroeconomic https://www.bankofengland.co.uk/boeapps/database/ Online Browser / CSV Download
Bank of Canada Sovereign Default Data Credit Events https://www.bankofcanada.ca/wp-content/uploads/2018/07/crag-database-update-13-07-18.xlsx XLSX Download
Regulatory Disclosures
BIS Statistics Banking System Statistics http://stats.bis.org/bis-stats-tool/org.bis.stats.ui.StatsApplication/StatsApplication.html Various
European Banking Authority Risk Dashboard European Banking System Statistics http://www.c-ebs.org/web/guest/risk-analysis-and-data/risk-dashboard XLS
Market Data
SEC Market Data Market Structure Data https://www.sec.gov/data CSV
Bloomberg SDR Swap Data http://www.bloombergsdr.com/slicefiles CSV
Depository Trust & Clearing Corp. (DTCC) SDR Swap Data https://rtdata.dtcc.com/gtr/dashboard.do ZIP/CSV
Zillow Real Estate https://www.zillow.com/research/data/ ZIP/CSV
Corporate Data
Open Corporates Corporate Web Addresses https://opencorporates.com/ JSON/XML/REST
Consumer Data
Fannie Mae Single Family Loan Performance Data https://loanperformancedata.fanniemae.com/lppub/index.html#Portfolio ZIP/CSV Account Required
Freddie Mac Single Family Loan-Level Dataset http://www.freddiemac.com/news/finance/sf_loanlevel_dataset.html ZIP/CSV Account Required
Lending Club Portfolio Performance Data https://www.lendingclub.com/info/download-data.action ZIP/CSV
UCI Machine Learning Repository German Credit Data https://archive.ics.uci.edu/ml/datasets/Statlog+%28German+Credit+Data%29 CSV 1000 records (Donated in 1994)
UCI Machine Learning Repository Taiwan Credit Card Data https://archive.ics.uci.edu/ml/datasets/default+of+credit+card+clients CSV 30000 records

Issues and Challenges

  • Sometimes data providers do not explicitly identify the applicable type of license
  • Accessibility of data (formats, API's) and overall Data Quality may vary greatly

Contributors to this article

» Wiki admin