大浩浩的笔记课堂——FRM考试学习笔记116

Topic 45 Validating Rating Models

1.Model validation:

⑴definition:

To validate a rating model,a financial institution must confirm the reliability of the results produced by the model and that the model still meets the financial institution´s operating needs and any regulatory requirements.The tools and approaches to validation are regularly reassessed and revised to stay current with the changing market and operating environment.

⑵methods:

①qualitative validation(more important):

A.factors:

a/ structure and model methodology

b/ data quality and treatment

c/ process and procedures

B.focusing:

It focus on non-numerical issues pertaining to model development.

②quantitative validation:

A.stages:

a/ building stage:

performance in and out of sample

b/ monitoring stage:

backtesting,internal benchmarking

B.including:

Comparing ex post results of risk measures to ex ante estimates,parameter calibrations,benchmarking and stress tests.

⑶practice:

Best practices for the roles of internal organizational units in the validation process are including active involvement of senior management and internal audit group.In general,all staff involved in the validation process must have sufficient training to perform their duties properly.

⑷group:

With regard to independence,the validation group must be:

①independent from the groups that are developing and maintaining 

validation models and the groups dealing with credit risk

②independent of the lending group and the rating assignment group

③should not report to any of those groups

④could be involved with the rating system design and development process

⑸documentation:

Given that validation is mainly done using documentation received by groups dealing with model development and implementation,the quality of the documentation is important.

⑹control:

Controls must be in place to ensure that there is sufficient breadth,transparency,and depth in the documentation provided.

2.Comparison of qualitative and quantitative validation processes:

⑴elements of qualitative validation:

①obtaining probabilities of default

②completeness of rating system

③objectivity of rating system

④acceptance of rating system(heuristic vs. Statistical rating models)

⑤consistency of rating system 

⑵elements of quantitative validation:

①sample representativeness

②discriminatory power

③dynamic properties:

It focuses on rating systems stability and attributes of migration matrices.

④calibration:

It focuses on examining the variances between estimates PD and actual rates of default.

3.Data Quality:

⑴challenges to data quality:

①general challenges:

completeness,availability,sample representativeness consistency and integrity,data cleaning procedures

②defaults:

Defaults are the key constraint in terms of creating sufficiently large data sets for model development,rating quantification,and validation purposes.

③sample size(the most controllable) and sample homogeneity:

With regard to sample size and sample homogeneity,it is difficult to create samples from a population over a long period using the same lending technology:

Lending technology is most likely to change.Unfortunately,the changes result in less consistency between the data used to create the rating model and the population to which the model is applied.

④time horizon:

The time horizon of the data may be problematic because the data should take into account of a full credit cycle.If it is less than a full cycle,the estimates will be biased by the favorable or unfavorable stages during the selected period within the cycle.

⑵validating a model´s data quality:

Validating data quality focuses on the stability of the lending technology and the degree of calibration required to infer sample results to the population.

⑶validating discriminatory power:

①including:

It involves backtesting of defaulting and non-defaulting items.

·特别注意！

·Accurately differentiate between defaulting from non-defaulting entities for a given forecasting period.

②tests:

Classification tests of discriminatory power are including:

A.statistical tests

B.migration matrices

C.accuracy indices

⑷validating calibration:

①definition:

It looks at the variances from the expected probabilities of default and the actual default rates,it looks at the relative ability to estimate PD.

②including:

Tests of calibration are including:

A.binomial test→single rating category at a time

B.chi-square test(or Hosmer-Lerreshow)→multiple rating category at a time

C.normal test→single rating category more than one period

大浩浩的笔记课堂之FRM考试学习笔记合集