Bank of Montreal 2015 Annual Report Download - page 102

MD&A
Model Development and Validation
Models are developed, implemented and used to meet specific business objectives, including applicable regulatory requirements. Model owners, in
consultation with model developers and other stakeholders, determine the design, objectives, intended use and desired functionality of models, and
have overall responsibility for ensuring models comply with bank policies and approved terms of use. Model developers assist the model owners by
proposing model solutions, identifying data availability and limitations and developing and implementing models for their intended purposes.
They do so by engaging model owners and other key stakeholders in the development and implementation processes, and by evaluating and
documenting alternatives and model characteristics, outputs, strengths and weaknesses. Our independent Model Validation group reviews model
development outputs to evaluate whether a proposed model is conceptually and statistically sound, achieves its objectives and is fit for its intended
use without creating material model risk. Observations are made to guide model owners, users and developers, remediation of material deficiencies
may be required and unless an exception is obtained in accordance with bank policy, approval from our Model Validation group is required before a
model is used.
Model Use and Monitoring
Model owners and other model users are accountable for using the models appropriately for business decision-making and for the proper care and
maintenance of models throughout the model life cycle. The development and validation processes provide guidance to ensure that models can be
used effectively within an appropriate range of use, that model limitations are known and that model risk mitigants are implemented. When in use,
models are subject to ongoing monitoring, including outcomes analysis and periodic reviews. Ongoing monitoring and outcomes analysis are part of
evaluation processes to confirm the continuing validity and adequate performance of each model over time. These techniques and other controls are
used to mitigate potential issues and to help ensure continuing acceptable model performance. All models in use are subject to periodic scheduled
reviews, with the frequency based on a model’s risk rating, and to earlier reviews if business judgment, triggers or other ongoing monitoring tools
indicate that model performance may be inadequate. Scheduled reviews require the model owner and developers to assess a model’s continuing
suitability for use and such assessment is subject to independent review by our Model Validation group.
Model Validation, Outcome Analysis and Back-Testing
Once the models are validated, approved and in use, they are subject to ongoing validation, which includes ongoing monitoring and outcomes
analysis. As a key component of the outcomes analysis, back-testing measures model outputs against actual observed outcomes. This analysis is used
to confirm the validity and performance of each model over time, and helps to ensure that appropriate controls are in place to address identified
issues and enhance a model’s overall performance.
Credit Risk – The Credit Risk Model Validation Guidelines are an important subset of BMO’s Model Risk Corporate Policy. These guidelines include clear
and detailed requirements for the back-testing of all credit risk rating models.
The process for back-testing the Probability of Default (PD) model computation includes comparing PD estimates generated by credit risk models
against the actual or realized default rates across all obligor ratings. This process also includes testing for statistical evidence that default rates
accurately capture sampling variability over time.
The comprehensive validation of a risk rating system involves various prescribed tests and analyses that measure discriminatory power,
calibration and dynamic properties, with support from migration analysis. Additional tests or analyses are used to validate borrower risk rating grades
and probability of default.
As with any analysis, judgment is applied in determining various factors, such as data limitations, which may affect the overall relevance of a
given validation approach or interpretation of statistical analysis. Similar back-testing is applied to the Loss Given Default (LGD) and Exposure at
Default (EAD) model computations.
Annual validations of all material models in use are conducted to ensure they perform as intended and to confirm they continue to be fit for use.
An annual review includes a qualitative assessment conducted by model developers and a quantitative validation conducted by the Model Validation
group, with all conclusions reported to senior management.
Trading and Underwriting Market Risk – All internal models used to calculate regulatory capital and Economic Capital for trading and underwriting
market risk have their Value at Risk (VaR) results back-tested regularly. The bank’s internal VaR model is back-tested daily, and the one-day 99%
confidence level VaR at the local and consolidated BMO levels is compared against the realized theoretical Profit & Loss (P&L) calculation, which is the
daily change in portfolio value that would occur if the portfolio composition remained unchanged. If the theoretical P&L is negative and its absolute
value is greater than the previous day’s VaR, a back-testing exception occurs. Each exception is investigated, explained and documented, and the
back-testing results are reviewed by the Board and our regulators. This process monitors the quality and accuracy of the internal VaR model results
and assists in refining overall risk measurement procedures.
Structural Market Risk – Back-testing of our structural market risk models is performed monthly and reported quarterly. For products with a
scheduled term, such as mortgages and term deposits, the model-predicted prepayments or redemptions are compared against the actual outcomes
observed. For products without a scheduled term, such as credit card loans and chequing accounts, the modelled balance run-off profiles are
compared against actual balance trends.
The variances between model predictions and the actual outcomes experienced are measured against pre-defined risk materiality thresholds.
To ensure variances are within the tolerance range, actions such as model review and parameter recalibration are taken. Performance is assessed
by analyzing model overrides and tests conducted during model development, such as back-testing and sensitivity testing.
BMO Financial Group 198th Annual Report 2015 113