Model Risk Management for Financial Institutions

Model Risk Management For Financial Institutions

Program – May 26

9:00 a.m.  –  9:05 a.m.
Opening Remarks from the Chair

David Harris, Partner, PricewaterhouseCoopers LLP

9:05 a.m. – 10:30 a.m.
Understanding OSFI Model Risk Management Guideline (E-23) and Pending Changes in Draft Guideline

PricewaterhouseCoopers LLP
OSFI (Invited)

• Overview of Guideline E-23 and Draft Guideline E-23 
• Explanation of the regulatory definition of models
• Identifying models used in financial enterprises 
  • where are they used
  • how are they used and understanding the functions they perform

• Scope application to models – E-23 and Draft Guideline
• Explanation of the model lifecycle – key stages and processes
• Model Risks OSFI wants managed – financial and non-financial 
  • issues raised by AI and Machine Learning

• Required steps for model risk management
• Disclosure and reporting – what you need to know

10:30 a.m. – 10:45 a.m. – Morning Networking Break

10:45 a.m. – 12:00 p.m.
Examining Regulatory Regimes Affecting Models, Model Risk, AI/ML and Data – Canada, US, International

Claire Feltrin, Counsel, Borden, Ladner, Gervais LLP 

• Highlighting regulations and statutes affecting models and model risk (including related issues)
• Examining overlap of parallel requirements both Canadian and foreign 
• How to develop a comprehensive compliance and reporting process 
• Understanding differences in terms and objectives of differing requirements
• Establishing primary or coordinated governance responsibilities
• Ensuring your oversight framework is complete, comprehensive and remains up to date
• Implications of inadequate oversight/governance

Restrictions on Data Usage – OSFI, Canadian and International Regulations/Laws

• Legislative restrictions on data usage 
  • PIPEDA
  • Federal and provincial privacy laws
  • GDPR
  • Anti-spam regulations

• Specific legal and regulatory requirements for financial institutions
  • Bank Act
  • OSFI requirements for banks, insurers and pension funds including data governance
  • FSRA guidance for credit unions
  • provincial laws and regulations for financial services

• Emerging trends and challenges 
  • data used for artificial intelligence and machine learning
  • cross-border data transfers/usage
  • OSFI operational resilience compliance
  • increasing penalties for non-compliance

12:00 p.m. – 1:00 p.m. – Luncheon

1:00 p.m. – 2:00 p.m.
Data and Digital Risks for Effective Management

Nicolas Gemin, Risk Leader for Insurance, Deloitte LLP
Andreas Witz, Data Risk and Compliance Leader, Deloitte LLP

• Importance of Data Risk for Banks and Insurers
• OSFI E-21 Data Risk and Impact on Bank and Insurers
• Addressing Common Data Risk Challenges
• Evolving Data Governance for Modern Data Risk Management 
• Building a Mature AI Risk Management Framework and AI Solutions
• Leveraging Data Risk as Business Advantage

2:00 p.m. – 2:45 p.m.
Impact of E-23 on Organizational Structure/Processes and Governance

Carla Crichlow, Senior Manager, Actuarial Consulting, Oliver Wyman
Dean Rootenberg, Senior Principal, Actuarial Life Consulting, Oliver Wyman

• Is model management and model risk management integrated into strategy formulation?
• Implementing tools to measure/monitor model risk including data governance
• Inclusion of model related risks into the risk appetite framework
• Risk management oversight – what needs to be done?
• Defining roles and responsibilities for first line of defense
• Understanding organizational model/data exposures including reputational risk
• Delineating, and implementing a model risk management framework
  • identifying the essential elements of the framework

• Development of KPI’s 
• Identifying model related risks in all risk management frameworks
• New challenges for model risk management and oversight

2:45 p.m. – 3:00 p.m. – Afternoon Networking Break

3:00 p.m. – 4:00 p.m.
Governance and Internal Controls Related to Model Risk Reporting

Ernst & Young LLP

• Role of internal controls for model risk management and reporting
• Developing a reporting control framework methodology including non-financial considerations 
• Entity-level internal controls related to models and model risk including
  • managing changes to models
  • decommissioning of models
  • ongoing matters affecting model data – new data, obsolete data
  • completeness of model inventory
  • Defining the three lines of defense in model risk management
  • Reliance on 3rd party service providers for model and data services
  • evaluation of controls in place for service providers

• Applying the COSO framework for model risk reporting controls
• Data integrity and accountability 
• Continuous monitoring of processes and controls

4:00 p.m. – 5:00 p.m.
MRM Documentation Summary and Critical Highlight

Alison Rose, Partner – Life and Pensions Actuarial, KPMG LLP
Simmy Leung, Senior Manager, Actuarial, Risk and Insurance Services, KPMG LLP

• Summary of regulatory requirements for model risk management documentation
• Main topics of documentation and key elements
  • model development
  • model validation
  • model governance and oversight 
  • model performance and monitoring
  • model risk reporting and communication
  • regulatory compliance
  • stress testing and scenario analysis
  • model retirement and decommissioning

• Items to consider not addressed by specific headings
• Relationship to data management and governance documentation expectations
• Who owns the documentation process?
• Linking documentation to financial/regulatory reporting

5:00 p.m. – End of Day 1

Program – May 27

9:00 a.m.  –  9:05 a.m.
Opening Remarks from the Chair

David Harris, Partner, PricewaterhouseCoopers LLP

9:05 a.m. – 10:10 a.m.
“Fireside Chat” – Practical Perspectives on MRM and E-23 – Lessons Learned in Banking and Insurance

PricewaterhouseCoopers LLP

10:10 a.m. – 11:00 a.m.
Examination of Tools for Model Risk and Data Risk Management

Ernst & Young LLP

• Model risk governance platforms 
• Data management and quality tools 
• Model development and deployment tools 
• Model validation tools 
• Model monitoring and performance management tools
• Explainability and bias detection tools
• Stress testing and scenario analysis tools 
• Workflow automation and collaboration tools
• Regulatory compliance tools
• AI/ML-specific MRM tools

11:00 a.m. – 11:15 a.m. – Morning Networking Break

11:15 a.m. – 12:15 p.m.
Model Validation – Traditional Models, Gen AI and Others

Maximillian Boermann, Senior Manager, Risk Consulting Services, KPMG LLP
David Tang, Senior Manager, Data Analytics and Data Science, KPMG LLP

Model Validation

• Regulatory background for model performance validation
• Focus topics in model validations
  • Documentation and transparency for internal and external communication
  • Theoretical review
  • Performance testing metrics including back-testing

  Unique Challenges Gen AI Validation

  • Documentation and transparency for internal and external communication
  • Fairness audits for ethical and bias risks
  • Regulatory uncertainty
  • Differences in theoretical review between traditional and GenAI model validation
  • Performance review
  • Model comparison
  • Validation of non-deterministic outputs

  12:15 p.m. – 1:15 p.m.  –  Luncheon

  1:15 p.m. – 2:15 p.m.

  Applying E-23 to Non-Traditional Models Including Climate Risk Models 

  TBA

  • • • • • Examination of unique risks presented by climate risk models

  -scenario analysis

  -geospatial analysis

  -financial impact quantification

  -dynamic modelling

  • • • • • Highlighting the data requirements for climate risk management

  -data risks arising from climate risk management that must be managed

  -data quality

  -data relevance

  -emerging metrics

  -projections and uncertainty

  -third part dependencies

  • • • • • Integration into existing risk management processes
          • Integration with risk management frameworks 

  2:30 p.m. – 4:00/4:30 p.m.  (There will be a 15 minute break during this session)

  Industry-Specific Case Studies in Model Risk Management  –  Banking and Insurance

  Insurance Industry  – PwC, EY

  Liability Models 

  Actuarial Models 

  Financial Condition Testing

  Banking 

  Credit Risk

  Market Risk Models

  Stress Testing and Scenario Analysis

  AI/ML and Decision Making Automation

  NOTE – Each industry sector can be comprised of a few individual discussions allocated across firms or a single firm could take up the entire session to present.