Model Risk Management for Financial Institutions

Model Risk Management For Financial Institutions

Program – December 3

9:00 a.m.  –  9:05 a.m.
Opening Remarks from the Chair

Cassandra Tontini, Director- Risk Modelling Services, PricewaterhouseCoopers LLP

9:05 a.m. – 10:30 a.m.
Understanding OSFI Model Risk Management Guideline (E-23) and Pending Changes in Draft Guideline

John Belmonte, Senior Manager, PricewaterhouseCoopers LLP
Shin Ng, Senior Manager, PricewaterhouseCoopers LLP

• Overview of Guideline E-23 and Draft Guideline E-23 
• Explanation of the regulatory definition of models
• Identifying models used in financial enterprises 
  • where are they used
  • how are they used and understanding the functions they perform

• Scope application to models – E-23 and Draft Guideline
• Explanation of the model lifecycle – key stages and processes
• Model Risks OSFI wants managed – financial and non-financial 
  • issues raised by AI and Machine Learning

• Required steps for model risk management
• Disclosure and reporting – what you need to know

10:30 a.m. – 10:45 a.m. – Morning Networking Break

10:45 a.m. – 12:00 p.m.
Examining Regulatory Regimes Affecting Models, Model Risk, AI/ML and Data – Canada, US, International

Shane Morganstein, Associate, Borden, Ladner, Gervais LLP

Regulatory Treatment of Models and Model Risk in Financial Institutions

• Highlighting regulations and statutes affecting models and model risk (including related issues)
• Examining overlap of parallel requirements both Canadian and foreign 
• How to develop a comprehensive compliance and reporting process 
• Understanding differences in terms and objectives of differing requirements
• Establishing primary or coordinated governance responsibilities
• Ensuring your oversight framework is complete, comprehensive and remains up to date
• Implications of inadequate oversight/governance

Restrictions on Data Usage – OSFI, Canadian and International Regulations/Laws

• Legislative restrictions on data usage 
  • PIPEDA
  • Federal and provincial privacy laws
  • GDPR
  • Anti-spam regulations

• Specific legal and regulatory requirements for financial institutions
  • Bank Act
  • OSFI requirements for banks, insurers and pension funds including data governance
  • FSRA guidance for credit unions
  • provincial laws and regulations for financial services

• Emerging trends and challenges 
  • data used for artificial intelligence and machine learning
  • cross-border data transfers/usage
  • OSFI operational resilience compliance
  • increasing penalties for non-compliance

12:00 p.m. – 1:00 p.m. – Luncheon

1:00 p.m. – 2:00 p.m.
Data and Digital Risks for Effective Management

Nicolas Gemin, Risk Leader for Insurance, Deloitte LLP
Andreas Witz, Data Risk and Compliance Leader, Deloitte LLP

• Importance of Data Risk for Banks and Insurers
• OSFI E-21 Data Risk and Impact on Bank and Insurers
• Addressing Common Data Risk Challenges
• Evolving Data Governance for Modern Data Risk Management 
• Building a Mature AI Risk Management Framework and AI Solutions
• Leveraging Data Risk as Business Advantage

2:00 p.m. – 2:45 p.m.
Impact of E-23 on Organizational Structure/Processes and Governance

Carla Crichlow, Senior Manager, Actuarial Consulting, Oliver Wyman
Dean Rootenberg, Senior Principal, Actuarial Life Consulting, Oliver Wyman

• Getting E-23 structure in place – establishing working group and implementation plan
• Delineating, and implementing a model risk management framework
  • identifying the essential elements of the framework
• Understanding organizational model/data exposures including reputational risk
• Implementing tools to measure/monitor model risk including data governance
• Inclusion of model related risks into the risk appetite framework
• Defining roles and responsibilities relative to other parts of the organization, e.g., business, audit, compliance
• Inclusion of model related risks into the risk appetite framework
• Development of KPI’s 
• Risk management oversight – what needs to be done?
• New challenges for model risk management and oversight

2:45 p.m. – 3:00 p.m. – Afternoon Networking Break

3:00 p.m. – 4:00 p.m.
Governance and Internal Controls Related to Model Risk Reporting

Ulana Oswald, Partner, Ernst & Young LLP
Wissem Bouraoui, Senior Manager, Ernst & Young LLP

• Role of internal controls for model risk management and reporting
• Developing a reporting control framework methodology including non-financial considerations 
• Entity-level internal controls related to models and model risk including
  • managing changes to models
  • decommissioning of models
  • ongoing matters affecting model data – new data, obsolete data
  • completeness of model inventory
  • Defining the three lines of defense in model risk management
  • Reliance on 3rd party service providers for model and data services
  • evaluation of controls in place for service providers

• Applying the COSO framework for model risk reporting controls
• Data integrity and accountability 
• Continuous monitoring of processes and controls

4:00 p.m. – 5:00 p.m.
MRM Documentation Summary and Critical Highlight

Alison Rose, Partner – Life and Pensions Actuarial, KPMG LLP
Simmy Leung, Senior Manager, Actuarial, Risk and Insurance Services, KPMG LLP

• Summary of regulatory requirements for model risk management documentation
• Main topics of documentation and key elements
  • model development
  • model validation
  • model governance and oversight 
  • model performance and monitoring
  • model risk reporting and communication
  • regulatory compliance
  • stress testing and scenario analysis
  • model retirement and decommissioning

• Items to consider not addressed by specific headings
• Relationship to data management and governance documentation expectations
• Who owns the documentation process?
• Linking documentation to financial/regulatory reporting

5:00 p.m. – End of Day 1

Program – December 4

9:00 a.m.  –  9:05 a.m.
Opening Remarks from the Chair

Cassandra Tontini, Director- Risk Modelling Services, PricewaterhouseCoopers LLP

9:05 a.m. – 10:10 a.m.
“Fireside Chat” – Practical Perspectives on MRM and E-23 – Lessons Learned in Banking and Insurance

Matt Devine, Partner – Financial Services Risk and Regulatory, PricewaterhouseCoopers LLP
Banking Representative
Insurance Company Representative

10:10 a.m. – 11:00 a.m.
Examination of Tools for Model Risk and Data Risk Management

Ulana Oswald, Partner, Ernst & Young LLP
Wissem Bouraoui, Senior Manager, Ernst & Young LLP

• Model risk governance platforms 
• Data management and quality tools 
• Model development and deployment tools 
• Model validation tools 
• Model monitoring and performance management tools
• Explainability and bias detection tools
• Stress testing and scenario analysis tools 
• Workflow automation and collaboration tools
• Regulatory compliance tools
• AI/ML-specific MRM tools

11:00 a.m. – 11:15 a.m. – Morning Networking Break

11:15 a.m. – 12:05 p.m.
Applying E-23 to Non-Traditional Models Including Gen AI, MLM

Debanjan Bandyopadhyay, Manager,  KPMG LLP
Divya Tulapurkar, Director, KPMG LLP

• • E-23’s expectations for non=traditional models
  • LLM/RAG validation design
  • Mixture of Experts model design
  • Roadmap to compliance

12:05 p.m. – 1:00 p.m.  –  Luncheon

1:00 p.m. – 2:00 p.m.
Model Validation – Traditional Models, GenAI and Others

Maximillian Boermann, Senior Manager, Risk Consuting Services, Ernst & Young LLP
David Tang, Senior Manager – Data Analytics and Data Science,  Ernst & Young LLP

• Regulatory background for model performance validation
• Focus topics for model validation
• Theoretical Review
  • Use of models from traditional to GenAI
  • Quality and appropriateness of model with focus on GenAI
  • Model risk rating
• Performance Testing Metrics
  • Library of performance measurement metrics between traditional and GenAI
  • Fairness audit for ethical and bias risks

  2:00 p.m. – 2:50 p.m.
  Applying E-23 to Climate Risk Models 

  Anna Kryshtopenko, Senior Actuarial Consultant, KPMG LLP 

  • Examination of unique risks presented by climate risk models
    • scenario analysis
    • geospatial analysis
    • financial impact quantification
    • dynamic modelling
  • Highlighting the data requirements for climate risk management
    • data risk arising from climate risk management that must be managed
      • data quality
      • data relevance
      • emerging metrics
      • projections and uncertainty
      • third part dependencies
    • Integration into existing risk management processes
    • Integration with risk management frameworks

  2:50 p.m. – End of Conference