Model Risk Management for Financial Institutions

Model Risk Management For Financial Institutions

Program – May 26

9:00 a.m.  –  9:05 a.m.
Opening Remarks from the Chair

David Harris, Partner, PricewaterhouseCoopers LLP

9:05 a.m. – 10:30 a.m.
Understanding OSFI Model Risk Management Guideline (E-23) and Pending Changes in Draft Guideline

David Harris, Partner, PricewaterhouseCoopers LLP
OSFI (Invited)

• Overview of Guideline E-23 and Draft Guideline E-23 
• Explanation of the regulatory definition of models
• Identifying models used in financial enterprises 
  • where are they used
  • how are they used and understanding the functions they perform

• Scope application to models – E-23 and Draft Guideline
• Explanation of the model lifecycle – key stages and processes
• Model Risks OSFI wants managed – financial and non-financial 
  • issues raised by AI and Machine Learning

• Required steps for model risk management
• Disclosure and reporting – what you need to know

10:30 a.m. – 10:45 a.m. – Morning Networking Break

10:45 a.m. – 12:00 p.m.
Examining Regulatory Regimes Affecting Models, Model Risk, AI/ML and Data – Canada, US, International

Claire Feltrin, Counsel, Borden, Ladner, Gervais LLP 

Regulatory Treatment of Models and Model Risk in Financial Institutions

• Highlighting regulations and statutes affecting models and model risk (including related issues)
• Examining overlap of parallel requirements both Canadian and foreign 
• How to develop a comprehensive compliance and reporting process 
• Understanding differences in terms and objectives of differing requirements
• Establishing primary or coordinated governance responsibilities
• Ensuring your oversight framework is complete, comprehensive and remains up to date
• Implications of inadequate oversight/governance

Restrictions on Data Usage – OSFI, Canadian and International Regulations/Laws

• Legislative restrictions on data usage 
  • PIPEDA
  • Federal and provincial privacy laws
  • GDPR
  • Anti-spam regulations

• Specific legal and regulatory requirements for financial institutions
  • Bank Act
  • OSFI requirements for banks, insurers and pension funds including data governance
  • FSRA guidance for credit unions
  • provincial laws and regulations for financial services

• Emerging trends and challenges 
  • data used for artificial intelligence and machine learning
  • cross-border data transfers/usage
  • OSFI operational resilience compliance
  • increasing penalties for non-compliance

12:00 p.m. – 1:00 p.m. – Luncheon

1:00 p.m. – 2:00 p.m.
Data and Digital Risks for Effective Management

Nicolas Gemin, Risk Leader for Insurance, Deloitte LLP
Andreas Witz, Data Risk and Compliance Leader, Deloitte LLP

• Importance of Data Risk for Banks and Insurers
• OSFI E-21 Data Risk and Impact on Bank and Insurers
• Addressing Common Data Risk Challenges
• Evolving Data Governance for Modern Data Risk Management 
• Building a Mature AI Risk Management Framework and AI Solutions
• Leveraging Data Risk as Business Advantage

2:00 p.m. – 2:45 p.m.
Impact of E-23 on Organizational Structure/Processes and Governance

Carla Crichlow, Senior Manager, Actuarial Consulting, Oliver Wyman
Dean Rootenberg, Senior Principal, Actuarial Life Consulting, Oliver Wyman

• Getting E-23 structure in place – establishing working group and implementation plan
• Delineating, and implementing a model risk management framework
  • identifying the essential elements of the framework
• Understanding organizational model/data exposures including reputational risk
• Implementing tools to measure/monitor model risk including data governance
• Inclusion of model related risks into the risk appetite framework
• Defining roles and responsibilities relative to other parts of the organization, e.g., business, audit, compliance
• Inclusion of model related risks into the risk appetite framework
• Development of KPI’s 
• Risk management oversight – what needs to be done?
• New challenges for model risk management and oversight

2:45 p.m. – 3:00 p.m. – Afternoon Networking Break

3:00 p.m. – 4:00 p.m.
Governance and Internal Controls Related to Model Risk Reporting

Ulana Oswald, Partner, Ernst & Young LLP
Wissem Bouraoui, Senior Manager, Ernst & Young LLP

• Role of internal controls for model risk management and reporting
• Developing a reporting control framework methodology including non-financial considerations 
• Entity-level internal controls related to models and model risk including
  • managing changes to models
  • decommissioning of models
  • ongoing matters affecting model data – new data, obsolete data
  • completeness of model inventory
  • Defining the three lines of defense in model risk management
  • Reliance on 3rd party service providers for model and data services
  • evaluation of controls in place for service providers

• Applying the COSO framework for model risk reporting controls
• Data integrity and accountability 
• Continuous monitoring of processes and controls

4:00 p.m. – 5:00 p.m.
MRM Documentation Summary and Critical Highlight

Alison Rose, Partner – Life and Pensions Actuarial, KPMG LLP
Simmy Leung, Senior Manager, Actuarial, Risk and Insurance Services, KPMG LLP

• Summary of regulatory requirements for model risk management documentation
• Main topics of documentation and key elements
  • model development
  • model validation
  • model governance and oversight 
  • model performance and monitoring
  • model risk reporting and communication
  • regulatory compliance
  • stress testing and scenario analysis
  • model retirement and decommissioning

• Items to consider not addressed by specific headings
• Relationship to data management and governance documentation expectations
• Who owns the documentation process?
• Linking documentation to financial/regulatory reporting

5:00 p.m. – End of Day 1

Program – May 27

9:00 a.m.  –  9:05 a.m.
Opening Remarks from the Chair

David Harris, Partner, PricewaterhouseCoopers LLP

9:05 a.m. – 10:10 a.m.
“Fireside Chat” – Practical Perspectives on MRM and E-23 – Lessons Learned in Banking and Insurance

David Harris, Partner, PricewaterhouseCoopers LLP
Mark Jarvis, Assistant Vice President – Corporate Model Risk Management, Sun Life
Satish Kumaraswami, Vice President – Global Model Risk Management, Scotiabank

10:10 a.m. – 11:00 a.m.
Examination of Tools for Model Risk and Data Risk Management

Faran Bahri, Manager, Ernst & Young LLP
Shini Dewwath, National Leader – Risk Analytics Enablement, Ernst & Young LLP

• Model risk governance platforms 
• Data management and quality tools 
• Model development and deployment tools 
• Model validation tools 
• Model monitoring and performance management tools
• Explainability and bias detection tools
• Stress testing and scenario analysis tools 
• Workflow automation and collaboration tools
• Regulatory compliance tools
• AI/ML-specific MRM tools

11:00 a.m. – 11:15 a.m. – Morning Networking Break

11:15 a.m. – 12:15 p.m.
Model Validation – Traditional Models, Gen AI and Others

Maximillian Boermann, Senior Manager, Risk Consulting Services, KPMG LLP
David Tang, Senior Manager, Data Analytics and Data Science, KPMG LLP

Model Validation

• Regulatory background for model performance validation
• Focus topics in model validation 
  • Documentation and transparency for internal and external communication
  • Theoretical review
  • Performance testing metrics including back-testing

Unique Challenges for Gen AI Validation

• • Documentation and transparency for internal and external communication 
    • Fairness audits for ethical and bias risks
    • Regulatory uncertainty
  • Difference in theoretical review between traditional and Gen AI model validation 
  • Performance review
    • Model comparison
    • Validation of non-deterministic outputs

12:15 p.m. – 1:15 p.m.  –  Luncheon

1:15 p.m. – 3:15 p.m.
Sector-Specific Case Studies in Model Risk Management – Banking and Insurance

• Insurance 
  Liability Models 
  Actuarial Models 
  Financial Condition Testing
• Banking
  Credit Risk
  Market Risk Models
  Stress Testing and Scenario AnalysisAI/ML and Decision Making Automation