Data Scientist, AI Model Risk
On-siteToronto, Ontario, Canada
Job Summary
Validate LLM-based applications and agentic AI systems, while also assessing traditional machine learning models including classification, regression, and natural language processing. Challenge models to identify conceptual and empirical risks, exploring considerations such as metric reproducibility, uncertainty quantification, fairness, and explainability. Collaborate with cross-functional stakeholders across Internal Audit, Cybersecurity, and Technology Operations to establish best practices for MLOps and IT infrastructure. Read research papers to enhance validation methodologies, develop reusable software packages, and contribute to the team's knowledge pool. Work within the Enterprise Model Risk Management team to assess and manage emerging model risks associated with RBC's AI capabilities.
Required Qualifications
- Must-have Passionate about learning and staying up-to-date with research and technology
- Must-have Strong communication and interpersonal skills
- Must-have Progress towards a PhD or Master's degree in Statistics, Computer Science, Applied Mathematics, Econometrics, Engineering, Quantitative Finance, or a related quantitative field
- Must-have Proficient programming skills in Python
- Must-have Familiarity with popular LLMs and agentic frameworks
Desired Qualifications
- Nice-to-have A risk-oriented mindset: You are curious about the 'how' as well as the 'why'
- Nice-to-have Publication or prior research experience (applied or fundamental)
- Nice-to-have Experience with version control systems
- Nice-to-have Comfortable with command line tools
- Nice-to-have Familiarity with popular machine learning frameworks and libraries
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.