Sr Machine Learning Engineering Manager - AI Quality and Governance
$193,000–$308,000 year
RemoteUnited States
Job Summary
Lead a multidisciplinary team of software, machine learning, and quality engineers to establish end-to-end quality standards and governance capabilities for Workiva's AI platform and products. Define and drive a comprehensive quality strategy spanning unit, integration, and production testing for generative AI, agentic systems, RAG pipelines, and knowledge frameworks. Build a scalable, self-service evaluation platform enabling teams to create, manage, and reuse datasets, benchmarks, and metrics for offline and online experimentation. Translate Responsible AI principles into practical engineering controls, including traceability, lineage, risk classification, and auditable evidence for enterprise use. Partner with Product, Security, Legal, and Risk teams to embed evaluation and governance throughout the AI development lifecycle, ensuring systems are measurable, observable, and ready for regulated deployment.
Required Qualifications
- Bachelor's degree in Computer Science, Engineering, Data Science, or related field (or equivalent experience)
- 10+ years in software engineering, ML engineering, quality engineering, or related roles
- 4+ years leading an engineering team
- Strong software engineering and systems-design fundamentals
- Experience delivering and operating production SaaS or platform capabilities
- Demonstrated experience establishing automated quality practices for distributed, cloud-based products
- Practical understanding of the generative AI development lifecycle and challenges of evaluating nondeterministic systems
- Experience with generative AI concepts: LLMs, RAG, embeddings, vector/hybrid search, agents, tool use, and prompt orchestration
- Experience defining measurable quality criteria using data, experimentation, telemetry, and production signals
- Experience with cloud-native architectures on AWS, Azure, or GCP
- Proven ability to lead senior individual contributors, navigate technical disagreements, and build high-performance cultures
- Strong communication and cross-functional leadership skills
- Willingness to travel up to 15% for team and corporate meetings
- Reliable internet access for remote work
Desired Qualifications
- Master's degree in Computer Science, Engineering, ML, Data Science, or related field
- Experience building or operating AI/ML evaluation, experimentation, observability, model-governance, or ML platform capabilities
- Experience evaluating RAG and agentic systems, including retrieval quality, groundedness, task completion, tool use, and safety
- Familiarity with evaluation techniques: golden datasets, statistical metrics, model-based graders, human evaluation, red teaming, A/B testing, and drift/regression detection
- Working knowledge of ML/AI lifecycle practices: dataset management, model/prompt versioning, experiment tracking, deployment, monitoring, and feedback loops
- Experience translating Responsible AI, model-risk, privacy, security, or regulatory requirements into scalable engineering controls
- Familiarity with AI risk/governance frameworks (NIST AI RMF, ISO/IEC 42001, or comparable)
- Experience with Kubernetes, microservices, CI/CD, infrastructure as code, and modern DevOps/MLOps practices
- Experience supporting enterprise software in regulated or high-assurance environments
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.