Senior AI Engineer
On-siteLeeds, England, United Kingdom
Job Summary
Design and build eval and observability pipelines using platforms such as Braintrust, LangSmith, Arize, or Weights & Biases. Curate golden datasets and design LLM-as-judge and human-in-the-loop scoring pipelines. Build production observability for AI features including token-level tracing, quality signal monitoring, hallucination detection, and cost attribution. Wire eval gates into CI/CD promotion paths to enforce quality checks before release. Transform eval data into actionable decisions for engineering and product teams by interpreting scores and distinguishing signal from noise. This hands-on technical role focuses on evaluation, experimentation, and observability within the AI Engineering team.
Required Qualifications
- Strong Python skills
- Hands-on AI engineering experience, particularly around evals and observability
- Experience using Braintrust, LangSmith, Arize, Weights & Biases, or a similar platform in production
- Ability to design scoring pipelines and sanity-check whether a score means anything
- CI/CD fluency to wire eval gates into a promotion pipeline
- Ability to communicate what the data means clearly enough that engineering and product teams act on it
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.