[Data - FR] Senior Machine Learning Engineer - Orchestration
HybridParis, Île-de-France, France
Job Summary
Define and own the evaluation strategy for the AI agentic system, including metrics, protocols, datasets, and tooling. Implement and maintain automated pipelines to monitor model quality, safety, and alignment across iterations. Run systematic experiments to assess reasoning, factuality, robustness, and user experience while collaborating with model developers and research scientists to drive iterative improvement. Contribute to internal research and knowledge sharing on LLM evaluation methodologies. This permanent full-time role is hybrid in Levallois, with an immediate start date for a Senior/Staff engineer joining a cross-functional team of ML Engineers, Product Engineers, and Medical Experts.
Required Qualifications
- MSc or PhD in Computer Science, Machine Learning, Data Science, or related field
- 7+ years of hands-on experience working with large language models (e.g., GPT, Claude, Llama, or BERT-like architectures)
- Proven experience in evaluating agentic or reasoning systems (e.g., autonomous agents, tool-using LLMs, dialogue systems, or task-oriented assistants)
- Strong track record in experiment design, metric definition, and evaluation automation
- Ability to bridge research and production, influencing modeling and product decisions
- Excellent communication skills and a collaborative mindset
Desired Qualifications
- experience in the clinical or medical domain and sensitivity to ethical or regulatory challenges in healthcare AI
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.