Senior Software Engineer, AI Agent & LLM
$185,000–$230,000 year
HybridMountain View Santa Clara County, California, United States
Job Summary
Build AI-agent experiences for Otter AI Chat that help users reason over conversations, retrieve knowledge, and complete complex tasks. Advance the conversational knowledge engine through better extraction, annotation, and indexing. Develop robust evaluation datasets, automated graders, regression tests, and release gates for AI quality. Diagnose failures across models, prompts, retrieval, tools, data pipelines, backend services, and product workflows. Build shared agent infrastructure for orchestration, tracing, debugging, retries, sandboxed execution, and observability. Improve task completion, correctness, groundedness, reliability, latency, and cost. Turn production traces, customer feedback, and usage signals into measurable product and model improvements. Collaborate across AI, product, infrastructure, data, security, and application teams to deliver end-to-end capabilities.
Required Qualifications
- 5+ years of AI Agent engineering, machine learning engineering, or related experience
- strong backend or distributed-systems engineering skills
- hands-on experience building and shipping AI-agent or LLM-powered products
- a strong focus on AI quality and experience evaluating nondeterministic systems
- ability to use data, traces, logs, and qualitative examples to identify and resolve complex failures
- ability to work effectively across services, technical domains, and organizational boundaries
- ability to combine strong product judgment with rigorous engineering and evaluation practices
- ability to operate with high agency, strong ownership, and a bias toward action
- ability to take ambiguous problems from initial exploration through production launch and continuous improvement
Desired Qualifications
- Demonstrated ability to use coding agents effectively while rigorously reviewing and controlling the quality of their output
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.