AI Agents Applied Engineer - Senior Associate
On-site · Brooklyn, New York, United States
Job Summary
Lead end-to-end lifecycle of LLM-based agents: defining research directions in multi-step planning, tool use, and safety; building production systems that meet latency, memory, and cost constraints; partnering with Product, Engineering, Design, and Risk teams to bring these systems to market. Responsibilities include researching and deploying agentic AI systems with multi-step workflows, tool calling, and multi-agent orchestration; fine-tuning and optimizing LLMs using parameter-efficient fine-tuning, distillation, and quantization; applying reinforcement learning and preference optimization to improve personalization and dialogue policies; scaling LLM systems through caching, batching, prompt governance, and evaluation frameworks; implementing privacy, safety, and security controls including PCI compliance, jailbreak resistance, and auditability; designing rigorous experiments with strong baselines and meaningful metrics; defining and tracking success metrics for agent performance such as task completion rate, accuracy, latency, and customer satisfaction.
Required Qualifications
- BS with 3+ years or MS with 2+ years building and deploying AI systems in production
- Applied GenAI experience with LLMs including fine-tuning, prompt engineering, and RAG.
- Experience scaling LLM systems with caching, batching, governance, and evaluation.
- Strong foundation in ML, deep learning, statistical modeling, and experimental design.
- Experience in Information Retrieval (indexing, ranking, retrieval) and/or recommendation systems.
- Proficiency in Python and ML frameworks (PyTorch/TensorFlow, Hugging Face, scikit-learn)
- Demonstrated ability to set a technical research agenda and drive it from concept through production deployment.
Desired Qualifications
- 2+ years developing conversational AI systems, virtual assistants or LLM-based systems in production
- Experience with multi-agent orchestration, supervisor agents, and specialized toolkits
- Experience with reinforcement learning, bandit algorithms, and preference-based optimization (DPO, IPO)
- MLOps/LLMOps experience with CI/CD, monitoring, versioning, A/B testing, and rollbacks
- Track record of data-driven product development and experimentation
- Publications in top-tier AI/ML venues and/or open-source contributions
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.