AgentOps Engineer
$50,000–$70,000 year
On-siteMohali, Punjab, India
Job Summary
Deploy and maintain LLM-powered multi-agent systems across cloud and on-prem environments while integrating agents with enterprise APIs and knowledge bases. Implement observability frameworks to track performance, latency, cost, and accuracy through execution traces and context windows. Fine-tune configurations for cost efficiency and scalability, deploying fallbacks, guardrails, and redundancy mechanisms to ensure reliability. Build automated pipelines to evaluate agent performance, safety, and compliance, feeding results into continuous improvement loops with ML/AI engineers. Ensure agents adhere to enterprise security, governance, and compliance standards by collaborating with Responsible AI teams on trust and audit mechanisms. Work with AI engineers, DevOps, and product managers to align operational reliability with business outcomes, supporting customer deployments with customized monitoring and tuning.
Required Qualifications
- 2–5 years of experience in DevOps, MLOps, or AI systems engineering
- Hands-on with LLM orchestration frameworks (LangChain, LangGraph, LlamaIndex)
- Familiarity with AgentOps tools (LangSmith, PromptLayer, Weights & Biases, Arize AI)
- Proficiency in Python and scripting for automation
- Experience with cloud platforms (AWS, GCP, Azure) and containerization (Docker, Kubernetes)
- Knowledge of monitoring & observability tools (Prometheus, Grafana, ELK, OpenTelemetry)
- Understanding of RAG pipelines, vector databases, and context orchestration
Desired Qualifications
- Exposure to multi-agent orchestration (MCP, A2A messaging, AgentBridge)
- Experience in Responsible AI / model evaluation frameworks
- Familiarity with CI/CD pipelines for AI models and agents
- Background in BFSI, GRC, SOC, or enterprise SaaS systems
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.