LLM Operations Engineer
On-sitePune, Maharashtra, India
Job Summary
Design and build multi-agent AI systems capable of planning, reasoning, and task execution using LLMs with advanced prompt engineering and orchestration. Implement Agentic workflows including planner-executor-critic loops and RAG pipelines with vector databases for enterprise knowledge grounding. Develop tool-using agents that integrate with APIs and deploy scalable solutions using MLOps & LLMOps practices. Ensure AI safety, governance, and responsible AI practices while optimizing performance through fine-tuning and caching strategies. This role focuses on building LLM-powered, agent-driven architectures that can reason, collaborate, and execute complex workflows across enterprise systems.
Required Qualifications
- Generative AI
- Machine Learning Operations
- Large Language Models (LLMs)
- Agentic AI
- Minimum 3 year(s) of experience
- 15 years full time education
- Experience building multi-agent orchestration systems with role-based coordination
- Exposure to agent planning algorithms (ReAct, Plan-and-Execute, Tree of Thought)
- Experience with LLM evaluation frameworks (RAGAS, TruLens, Promptfoo)
- Knowledge of graph-based reasoning, knowledge graphs
- Building autonomous systems or copilots in enterprise environments
- Domain experience in industrial, energy, or IoT environments
- Systems thinking for designing autonomous AI architectures
- Strong problem decomposition for agent task design
- Ability to balance latency, cost, and accuracy in LLM systems
- Communication with business stakeholders to translate workflows into agent pipelines
- Innovation mindset with focus on applying agentic AI in production
- 3–8 years' experience in AI/ML with strong focus on Generative AI
- Strong Python development skills
- Hands-on experience with LLMs & GenAI frameworks
- Hands-on experience with OpenAI, Hugging Face Transformers
- Hands-on experience with Agent frameworks: LangChain, AutoGen, CrewAI, Semantic Kernel
- Hands-on experience with RAG pipelines & vector DBs- FAISS, Pinecone, Weaviate
- Experience building API-driven, tool-integrated AI agents
- Strong understanding of Prompt engineering & prompt optimization
- Strong understanding of Chain-of-thought reasoning and tool augmentation
- Strong understanding of Context management and token optimization
- Experience with cloud platforms (Azure OpenAI preferred, AWS/GCP acceptable)
- Knowledge of Docker, Kubernetes, CI/CD pipelines
- This position is based at our Pune office
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.