Principal Engineer
$216,000–$298,000 year
HybridSan Mateo, California, United States
Job Summary
Architect and build the core AI Agent Platform, including agent runtime, orchestration, tool invocation, and state management for high-throughput, multi-tenant workloads. Design agentic operations workflows where autonomous agents monitor telemetry, perform root-cause analysis, and execute remediation to shift operations from human-driven to agent-driven. Instrument the platform end-to-end with distributed tracing, metrics, and structured logging to define golden signals for system health and agent quality. Engineer resilient, cloud-native distributed systems using Kubernetes and event-driven microservices, while applying LLMs to operations for incident summarization and natural-language telemetry querying. Set technical direction for the self-operating platform and mentor engineers across the organization.
Required Qualifications
- 10+ years building production software
- deep experience designing and operating large-scale distributed systems and platforms
- Proven experience building scalable AI/agent platforms or high-throughput ML serving systems in production
- Hands-on experience designing agentic or autonomous workflows
- Strong AIOps and SRE background
- Hands-on experience applying LLMs to production workflows
- Strong proficiency in Python and a systems language (Go/Java)
- cloud-native architecture (Kubernetes, event-driven microservices, streaming pipelines)
- Working familiarity with RAG and knowledge systems
- For Principal level: demonstrated org-wide technical influence
Desired Qualifications
- ML-driven anomaly detection, alert correlation, and predictive operations at scale
- SRE leadership operating AI/ML or data-intensive platforms
- Familiarity with agentic frameworks (LangChain, LangGraph) and vector/graph stores
- AI safety and governance grounding for autonomous enterprise systems
- Contributions to open-source agentic, observability, or AIOps frameworks
- depth here is a plus, not the primary bar
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.