Principal Data Engineer
$197,300–$313,700 year
On-siteSan Francisco, California, United States
Job Summary
Architect, build, and scale foundational data platforms, lakehouses, and high-throughput pipelines powering AgentExchange product analytics and agentic workflows. Design and deploy production-grade data pipelines, feature stores, and event-driven architectures enabling autonomous AI agents to operate reliably at enterprise scale. Establish end-to-end data governance, quality frameworks, lineage tracking, and performance monitoring to ensure zero-downtime reliability for mission-critical data assets. Serve as the principal technical authority on data architecture, partnering with decision scientists, software engineers, and product managers to translate complex business needs into elegant technical systems. Co-own the long-term data technology roadmap, anticipating scale bottlenecks and driving architectural evolution ahead of product growth. Elevate the engineering bar across Cloud Analytics by establishing standards for code quality, testing, CI/CD, and system design while mentoring engineers.
Required Qualifications
- 7+ years of hands-on data engineering experience building complex, enterprise-scale data platforms, distributed systems, and real-time data pipelines
- Must have a track record of actively shipping production code alongside architectural leadership
- Mastery of modern distributed computing, data lakehouse architectures, and cloud data warehouses
- Deep expertise in orchestration tools (e.g., Airflow, Dagster)
- Demonstrated experience engineering data systems for LLMs, agentic workflows, feature stores, or vector retrieval pipelines
- Expert proficiency in Python, Scala, or Java
- Expert-level SQL tuning
- Deep familiarity with software engineering best practices (Docker, Git, CI/CD, IaC)
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related technical discipline (or equivalent practical impact)
- Strong communication skills with a proven track record of distilling complex system architecture decisions into clear business trade-offs for technical and non-technical executives alike
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.