Tech Lead - Databricks & Modern Data Stack
On-siteAhmedabad, Gujarat, India
Job Summary
Architect and build Databricks-based data platforms for US mid-market clients, designing medallion architectures, Delta Lake pipelines, and Unity Catalog governance. Lead a pod of 3 to 5 data engineers across 1 to 2 engagements, conducting code reviews, mentoring staff, and interviewing new hires. Translate business requirements into technical solutions while supporting pre-sales through scoping calls and effort estimates. Own technical decisions on architecture, tooling, and trade-offs, and contribute to internal IP like reusable accelerators and reference architectures. Write production PySpark and SQL, set engineering standards for CI/CD and IaC, and co-author technical blog posts. This hands-on role balances 60% deep technical work, 30% team leadership, and 10% client support within a brand new practice focused on the modern data stack.
Required Qualifications
- 6 to 10 years in data engineering
- at least 3 years on Databricks or Spark at production scale
- Strong PySpark and SQL
- ability to write, debug, and optimize Spark jobs without help
- Databricks ecosystem fluency: Delta Lake, Unity Catalog, Workflows or DLT, MLflow basics
- Databricks Certified Data Engineer Associate or Professional
- willingness to get certified within 60 days of joining
- One cloud at strong level, ideally AWS or Azure
- Comfort with IAM, networking basics, and Terraform or equivalent IaC
- Modern data stack exposure beyond Databricks: Airflow, dbt, Fivetran, or similar
- Prior lead or senior engineer experience
- led at least one team of 3+ engineers on a real project
- ability to run a code review, push back on bad design, and unblock a junior
- Solid English, written and spoken
- ability to be on calls with US clients regularly
Desired Qualifications
- Databricks Certified Data Engineer Professional or ML Associate
- Experience working with US clients in a services or consulting setting
- GenAI or LLM work on Databricks (Mosaic AI, vector search, RAG pipelines)
- DevOps fluency: CI/CD pipelines, Docker, Kubernetes, observability tooling
- Open source contributions, technical writing, or conference talks
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.