Databricks SME
Remote · United States or Atlanta, Georgia, United States
Job Summary
Databricks SME responsible for architecture and platform engineering of Databricks Lakehouse environments. Designs enterprise architectures for batch, streaming, analytics, and ML workloads; standardizes cluster/compute/workspace strategies; builds multi-workspace dev/test/prod architectures; ensures scalability, availability, and performance. Applies Databricks Well-Architected Framework across security/governance, reliability, performance, cost, and operational excellence. Leads Databricks workspace, cluster, and Unity Catalog implementations; builds ETL/ELT pipelines with Spark/SQL; integrates with cloud services; establishes CI/CD for notebooks, jobs, and infrastructure. Defines governance, data classification, lineage, and audit strategies; enforces least-privilege access and secure networking; supports compliance (HIPAA, SOC 2, PCI, GDPR as applicable). Monitors platform health, troubleshoots production issues, tunes performance, and defines SLOs/operational metrics. Partners with Data Eng, Analytics, ML, Platform, and Security teams; communicates risks and recommendations to leadership; provides architectural leadership and mentorship. Nice-to-have: MLflow/MLOps, real-time analytics, multi-region/cross-account data architectures, consulting/MSP delivery.
Required Qualifications
- 7+ years in data engineering, analytics, or platform architecture
- 3–5+ years hands-on Databricks experience in production environments
- Experience applying the Databricks Well-Architected Framework
- Experience designing cloud-native lakehouse architectures
- Experience supporting mission-critical data platforms
- Databricks Lakehouse Platform experience
- Apache Spark (PySpark / Scala / Spark SQL)
- Delta Lake, Delta Live Tables, Structured Streaming
- Unity Catalog governance and access controls
- Cloud platforms: AWS, Azure, or GCP
- Infrastructure as Code (Terraform strongly preferred)
- CI/CD tools (GitHub Actions, Azure DevOps, GitLab, etc.)
- Data formats and protocols (Parquet, JSON, Avro)
- Databricks Certified Data Engineer Professional
- Databricks Certified Professional Architect (or equivalent advanced certification)
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.