16194 - Senior Data Engineer (Databricks & Cloud Analytics)
On-siteSalt Lake City, Utah, United States
Job Summary
Design, develop, test, deploy, and maintain enterprise data engineering solutions using Databricks, Apache Spark, Azure Data Factory, and Kafka. Build scalable ETL/ELT pipelines, high-performance data ingestion frameworks, and streaming architectures while optimizing SQL queries and stored procedures. Monitor production pipelines, perform root cause analysis, and implement data quality, governance, and security best practices. Manage source code via Git within CI/CD workflows and collaborate with architects and cross-functional teams in Agile ceremonies to translate business requirements into technical solutions. Mentor junior engineers and create technical documentation and operational runbooks.
Required Qualifications
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related technical discipline (or equivalent professional experience)
- 6+ years of experience designing, developing, and supporting enterprise data platforms, cloud analytics solutions, or large-scale data engineering initiatives
- Strong hands-on experience with: Databricks, Apache Spark (PySpark), Python, SQL, Scala
- Experience working with the Databricks ecosystem, including: Databricks Workspaces, Delta Lake, Unity Catalog, Delta Live Tables (DLT), Databricks SQL, MLflow, Databricks Jobs
- Strong SQL development experience, including query optimization and performance tuning
- Experience designing and implementing ETL/ELT solutions using Azure Data Factory or comparable cloud integration platforms
- Experience with Apache Kafka or other event streaming technologies
- Experience integrating enterprise applications using REST APIs, JSON, XML, and SOAP web services
- Experience with Azure Data Lake Storage (ADLS Gen2) or comparable cloud storage platforms
- Experience using Git for source code management and collaborative software development
- Experience working within Linux environments, including shell scripting and command-line utilities
- Strong analytical, troubleshooting, and problem-solving skills
- Experience working in Agile environments utilizing Scrum, Kanban, or SAFe methodologies
- Demonstrated ability to manage multiple priorities while delivering high-quality solutions
- Proven ability to work independently while mentoring teammates and contributing to technical leadership
Desired Qualifications
- Familiarity with Infrastructure as Code (IaC) tools such as Terraform
- Experience implementing CI/CD pipelines using Azure DevOps, GitHub Actions, or similar DevOps platforms
- Microsoft Azure cloud services
- Azure Synapse Analytics
- Microsoft Fabric
- Power BI
- Data governance and metadata management
- DataOps and MLOps practices
- Enterprise data warehousing
- Master Data Management (MDM)
- Financial services or public sector data environments
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.