Senior Data Engineer - Azure Databricks
RemoteColombia or Costa Rica
Job Summary
Own end-to-end delivery of data pipelines and products from design through production support. Develop, maintain, and optimize scalable pipelines using Azure Databricks, DBT, and Python while executing large-scale migrations from Snowflake to Azure. Build infrastructure-as-code solutions with Terraform and ensure seamless integration across enterprise platforms. Implement data quality frameworks, testing, and validation processes to enable observability through monitoring and alerting. Contribute to scalable Lakehouse architecture designs and monitor production environments for performance and reliability. Collaborate with cross-functional teams and mentor junior engineers through code reviews.
Required Qualifications
- 5 to 8+ years of experience in Data Engineering within enterprise-scale environments
- Strong hands-on expertise with Azure Databricks, DBT, and Snowflake
- Strong proficiency in Python and SQL
- Proven experience building and managing large-scale data pipelines and transformations
- Experience working with high-volume datasets and complex data models
- Solid understanding of data modeling, ETL/ELT frameworks, distributed systems, and Lakehouse architectures
- Experience with Terraform and Infrastructure as Code (IaC) principles
- Strong experience with CI/CD pipelines and Git-based version control
- Experience supporting production systems, including monitoring, alerting, and incident resolution
- Knowledge of data quality, testing, and observability frameworks
- Excellent analytical, problem-solving, and communication skills
Desired Qualifications
- Experience in insurance or financial services environments
- Experience with real-time and streaming technologies such as Kafka, Event Hubs, or Delta Live Tables
- Knowledge of security, compliance, and data protection standards in regulated industries
- Experience with cloud cost optimization and performance tuning
- Exposure to multi-cloud or hybrid architectures
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.