Staff Data Engineer
RemoteCanada
Job Summary
Design and maintain scalable data pipelines and ETL/ELT processes while architecting optimized data models and storage solutions for analytics and operational use. Collaborate with data scientists and engineers to deliver trusted datasets, own platform evolution using Databricks and Spark, and implement observability, alerting, and data quality monitoring for critical workflows. Drive best practices in documentation, testing, and CI/CD, contributing to the design of next-generation data lakehouse architectures. Partner with stakeholders to ensure data solutions support business outcomes, helping drive long-term architectural vision and mentor the team on engineering standards.
Required Qualifications
- 8+ years of experience as a Data Engineer or in a similar backend engineering role
- Bachelor's degree in Computer Science, Engineering, or a related technical field
- Databricks Optimization: Tune Spark jobs, optimize join performance, and manage Delta Lake architecture for batch and streaming data
- Experience leveraging AI-assisted development tools and AI/ML technologies to improve data engineering workflows, developer productivity, data quality and ops
- Strong programming skills in Python, Scala, or Java
- Hands-on experience with distributed data systems like Spark or Kafka
- Proficient in writing complex SQL and NoSQL queries and optimizing queries for performance
- Experience building and maintaining robust ETL/ELT pipelines in production
- Understanding of data modeling techniques (star schema, dimensional modeling, etc.)
Desired Qualifications
- Familiarity with software supply chain, cybersecurity, or large-scale software ecosystem data
- A track record of improving data platform reliability, scalability, performance, and cost efficiency
- Familiarity with workflow orchestration tools (Airflow, Dagster, or similar)
- Hands-on experience with cloud data platforms, particularly AWS
- Familiarity with modern table formats such as Delta Lake, Apache Iceberg, or Apache Hudi
- Experience implementing data observability, lineage, governance, and automated data quality frameworks
- Experience designing real-time or streaming data architectures using data lake technologies
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.