Senior Data Engineer
$120,000–$160,000 year
Hybrid · Chicago, Illinois, United States
Job Summary
Senior Data Engineer responsible for owning the iMData platform and pipelines on the Databricks-based data architecture. Lead the design and implementation of the data platform, mentor junior engineers, and collaborate directly with business stakeholders to gather requirements and deliver scalable data solutions. Architect and build the Databricks Datamap platform and MCP connector; implement medallion architecture with PySpark ETL to load data into the Enterprise Data Platform; design gold-layer star schemas to support AI, analytics, and reporting workloads; drive engineering best practices (code standards, testing, documentation, pipeline design patterns); integrate data from enterprise SaaS apps like Salesforce and Zendesk; perform one-time migrations from Synapse to Databricks; translate business requirements into deliverables; work in an Agile environment; ensure data security and privacy; develop CI/CD workflows; maintain Infrastructure as Code with Terraform. This role also emphasizes mentoring and collaborating with Machine Learning Engineers and stakeholders to deliver critical data assets.
Required Qualifications
- Bachelor’s degree or higher in Computer Science or equivalent
- 5–7 years of experience in data engineering
- Hands-on production experience with Databricks, Delta Lake, and medallion architecture
- Familiarity with Terraform for infrastructure as code
- Understanding of CI/CD practices, specifically GitHub Actions and Databricks Asset Bundles (DABs) for deployment
- Deep expertise in PySpark
- Strong data modeling skills including star schemas in gold layers
- Ability to work independently and collaborate across teams
- Comfort working with data analysts, data scientists, and business stakeholders
- Willingness to learn new technologies and contribute creatively
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.