Sr Data Engineer
$141,900–$199,400 year
On-site · New York City, New York, United States or Seattle, Washington, United States
Job Summary
Senior Data Engineer to design, build, and optimize data pipelines and transformation frameworks that support high-visibility acquisition reporting and marketing analytics use cases. You will work across AWS, Databricks, Unity Catalog, Snowflake and Airflow to create reliable, scalable solutions for ingesting and modeling marketing platform data. You will collaborate with product, analytics, marketing operations, and stakeholder teams to ensure data accuracy, reliability, SLAs, and transparency for downstream dashboards. Key responsibilities include architecting, building, and maintaining scalable ETL/ELT pipelines for acquisition reporting using Databricks, PySpark, SQL, and Unity Catalog; leading the modernization to migrate Snowflake-based SQL scripts to Databricks UC; designing robust ingestion frameworks; implementing data quality checks and automated remediation; developing metadata-driven components; partnering with Data Reliability Engineering to integrate SLA-based incident detection and recovery workflows; collaborating with analytics and marketing stakeholders to ensure reliable dashboard data; improving pipeline performance, logging, and observability; contributing to engineering best practices and code reviews; mentoring junior engineers.
Required Qualifications
- 5+ years of experience as a Data Engineer or similar role
- Strong proficiency in SQL (analytical SQL, complex joins, window functions)
- Hands-on experience with PySpark and/or Spark SQL in production
- Strong understanding of data modeling, ETL/ELT design patterns, and distributed data processing
- Experience building pipelines in Databricks, including Delta Lake, Unity Catalog, data governance, and Lakehouse patterns
- Strong experience in AWS (S3, IAM, EC2, Glue, Lambda, or related services)
- Proficiency with Airflow or similar orchestration tools
- Experience building robust ingestion pipelines and working with semi-structured formats (JSON, Parquet, CSV)
- Experience with Git/GitHub, CI/CD, and modern DevOps practices
- Bachelor’s degree in computer science, Information Systems or related field
- Preferred Qualifications Master’s degree in computer science, Information Systems or related field a plus
- Experience with marketing or customer acquisition data (Meta, Google Ads, Google CM360, TikTok, Twitter, Snapchat, Branch, AppsFlyer, Salesforce, etc.)
- Familiarity with data observability, SLA monitoring, incident workflows, or reliability engineering concepts
- Exposure to data quality frameworks (Great Expectations, Deequ, Monte Carlo, or custom frameworks)
Additional Requirements
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.