Senior Data Engineer (Python)
$96,550–$138,061 year
On-siteIrvine, California, United States
Job Summary
Design and maintain scalable Python-based data pipelines for ingesting, processing, and transforming large volumes of data within Lakehouse and Medallion Architecture frameworks. Build and manage Apache Airflow workflows for scheduling, monitoring, and troubleshooting production jobs while developing reusable frameworks to standardize ETL/ELT solutions across databases, APIs, and streaming sources. Apply rigorous data quality checks, validation rules, and reconciliation processes to ensure accuracy, optimizing pipelines for performance, fault tolerance, and cost efficiency. Collaborate with cross-functional teams to design data models supporting analytics, reporting, and machine learning use cases, implementing CI/CD best practices and monitoring dashboards for operational health. Manage production workloads through root cause analysis and performance tuning, ensuring adherence to data governance, security, and compliance standards.
Required Qualifications
- 7+ years of experience in data warehouse or MDM applications
- Bachelor's degree in computer science, Information Technology, or related field
- Extensive knowledge of SQL and experience with relational databases
- Strong programming and scripting skills, including Python and PL/SQL
- Hands-on experience developing ETL/ELT data pipelines and supporting production data workloads
Desired Qualifications
- Experience implementing Lakehouse architectures and Bronze/Silver/Gold (Medallion) data models
- Experience with cloud-based data platforms such as Azure Data Lake, AWS, or GCP
- Familiarity with Spark/PySpark and distributed data processing frameworks
- Experience with Git, CI/CD pipelines, and DevOps practices
- Experience designing reusable frameworks for data ingestion, transformation, validation, and monitoring
- Familiarity with data governance, security, compliance, metadata management, master data management, or reference data management standards
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.