Data Engineer
$106,000–$143,000 year
On-sitePasadena, California, United States
Job Summary
Design and optimize ETL/ELT pipelines to ingest data from various sources, implementing validation and cleaning processes to maintain a reliable data foundation. Collaborate with the Data Architect and stakeholders to build scalable infrastructure including data lakes and warehouses, while leveraging Docker and Kubernetes for deployment. Enforce data governance and security policies to ensure compliance throughout the data lifecycle, and support AI automation using LLMs and agents. Stay current with emerging technologies and best practices to continuously improve the data ecosystem. This role requires strong Python and SQL skills, experience with cloud platforms like AWS or Azure, and familiarity with Apache Spark and Airflow. Only applicants living in Southern California will be considered.
Required Qualifications
- Bachelor's degree in Computer Science, Data Science, or a related field
- Strong programming skills in Python or a similar language
- Proficiency in SQL and experience with relational and non-relational databases
- Demonstrated experience in designing scalable, cost-effective, and maintainable data systems (e.g., data lakes, warehouses, lakehouses)
- Proven ability to automate and orchestrate complex data pipelines and create resilient, self-healing data systems
- Solid understanding of data governance principles and experience implementing data security best practices
- Experience translating ambiguous business requirements into precise technical specifications
- Experience with cloud platforms like AWS, Azure, or Google Cloud
- Familiarity with data processing frameworks such as Apache Spark and experience with a data lake environment
- Experience with data orchestration tools like Apache Airflow
- Experience with Fabric and Databricks
- Knowledge of data modeling techniques and experience with a business intelligence tool such as Power BI or Tableau
- Must be available for weekend shifts
- Must be able to lift 50 lbs
- Valid driver's license
Desired Qualifications
- Master's Degree in Data Science or equivalent
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.