Data Engineer
RemoteUnited States
Job Summary
Design and implement scalable data ingestion and processing pipelines using AWS services and modern data tools. Perform data ingestion, validation, and transformation using existing frameworks, processing structured and semi-structured data via batch methods to store in S3. Transform and load data into Redshift while managing operations with DBT, and build workflow orchestration using Airflow or Step Functions. Export processed data to external systems and implement validation, auditing, and monitoring to ensure reliability. Create configurable pipeline steps for non-technical users and support containerized deployments with Docker. Requires 8–11 years of experience in Data Engineering with strong proficiency in Python, PySpark, and SQL.
Required Qualifications
- 8–11 years of experience in Data Engineering or related roles
- Strong hands-on experience with AWS services such as Glue, Lambda, S3, and Step Functions
- Proficiency in Python, PySpark, and advanced SQL
- Experience with DBT for data transformation and modeling
- Expertise in workflow orchestration tools like Airflow or Step Functions
- Hands-on experience with cloud data warehouses such as Redshift, Snowflake, or BigQuery
- Experience handling structured and semi-structured data pipelines
- Knowledge of Shell scripting and containerization using Docker
- Basic understanding of data governance and DevOps deployment practices
Desired Qualifications
- Familiarity with Terraform for infrastructure management (nice to have)
- Strong problem-solving skills and ability to work in fast-paced environments
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.