Data Engineer
HybridBristol, England, United Kingdom
EXPIREDBristol, England, United KingdomHybridFull TimeLarge
Full TimeLarge
Job Summary
Design and build scalable ETL pipelines, data warehouses, and data lakes on AWS using Java, Python, Scala, Spark, and SQL. Extract data via APIs, process large structured and unstructured datasets, and integrate information from multiple sources within big data architectures. Collaborate cross-functionally with product, design, and client stakeholders to solve complex business challenges using agile best practices and live whiteboarding sessions. Work hybrid, attending a minimum of two days in the office or on client sites, with assignments varying up to five days on-site.
Required Qualifications
- You thrive in problem-solving and analytical thinking
- You enjoy collaborating with multiple stakeholders in a fast-paced environment
- Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL
- Experience performing tasks such as writing scripts, extracting data using APIs, writing SQL queries etc
- Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources through ingestion and curation functions on AWS cloud using AWS native or custom programming
- Hybrid working with the team on client site or in our office a minimum of two days per week
- Experience in AWS cloud technologies for ETL pipeline, data warehouse and data lake design/building and data movement
- AWS data and analytics services (or open-source equivalent) such as EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.