Data Engineer II
On-siteMumbai, Maharashtra, India
Mumbai, Maharashtra, IndiaOn-siteFull TimeMedium
Full TimeMedium
Job Summary
Build efficient, scalable data pipelines using PySpark and Python to integrate multi-source systems into unified targets. Orchestrate workflows with Apache Airflow to ensure reliable execution, while enhancing existing infrastructure for improved performance and maintainability. Troubleshoot pipeline failures through root cause analysis and implement corrective measures promptly. Collaborate with stakeholders across teams to address evolving business needs and optimize data processing operations. Support continuous operations by working weekends and shifts as required.
Required Qualifications
- 3-5 years of experience as a data engineer
- Solid understanding of data engineering principles
- Proficiency in SQL
- Proficiency in Python
- Proficiency in PySpark
- Knowledge of data warehousing techniques
- Expertise in dimensional modeling
- Experience with Apache Airflow
- Familiarity with Docker
- Familiarity with version control systems
- Strong proficiency in data modeling techniques
- Expertise in designing and implementing effective data structures
- Knowledge of DBT (Data Build Tool)
- Ability to collaborate effectively with various stakeholders
- Willingness to adapt to flexible working hours
- Ability to work on weekends and in shifts if necessary
Desired Qualifications
- AWS knowledge
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.