Manager - Data Engineering
On-siteHyderabad, Telangana, India
EXPIREDHyderabad, Telangana, IndiaOn-siteFull TimeSenior LevelMedium
Full TimeSenior LevelMedium
Job Summary
Lead and manage a team of Data Engineers, providing technical guidance and mentorship while driving project delivery. Design, develop, and optimize scalable data pipelines in an on-premise environment using Python, Apache Spark, SQL, and Airflow. Build robust ETL/ELT workflows, manage complex DAGs for orchestration, and troubleshoot pipeline failures or performance issues. Conduct code reviews, monitor team performance, and collaborate with stakeholders to understand requirements and deliver solutions. Requires 8+ years of experience in data engineering with strong leadership skills.
Required Qualifications
- 8+ years of experience in building and managing data engineering solutions in an on-premise environment
- 8+ years of overall experience in Data Engineering
- Strong hands-on expertise in Python
- Strong hands-on experience with Python
- Strong experience with Apache Spark / PySpark
- Advanced SQL skills, including complex joins, CTEs, window functions, subqueries, and query optimization
- Strong experience with Apache Airflow for workflow orchestration and scheduling
- Experience working with on-premise data environments
- Strong understanding of ETL/ELT processes and data pipeline architecture
- Experience handling large volumes of data and optimizing data processing workloads
- Good understanding of data quality, monitoring, troubleshooting, and performance optimization
- Proven experience managing or leading Data Engineering teams
- Strong stakeholder and client management skills
- Ability to provide technical direction while managing project delivery
- Experience with resource planning, task allocation, mentoring, and performance management
- Strong communication and problem-solving skills
- Ability to work in a fast-paced, Agile environment
Desired Qualifications
- Experience in large-scale enterprise data platforms
- Experience with data warehouse concepts and data modelling
- Experience with on-premise Hadoop/data ecosystems
- Experience in migrating or modernizing legacy/on-premise data platforms
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.