Senior Data Engineer
$1,000,000–$1,200,000 year
HybridNew Delhi, Delhi, India
Job Summary
Develop scalable data infrastructure using Prefect and Airflow to automate mining, cleaning, and transformation for AI readiness. Create and oversee APIs that collect, manage, and analyze data from diverse public sources while maintaining existing open data platforms like Open Budgets India and Justice Hub. Lead strategic planning for data engineering projects, mentor a team through code reviews, and ensure compliance via structured checklists for licensing and PII. Document all algorithms, workflows, and methodologies rigorously. Work with data strategists and public policy researchers to co-create analytics tools, requiring public-sector experience and familiarity with FOSS communities.
Required Qualifications
- 5+ Years of experience as a Data engineer
- Proficiency in Python, SQL, and Excel
- Experience building dashboards (Superset, Tableau, or Python frameworks)
- Excellent written/verbal communication and documentation skills
- Public-sector experience: Demonstrated work with government departments/PMUs or allied institutions; comfortable with consultations and public-sector processes
- Onboarding literacy: Familiarity with dataset discovery, metadata, licensing/ownership, and basic PII/privacy checks; rigorous documentation and follow-through
- Operations & collaboration: Excellent coordination and communication skills; readiness for periodic intra-state travel
- Extensive working knowledge of API or Stream-based data extraction processes
- Ability to gauge requirements for the project, make decisions and coordinate with other team members
- Prior experience in actively working and contributing to FOSS (Free and Open Source Software) communities
- Familiarity with working with Agile methodologies and Scrum processes
- Experience with scalable infrastructure practices such as microservice architecture, infrastructure as code, distributed systems, scaling methods, load balancing and more
Desired Qualifications
- Any graduate, however, candidates with a B.Sc./M.Sc. (or higher) in Computer Science, Data Science, or a related field will be preferred
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.