Senior IT Data Engineer - R&D IT
On-site · Bengaluru, Karnataka, India
Job Summary
Senior IT Data Engineer responsible for shaping and maintaining the R&D data analytics platform. Design, build, and maintain ETL pipelines ingesting and transforming data into a cloud-based R&D data lake; develop reusable data models and libraries; implement data quality through validation and anomaly detection; ensure security and compliance with RBAC, lineage, auditing, and regulatory standards; optimize pipelines for performance and cost; automate deployments using CI/CD and Infrastructure as Code; monitor and troubleshoot production pipelines; collaborate with data scientists, project/resource managers, IT teams, and analysts to deliver reliable, model-ready datasets and support data-driven decisions.
Required Qualifications
- Master’s degree or equivalent practical experience in Data Engineering, Software Engineering, Computer Science, or related field
- 7+ years of professional data engineering experience
- Hands-on experience with AWS native data lake services (AWS Glue, S3, Athena, Lake Formation)
- Databricks experience for ETL pipelines and migrations to Databricks platform
- Delta Lake design, optimization, and schema evolution
- Cloud-native engineering (AWS/Azure), IaC, CI/CD, DevOps/MLOps best practices
- Programming proficiency in Python and SQL
- Data quality, observability, lineage, metadata management, secure data access patterns
- Experience with performance optimization in large data lake environments
- Strong communication and collaboration across data science, architecture, and wider teams
- Agile and Scrum methodologies
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.