Manager, Data Storage Platforms
On-siteBengaluru, Karnataka, India
Job Summary
Lead a team of platform engineers overseeing the planning, build, and operations of large-scale, cloud-native data platforms on AWS and Azure. Architect and optimize event-driven data architectures, CDC frameworks, and scalable ETL/ELT processes while managing datasets using Apache Hudi and Terraform. Collaborate with cross-functional teams to deliver data platform initiatives, ensure alignment with industry best practices for governance and security, and monitor pipeline performance against SLAs. Evaluate new technologies to improve engineering outcomes and assist business partners in identifying cost-effective data solutions. Partner with Domain Data Product Delivery teams to support global data transformation initiatives within a fail-fast-learn-fast culture.
Required Qualifications
- Bachelor's degree or higher in computer science or relevant subject area
- Minimum 5+ years of progressive technical experience in one or more of the cloud-based data warehouse/data lake platforms such as AWS S3, Snowflake, Databricks, Microsoft Synapse
- Significant experience in leading teams in Data & Analytics programs with experience in native digital technologies (cloud, big data and predictive) in organizations driving product development
- Change Agent – Catalyst for the teams and expected to lead the transformation to a fail-fast-learn-fast culture enabling continual improvement
- Servant Leader – Able to gain respect from their team and willing to get their hands dirty to get the job done
- Courageous - Able to effectively challenge and influence peers and teams, leading them through difficult or uncertain situations
- Significant experience working in a global matrix environment
- Must be a strategic, innovative problem solver, and have strong influencing and communication skills
- Strong knowledge of business process flow and/or entity relationship modeling
- Demonstrated outstanding written and verbal communication skills
- Ability to work effectively with other technology teams as well as business partners
- Ability to effectively present information, interact with, and respond to questions from managers, employees, customers, and vendors
- Experience in applying best practices to ELT/ETL data transformation from variety of data sources to AWS and other data warehouse platforms like Snowflake, Databricks, Microsoft Synapse etc
- AWS CDK (TypeScript/Python)
- AWS Glue (ETL, Catalog, Crawlers, Job Bookmarks)
- AWS Lambda, API Gateway, SQS, SNS, Step Functions, EventBridge
- Amazon Kinesis, Route 53, S3 (events, triggers)
- Terraform (0.12+)
- Apache Hudi (0.11+) and CDC methodologies
- AWS DMS
- Apache Spark (2.4+) and PySpark
- Amazon EMR (5.x+)
- Python (3.6+) for automation and pipeline development
- IBM DataStage (for supplemental ETL workflows)
- Snowflake, Databricks, Microsoft Synapse
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.