Senior Data Engineer- Pyspark,Databricks,SQL
On-siteBengaluru, Karnataka, India
Bengaluru, Karnataka, IndiaOn-siteFull TimeSenior LevelEnterprise
Full TimeSenior LevelEnterprise
Job Summary
Write reusable, testable, and efficient code for data pipelines, ensuring performance, security, and scalability. Integrate data storage solutions including databases, key-value stores, and blob stores while building robust end-to-end testing and debugging workflows. Utilize PySpark, SQL, and Databricks to optimize stream processing and data engineering problems, with specific focus on Delta Live Tables. Leverage Git, JIRA, and Confluence for version control and project management. Maintain dashboarding exposure using Databricks and Tableau to support decision-making.
Required Qualifications
- 5yrs-7yrs relevant experience as a Data Engineer
- Expert in Pyspark
- SQL
- knowledge on Databricks
- cloud platforms
- Stream processing
- Mandatory experience working with delta live tables
- Familiarity with database technologies such as SQL and MYSQL
- Experience with Git
- Experience with JIRA
- Experience with Confluence
Desired Qualifications
- Good to have understanding about AI tools
- Dashboarding tool exposure : Databricks dashboarding , Tableau
- Strong unit test and debugging skills
- Responsible for writing reusable, testable and efficient code - ensuring the application's performance, security and scalability
- Integration of data storage solutions (may include databases, key-value stores, blob stores, etc.)
- building data pipelines
- Support end to end testing
- Optimization Skills
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.