Python PySpark Developer
$2,500,000–$2,500,000 year
On-siteBengaluru, Karnataka, India
Bengaluru, Karnataka, IndiaOn-siteFull Time$2,500,000–$2,500,000 yearMasters DegreeSmall
Full TimeMasters DegreeSmall
Job Summary
Design and develop robust PySpark applications for large-scale data processing, building and optimizing ingestion, transformation, and storage processes. Implement efficient algorithms for distributed computing and write optimized SQL queries for data extraction. Collaborate with cross-functional teams to integrate data-driven solutions while troubleshooting performance bottlenecks to ensure pipeline reliability. Requires 3-10 years of Python development experience with Apache Spark, strong SQL skills, and familiarity with cloud platforms and containerization technologies.
Required Qualifications
- Bachelor's/Master's degree in Computer Science, Engineering
- Proven experience (3-10 years) in Python development with a focus on PySpark
- Strong understanding of distributed computing principles and experience with Apache Spark
- Proficiency in SQL and experience with relational databases (MySQL, PostgreSQL, etc.)
- Experience with data serialization formats such as JSON, Parquet, Avro
- Excellent problem-solving skills and ability to work independently or as part of a team
- Good communication skills with the ability to effectively collaborate with stakeholders
Desired Qualifications
- Familiarity with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes) is a plus
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.