Data Engineer III
RemoteIndia
IndiaRemoteFull TimeSenior LevelSmall
Full TimeSenior LevelSmall
Job Summary
Build, test, and optimize data pipelines for real-time and batch processing using SQL, PySpark, and Python. Perform source-to-target data analysis and mappings while collaborating with admins, developers, and analysts to drive delivery cycles. Support the evolution of the Enterprise Data Platform architecture and ensure data quality and integrity across DWH and AWS platforms. Collaborate with leadership and stakeholders to mitigate risks and formulate action plans. Follow agile development methodologies and DevOps practices to deliver solutions and product features.
Required Qualifications
- 6+ years of overall work experience
- 4+ years exclusively in delivering data solutions
- 3+ years of proven experience building Cloud BI solutions using AWS or Azure
- Experience with agile development methodologies
- Experience following DevOps, Data Ops and Dev Sec Ops practices
- 4+ years of programming in SQL
- 4+ years of programming in Pyspark
- 4+ years of programming in Python
- Excellent written, verbal and interpersonal and partner communication skills
- Excellent Analysis and business requirement documentation skills
- Ability to work with multi-functional teams from multiple regions/ time zones
- Excellent prioritization and problem-solving skills
- Knowledge of Large Language models
- Proven experience in developing LLM applications
- Hands on experience with Snowflake or Azure data engineering
- Knowledge of SQL and NoSQL databases like PostgreSQL, MySQL, MongoDB Cassandra
- Experience in building data pipelines in Databricks
- Knowledge of data governance practices, data quality, and data security
- Relevant certifications in data engineering on cloud platforms
- Basic understanding of machine learning and Generative AI concepts
Desired Qualifications
- Knowledge of GPT
- Knowledge of Retrieval augmented Generation
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.