Senior Data Engineer
On-siteJersey City, New Jersey, United States
Job Summary
Create data transformation logics in Azure Databricks and build reusable ETL transformations using Python, Apache Spark, and PySpark. Write complex SQL, Transact SQL, and stored procedures while developing and maintaining data models, warehouses, and data marts in Snowflake. Streamline ETL complexity and tune performance through automation. Collaborate with data scientists, analysts, and stakeholders to gather requirements and define models that support business needs. Demonstrate analytical problem-solving abilities within an Agile framework involving daily scrums, sprint planning, and backlog refinement. Manage client discussions effectively while working independently in an individual contributor role.
Required Qualifications
- Hands-on experience with Azure Services: Databricks, Data Factory and DevOps
- Proficient in Python, Apache Spark, and PySpark
- Write advance and complex SQL queries with solid experience in database management
- Develop and maintain data models, data warehouses, and data marts in Snowflake
- Automation, build re-usable transformations, performance tuning
- Streamline ETL complexity- logics and transformations
- Collaborate with data scientists, analysts, and stakeholders to gather requirements and define data models that effectively support business requirements
- Demonstrate decision-making, analytical and problem-solving abilities
- Strong verbal and written communication skills to manage client discussions
- Familiar with working on Agile methodologies - daily scrum, sprint planning, backlog refinement
- Azure Databricks ability to create data transformation logics
- Strong programming skills in Python and experience with SQL
- Should be able to write complex SQL, Transact SQL, Stored Procedures
- ETL tool Azure data factory, Data Bricks
- Experience with Data Modelling
- DWH - Snowflake
- Excellent communication skills and stakeholder management
- Ability to work independently in IC role
Desired Qualifications
- Knowledge on Snowflake platform
- Knowledge on PowerBI
- Familiarity with CI/CD practices and version control (e.g., Git)
- Familiarity with Azure DevOps
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.