Data engineer (Azure)
$4,000–$4,500 year
On-siteSingapore, Singapore or Central, Louisiana, United States
Job Summary
Design, develop, and test PySpark scripts for data pipelines, including batch ingestion and orchestration within Azure Synapse. Establish connections to source systems such as on-prem databases, IoT devices, and APIs, then manage data in SQL servers, Hadoop, or HANA as required. Create PowerBI reports and dashboards with row-level security access controls, and perform data integration using table joins or other mechanisms. Deploy pipeline artifacts across environments via Azure DevOps, troubleshooting issues and maintaining ETL processes. Requires 2+ years of Azure data engineering experience with onsite hours from 8:30am to 6pm Monday through Friday.
Required Qualifications
- Bachelor's Degree in Computer Science or Engineering
- 2 years of experience in Azure Data engineering
- Python
- Pyspark
- Big Data development
- Sound Knowledge of Azure Synapse analytics
- Azure Synapse analytics for pipelines, orchestration, set up
- 1-2 experience in Visualization design and development with Power BI
- Knowledge on row-level security, access control
- Sound experience in SQL
- Datawarehouse
- data marts
- data ingestion with Pyspark and Python
- Expertise in developing and maintaining ETL processing pipelines in cloud-based platforms such as AWS, Azure, etc.
- Team player
- good interpersonal skills
- communication skills
- problem-solving skills
- 8:30am to 6pm (Monday to Friday) onsite
- no hybrid option
Desired Qualifications
- Devops expertise
- Azure Devops
- Azure Data Explorer
- Hadoop
- HANA
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.