Data Operations Engineer
HybridAuckland, Auckland, New Zealand
Auckland, Auckland, New ZealandHybridFull TimeMid LevelLarge
Full TimeMid LevelLarge
Job Summary
Monitor ETL/ELT pipeline health and performance across Azure SQL Data Warehouse and Microsoft Fabric environments. Investigate and resolve pipeline failures, data-quality issues, and performance bottlenecks to ensure reliable data delivery. Develop monitoring frameworks, documentation, and validation checks to maintain data accuracy and improve visibility into data flows. Collaborate with analysts, engineers, and IT teams to support Power BI reporting and AI-powered automation platforms. This role focuses on operational excellence within a modernising data infrastructure transition.
Required Qualifications
- Strong SQL skills with the ability to write, troubleshoot, and optimise complex queries
- Experience operating, monitoring, and troubleshooting ETL/ELT data pipelines
- Experience with Microsoft Fabric Data Factory and/or Azure Data Factory
- Working knowledge of PySpark for data processing and transformation
- Understanding of Microsoft Fabric ecosystem, including OneLake, Data Engineering, and Lakehouse concepts
- Knowledge of modern Data Lakehouse and Medallion architecture principles
- Experience with monitoring, logging, and alerting approaches for data pipelines
- Strong documentation skills with the ability to make complex systems easy to understand
- Understanding of how data feeds downstream Power BI datasets and reports
- Strong analytical and troubleshooting skills with the ability to manage incidents calmly and effectively
- Excellent communication skills, with the ability to explain technical issues to both technical and non-technical stakeholders
Desired Qualifications
- Experience building AI agents, copilots, or automation solutions using Microsoft Copilot Studio, Azure AI Foundry, Azure OpenAI, or Power Platform
- Experience implementing Retrieval-Augmented Generation (RAG) solutions using enterprise data sources
- Knowledge of AI governance, responsible AI practices, and model evaluation
- Exposure to DataOps or DevOps practices, including CI/CD, version control, and deployment pipelines
- Experience with data cataloguing and lineage tools
- Previous experience supporting data platforms within a regulated or healthcare environment
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.