Sr Data Engineer BI
HybridBloomington, Minnesota, United States
Job Summary
Design, build, and optimize scalable data pipelines using Azure Data Factory, Fabric Data Pipelines, and Python/PySpark to modernize legacy SAP Data Services workloads. Lead the end-to-end architecture for BI domains, including medallion patterns on Microsoft Fabric/OneLake, while integrating data from SAP S/4HANA, HighJump WMS, Anaplan, and eCommerce systems. Develop and operationalize machine learning models and agentic workflows for forecasting and anomaly detection, enforcing data governance standards and mentoring engineers. Provide expert-level production support for dataset refreshes and BI platform stability, collaborating with SAP, WMS, and security teams to align on architectural direction.
Required Qualifications
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related field.
- 8+ years of progressive experience in data engineering, business intelligence, or analytics platform development.
- Expert-level SQL (T-SQL, PL/SQL): complex queries, stored procedures, window functions, partitioning, performance tuning.
- 5+ years designing and building ETL/ELT pipelines with tools such as Azure Data Factory, SAP Data Services, Fabric Data Pipelines, or equivalent.
- Strong understanding of dimensional modeling (star/snowflake schemas, slowly-changing dimensions) and modern lakehouse / medallion architecture patterns.
- Proven experience integrating BI/data platforms with enterprise COTS systems - SAP (ECC, S/4HANA, or BW), HighJump WMS, Anaplan, or equivalent.
- Experience with Power BI semantic modeling and DAX.
- Demonstrated experience leading technical design, mentoring engineers, and driving best practices.
- Excellent communication skills with the ability to translate complex technical concepts for both technical and non-technical stakeholders.
- Ability to perform work on a phone and computer extensively.
Desired Qualifications
- Degree in Data Science, Analytics, Computer Science, or related field.
- 3+ years programming experience in Python and/or PySpark for data engineering and ML workloads.
- Hands-on experience building and deploying machine learning models (scikit-learn, TensorFlow, PyTorch, Azure ML, or Fabric Data Science).
- Experience designing and automating AI agents and agentic workflows using frameworks such as Microsoft Copilot Studio, Azure AI Foundry, or equivalent.
- Familiarity with LLM integration patterns (RAG, function/tool calling, prompt engineering) and vector databases.
- Experience with MLOps practices and tooling.
- Experience with ETL tools (CDS Views, BDC, DataSphere, Theobald Xtract Universal, or dab Nexus).
- Experience with streaming/real-time data (Kafka, Event Hubs, Fabric Real-Time Intelligence, or Spark Structured Streaming).
- Experience with DataOps / CI-CD for data.
- Familiarity with Microsoft Purview or other data governance/cataloging platforms.
- Working knowledge of AWS/Fabric for cross-cloud integration.
- Experience with Agile/Scrum methodologies.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.