RQ09222 - Software Developer - ETL - Senior
On-siteToronto, Ontario, Canada
Toronto, Ontario, CanadaOn-siteContractSenior LevelSmall
ContractSenior LevelSmall
Job Summary
Analyze existing data in legacy applications to design migration strategies for the new Dynamics 365 OPGT solution and data warehouse/lakehouse. Map, transform, and extract data elements while cleansing and validating records to ensure accuracy and consistency. Execute the migration plan using a delta-load approach, conduct thorough testing, and perform data reconciliation to verify integrity. Document mapping rules, transformation logic, and migration scripts, then provide post-migration support and optimize data management processes in the new environment. Expand the data lakehouse implementation, build a gold layer for reporting, and develop transformation pipelines. Complete work within assigned deadlines and submit deliverables for approval while complying with government security and architecture standards.
Required Qualifications
- 5+ years of proven working experience in an ETL role
- strong understanding of ETL principles, including data extraction, transformation, and loading processes
- knowledge of common ETL design patterns
- Understanding of data pipeline architectures, Azure workflow orchestration tools, and concepts related to data ingestion, transformation, and movement
- Proficiency in Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers, and monitoring
- Knowledge of integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs
- Must be able to lift 50 lbs
- Must be available for weekend shifts
- Valid driver's license
Desired Qualifications
- Azure cloud certifications (e.g. Azure fundamentals, Azure Data Engineer associate, Azure Database Administrator associate)
- Experience with PowerApps platform, Power Automate, Dynamics CE and F&O
- 5+ years of proven working experience in an ETL role
- strong understanding of ETL principles, including data extraction, transformation, and loading processes
- knowledge of common ETL design patterns
- Understanding of data pipeline architectures, Azure workflow orchestration tools, and concepts related to data ingestion, transformation, and movement
- Experience in integrating various data sources and systems, both on-premises and in the cloud, using Azure ETL services or other ETL tools
- Knowledge of integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs
- Expertise in data transformation techniques, such as data cleansing, aggregation, enrichment, and normalization using Azure cloud technologies
- Understanding of data quality management practices, including data profiling, data validation, and error handling within ETL processes
- Understanding of data governance principles, data privacy regulations and experience working with high-sensitivity data, and knowledge of best practices for data security and compliance in Azure
- Ability to monitor and troubleshoot ETL processes, optimize query performance, and implement efficient data processing techniques in Azure
- Proficiency in Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers, and monitoring
- Strong SQL skills and experience working with Azure SQL Databases and Dataverse; good understanding of Azure storage concepts and technologies
- Proficiency in scripting languages like Python, and experience with Azure-specific scripting using PowerShell or Azure CLI
- Expert in data manipulation languages (T-SQL, PL/SQL), data definition languages, physical database design, data modelling, query performance analysis & tuning
- Familiarity with version control systems (e.g., Azure Repos) and collaboration tools (e.g., Azure DevOps) for managing code, tracking changes, and collaborating with team members
- Experience with continuous integration/continuous deployment (CI/CD) processes around DevOps, data workflows, Synapse workspaces
- Experience with SQL Server Management Studio, Azure data management tools, XRM toolbox, data modeling tools (preferably, ERWIN)
- Demonstrated experience with ETL development, data pipelines, workflow orchestration and data ingestion, transformation, and movement
- Demonstrated experience in integrating various data sources and systems, both on-premises and in the cloud, using Azure ETL services or other ETL tools
- Demonstrated experience working with Azure Data Factory, Azure Synapse workspaces, PolyBase, including knowledge of pipeline creation, data flows, integration runtimes, triggers, and monitoring
- Demonstrated experience with data manipulation languages (T-SQL, PL/SQL), data definition languages, query performance analysis & tuning
- Demonstrated experience with SQL Server, Oracle, Azure SQL Databases
- Demonstrated experience with data modeling tools (preferably, ERWIN)
- Experience with software development lifecycle
- Experience with data modeling, physical database design, data flow diagrams
- Experience with Azure Data Factory (ADF) and Synapse Workspaces
- Demonstrated experience Azure data management tools, DevOps, Synapse Studio
- Experience in Azure resource configuration and administration such as Azure Data Lake, Blob Storage, Key Vault, Application Insight resources, resource groups and subscriptions
- Familiar with Azure cloud platform
- Azure cloud certifications
- Demonstrated experience working with integration technologies commonly used with Dynamics, such as DataVerse / Common Data Service (CDS), Data Entities, and APIs
- Demonstrated experience with PowerApps platform, Power Automate, Dynamics CE &F&O
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.