Data Solutions Engineer - HYBRID/NC RESIDENTS ONLY
On-siteDurham, North Carolina, United States
Job Summary
Design, implement, and support end-to-end ELT pipelines in Databricks and Azure Data Factory to ingest, transform, and publish high-quality data for analytics. Enforce data quality using Delta Live Tables expectations, unit tests, and alerting runbooks while managing schema drift, CDC patterns, and partitioning strategies. Build curated Delta tables aligned with dimensional modeling for BI consumption and partner with the Data Architect to ensure governance, security, and cost efficiency. Operate jobs with monitoring, logging, and on-call rotation as needed. Document pipelines, contracts, and SLAs to continuously improve performance and reliability.
Required Qualifications
- 2+ years of hands‐on data engineering (or comparable software engineering with significant data work)
- 2+ years building pipelines on Azure and Databricks (or equivalent cloud + Spark)
- Strong SQL (analytical queries, window functions), PySpark/Spark SQL, and data modeling fundamentals
- Bachelor's degree in MIS, Computer Science, Engineering, or equivalent experience
- Proficiency with SQL and Python (PySpark), including performance tuning on large datasets
- Experience with Azure Databricks, Delta Lake, Delta Live Tables (DLT), Azure Data Factory (or Fabric Data Pipelines), ADLS Gen2, and Azure DevOps/Git for CI/CD
- Working knowledge of Unity Catalog and/or Microsoft Purview for governance, lineage, and security
- Familiarity with data ingestion patterns (files, APIs, JDBC), schema evolution, CDC, and deletion detection patterns
- Understanding of dimensional modeling to produce analytics‐ready datasets for Power BI
- Exposure to orchestration/monitoring, cost optimization, alerting, and runbook‐driven operations
- Data pipeline design (batch & streaming), DLT expectations for data quality, and robust error handling
- Source control, branching strategies, and CI/CD for data assets (notebooks, jobs, workflows)
- Practical understanding of privacy, security, and RBAC in cloud data platforms
- Excellent communication, documentation, and cross‐functional collaboration skills
- Analytical mindset; bias toward automation and measurable reliability
- Applicants must be legally authorized to work in the United States and should not require now, or in the future, sponsorship for employment visa sponsorship
- Occasional,travel may be required for training or meetings
- Must be able to work flexible hours to accommodate different time zones (Central/Eastern/Mountain/Pacific), if necessary
- Be flexible for after-hours tasks, as needed
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.