Lead Data Platform Engineer
HybridLondon, England, United Kingdom
Job Summary
Design scalable, secure cloud platform architectures for multi-region operations and NHS data pipelines, defining integration patterns and component specifications for the engineering team. Own day-to-day delivery management of platform and data engineering workstreams, including technical workplanning, capacity management, and QA coordination. Mentor data engineers on architectural thinking and best practices while acting as the technical reference point for system design decisions. Contribute technical expertise to vendor partnerships and commercial proposals, ensuring all platform designs align with regulatory frameworks like NHS IG and ISO 27001.
Required Qualifications
- Around 4 years of experience in data or platform engineering, preferably including senior or lead responsibility for a production data platform
- Deep expertise in cloud platform design and secure data architecture, including identity and access management, network design and access control; Databricks and OCI, or comparable platforms
- Strong Python and SQL, and experience building and monitoring production pipelines using PySpark or Spark
- Experience with infrastructure as code, CI/CD and a Git and GitHub workflow
- A track record of setting engineering standards and design patterns, conducting architectural review, and producing design documentation that others build from
- Experience owning delivery of technical workstreams: technical workplanning, capacity management, QA and early escalation of risk
- Experience mentoring or managing engineers, including objective setting and development
- Experience designing to regulated data requirements such as NHS IG, ISO 27001 and GDPR, or comparable frameworks
- A clear communicator who can explain design decisions and trade-offs to technical and non-technical audiences
- Comfortable acting as the technical authority for platform and infrastructure decisions, and as the reference point the engineering team comes to
- Confident in commercial and partner-facing technical discussions, including vendor conversations and the technical sections of proposals
Desired Qualifications
- Experience with NHS or other routine healthcare data at patient level
- Unity Catalog, Delta Lake and medallion structure, bronze to silver to gold
- Exposure to dbt and dimensional modelling
- Experience contributing to vendor or partner relationships, such as with Databricks
- Experience designing for multi-region operations (UK, Middle East, EU)
- Flexible working
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.