Principal Data Engineer
HybridCity of London, England, United Kingdom
Job Summary
Own the technical architecture of IG's GCP data platform end-to-end, managing Core Lake, batch and real-time ingestion, and transformation within the Medallion architecture. Lead delivery of complex migration work from legacy AWS and on-premises systems onto GCP-native capabilities while defining platform contracts and self-service patterns. Drive data quality as a firm-wide concern by establishing contracts, observability, SLA/SLO monitoring, and automated remediation across all layers. Act as the senior technical point of contact for stakeholders across Compliance, Marketing, Finance, and global divisions, translating requirements into well-scoped engineering deliverables. Set and uphold engineering standards through code reviews, technical mentoring, and reusable frameworks for squads in London, India, and Poland. Provide technical direction and quality assurance for third-party staff augmentation, ensuring all assets meet IG's standards and leave behind maintainable, compliant code.
Required Qualifications
- Hands-on experience designing and operating large-scale data pipelines on GCP (BigQuery, Cloud Composer/Airflow, GCS)
- Proven knowledge of Apache Kafka and real-time streaming architectures
- Demonstrated experience leading or technically mentoring a team of data engineers
- Ability to set standards, conduct code and design reviews, and grow engineers' capabilities
- Strong grasp of data quality practices: data contracts, pipeline observability, SLA/SLO definition, and automated alerting and remediation
- Solid understanding of SQL transformation patterns and modern tooling such as dbt
- Experience managing ingestion estates with third-party connectors and in-house integrations
- Clear and confident communicator able to work cross-functionally with non-technical stakeholders
- Comfortable in a regulated financial services environment with structured delivery disciplines (sprint cadences, change control, escalation)
- Hybrid working model with 3 days in the office
Desired Qualifications
- Curious and forward-thinking mindset
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.