Senior Cloud Data Platform Engineer
$130,000–$150,000 year
HybridToronto, Ontario, Canada
Job Summary
Design, build, and operate high-throughput, low-latency Kafka streaming pipelines for market data, transactional events, and operational telemetry. Architect Change Data Capture (CDC) solutions using Debezium and Kafka Connect to enable real-time synchronization, while building, tuning, and maintaining ClickHouse OLAP clusters for schema design and query optimization. Create optimal data pipeline architectures across streaming and batch workloads, identifying and implementing internal process improvements to automate manual tasks and enhance scalability. Monitor Kafka cluster health, consumer lag, and schema governance, then work with stakeholders to resolve data infrastructure issues and explore AI integrations into the development cycle.
Required Qualifications
- Minimum 3+ years of data engineering experience
- At least 2 years focused on streaming and event-driven architectures
- Strong expertise in Apache Kafka
- Hands-on experience with Change Data Capture (CDC) patterns and tooling
- Experience with ClickHouse
- Proficiency in SQL
- Proficiency in Python
- Experience with any of the major cloud platforms
- Strong knowledge of message broker and streaming systems
- Experience with CI/CD pipeline tooling and development practices
- Experience building Terraform scripts for infrastructure-as-code
- Good knowledge of Airflow operators and DAG configuration
- Good knowledge of popular data standards and formats
- Data modeling skills with experience in both streaming and batch paradigms
Desired Qualifications
- GCP data platform knowledge
- Experience with stream processing frameworks (Apache Flink, Spark Structured Streaming, or GCP Dataflow)
- Experience with OLAP databases beyond ClickHouse
- Experience with ML Model Training using Google Vertex AI
- Experience in the financial services industry
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.