Data Engineer-GCP
On-siteSydney, New South Wales, Australia
Sydney, New South Wales, AustraliaOn-siteContractSmall
ContractSmall
Job Summary
Design scalable data solutions using native GCP tools including BigQuery, Dataflow, Pub/Sub, and Cloud Storage. Build and maintain batch and streaming ETL/ELT pipelines with real-time processing capabilities. Develop data processing frameworks using Python, SQL, Apache Beam, or Spark while optimizing star and snowflake schema data warehouses. Manage containerized workloads via Docker and Kubernetes, and enforce data governance, lineage, and security compliance within GCP environments.
Required Qualifications
- Strong GCP Data Services Expertise - Hands-on experience with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Ability to design scalable data solutions using native GCP tools
- Experience in Data Pipeline Development (Batch & Streaming) - Building and maintaining ETL/ELT pipelines, Experience with real-time streaming and batch processing
- Strong Programming & Query Skills - Proficiency in Python and SQL, Experience with data processing frameworks like Apache Beam or Spark
- Understanding of Data Modeling & Warehousing - Strong understanding of data modeling (star/snowflake schemas), Experience designing and optimizing data warehouses (BigQuery)
Desired Qualifications
- Experience in Multi-Cloud Exposure : Understanding of cross-cloud data integration and migration, Experience working with AWS or Azure alongside GCP
- Knowledge of Streaming & Messaging Frameworks : Experience with Kafka or advanced Pub/Sub patterns, Knowledge of real-time data architectures
- Basic Understanding of Containerization & Orchestration : Hands-on with Docker and Kubernetes (GKE), Ability to deploy and manage data workloads in containerized environments
- Data Governance & Security : Familiarity with data lineage, cataloging, and governance tools, Understanding of IAM roles, data security, and compliance in GCP
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.