Senior Engineer_Data Engineer_7
$140,000–$150,000 year
On-siteCalifornia, United States
Job Summary
Write efficient, optimized SQL queries to extract, clean, transform, and analyze large datasets. Automate repetitive data workflows and reporting processes to improve efficiency. Build ingestion scripts, transformation logic, and end-to-end data pipelines on Google Cloud Platform. Execute migration tasks from Teradata to BigQuery, including support for Storage Transfer Service and bq CLI. Implement pipeline monitoring, logging, and alerting to ensure reliability. Support testing, validation, and reconciliation of migrated datasets to ensure data integrity. Document methodologies, code, and findings clearly for reproducibility. Collaborate with data analysts, architects, and business stakeholders to translate requirements into robust, production-ready solutions.
Required Qualifications
- Strong SQL and Python skills
- Basic-to-intermediate experience with GCP services, including BigQuery, GCS, Composer, and Cloud Functions
- Solid understanding of ETL/ELT data flows and pipeline design
- Experience working with APIs, file-based ingestion, and orchestration tools
- Ability to test, validate, and debug data pipelines effectively
- Hands-on exposure to migration tools such as Storage Transfer Service, gsutil, and bq CLI
- Understanding of Teradata-to-GCP mapping concepts (e.g., Teradata → GCS, ADF → Composer)
- Familiarity with CI/CD concepts and practices
- Exposure to version control systems such as Git for code management and collaboration
- Understanding of data security, access controls, and governance best practices in cloud environments
- Familiarity with data quality frameworks and reconciliation techniques
- Good communication and problem-solving skills, with the ability to work independently and in a team setting
Desired Qualifications
- Working knowledge of Spark via Dataproc
- Prior experience with large-scale data migration projects from on-premise or legacy systems to cloud platforms
- Exposure to data cataloging or metadata management tools
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.