Sr. Data Engineer
On-siteBeaverton, Oregon, United States
Job Summary
Design, develop, and optimize scalable data pipelines and solutions using Databricks, PySpark, SQL, and Python to support analytics, machine learning, and business intelligence. Build reliable data models and ETL/ELT processes while establishing governance, security, and best practices across Azure and Databricks platforms. Evaluate new capabilities, recommend adoption strategies, and monitor pipeline performance for reliability and cost efficiency. Partner with cross-functional teams to solve complex data challenges and provide technical guidance on architecture. Mentor junior engineers and contribute to a culture of technical excellence. This role requires 5+ years of experience in data engineering with strong hands-on skills in distributed systems and modern table formats.
Required Qualifications
- 5+ years of experience in Data Engineering, Data Platform Engineering, or a related field
- Strong hands-on experience with Databricks, Apache Spark, PySpark, Python, SQL, Azure Data Lake Storage, and Delta Lake
- Deep understanding of distributed data processing and large-scale data architectures
- Strong knowledge of data engineering principles, including data modeling, ETL and ELT design patterns, Data quality and observability, Metadata management, Data lifecycle management, Master Data Management concepts
- Strong understanding of modern table formats and storage architecture
- Experience with performance optimization techniques
- Experience implementing CI/CD and infrastructure automation using tools
- Excellent problem-solving, analytical, and communication skills
- Ability to operate independently while leading initiatives in a fast-paced, agile environment
- Passion for building scalable, maintainable, and well-governed data platforms
Desired Qualifications
- Experience with Unity Catalog, Databricks Workflows, Delta Live Tables / Spark Declarative Pipelines, Structured Streaming, Kafka or Event Hubs
- Experience implementing data governance frameworks and cloud security best practices
- Experience with RBAC, ABAC, data classification, auditing, and compliance controls
- Experience supporting self-service analytics platforms and integrating Databricks with Power BI or other analytics tools
- Experience evaluating and adopting emerging cloud and data platform technologies
- Experience creating engineering standards, reference architectures, and operational best practices for enterprise data platforms
- Experience building and delivering internal training or enablement programs for technical platforms
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.