Senior Data Engineer - NBA
$117,600–$161,700 year
HybridNashville, Tennessee, United States
Job Summary
Design and maintain Bronze, Silver, and Gold lakehouse pipelines on Databricks that transform raw healthcare, member, and behavioral data into trusted assets for decision intelligence and machine learning. Develop reusable feature tables and real-time ingestion patterns using Spark, Delta Lake, and event-driven architectures to support model training and scoring. Implement data quality validation, monitoring, and alerting controls while optimizing Spark workloads for performance and cost efficiency. Partner with Data Science, AI Engineering, and Platform teams to ensure high-quality data availability across the NBA ecosystem. Diagnose and resolve production incidents to maintain reliable platform operations.
Required Qualifications
- 5+ years of data engineering experience building and operating production data platforms
- Strong SQL and Python skills with hands-on experience developing Spark-based data pipelines
- Experience with Databricks, Delta Lake, or comparable lakehouse platforms
- Experience implementing Medallion Architecture (Bronze/Silver/Gold) data patterns
- Experience building batch and streaming data processing pipelines
- Strong understanding of data modeling, pipeline testing, data quality controls, and operational support practices
- Familiarity with modern cloud-based data platforms and distributed data processing systems
- Strong communication skills and the ability to collaborate across engineering, analytics, and business teams
- Occasional travel to Humana's offices for training or meetings may be required
- Download speed of 25 Mbps and an upload speed of 10 Mbps is required
- Work from a dedicated space lacking ongoing interruptions to protect member PHI / HIPAA information
Desired Qualifications
- Experience with Databricks Feature Store, Unity Catalog, Delta Live Tables, and Databricks Workflows
- Experience with Spark Structured Streaming and real-time feature engineering
- Experience with Kafka and event-driven data architectures
- Familiarity with machine learning, recommendation systems, reinforcement learning, or decision intelligence platforms
- Experience with Azure Data Factory, Azure Data Lake Storage, Azure Event Hubs, or similar cloud-native data services
- Experience with data observability and quality platforms such as Great Expectations, Monte Carlo, or equivalent tools
- Experience integrating external healthcare, claims, CMS, CDC, consumer, or social determinants of health datasets
- Background in healthcare, insurance, or another regulated industry with PHI/HIPAA handling requirements
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.