Sr. Data Engineer
On-siteAustin, Texas, United States
EXPIREDAustin, Texas, United StatesOn-siteFull TimeSenior LevelMedium
Full TimeSenior LevelMedium
Job Summary
Design and maintain streaming data pipelines using Databricks and Spark, creating ML pipelines to feed data and update models. Develop predictive analytics models to support decision-making processes across departments. Take ownership of operational excellence in your data domain, focusing on automation, testing, CI/CD, and documentation. Work independently while collaborating on ambiguous requirements in a rapidly changing environment. This role sits within a small Data Platform team architecting enterprise-grade systems for Overhaul's AI-driven logistics solutions.
Required Qualifications
- Strong product ownership
- Good intuition on how data is used to drive business decisions
- Mastery over Python programming language
- Mastery over SQL
- Strong background in at least one of the following: distributed data processing or software engineering of data services
- Familiarity with big data technologies like Spark
- Comfort working with web-scale datasets
- Passion for the end-to-end software development lifecycle
- Focus on automation, testing, CI/CD, and documentation
- Eye for detail
- Good data intuition
- Passion for data quality
- Ability to work independently
- Collaboration and giving/receiving candid feedback
- Comfort working in a rapidly changing environment with ambiguous requirements
- Nimbleness
- Taking intelligent risks
- Experience with Cloud based Data Lake Tools
- Experience with designing and implementing data pipelines
- Python and SparkSQL/Spark experience
- BS in Computer Science or Related field
Desired Qualifications
- Azure Data Tools, such as Data Lake, Synapse, or Fabric
- Power BI Experience
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.