Lead Data Engineer (Python, AWS, Spark, Kafka, SQL, Snowflake, Databricks, GenAI)
$197,300–$225,100 year
On-siteNew York City, New York, United States or Richmond, Virginia, United States
New York City, New York, United States or Richmond, Virginia, United StatesOn-siteFull Time$197,300–$225,100 yearSenior LevelBachelors DegreeEnterprise
Full TimeSenior LevelBachelors DegreeEnterprise
Job Summary
Design, develop, test, and support scalable data pipelines and cloud-based solutions using Python, SQL, Snowflake, Databricks, and Kafka. Collaborate with Agile teams and product managers to deliver robust systems that drive personalized messaging experiences for millions of customers. Optimize information systems for downstream application consumers through rigorous code reviews, unit testing, and sound data design practices. Mentor engineering community members and experiment with emerging technologies to accelerate delivery.
Required Qualifications
- Bachelor's Degree
- At least 4 years of experience in application development
- At least 2 years of experience in big data technologies
- At least 1 year experience with cloud computing (AWS, Microsoft Azure, Google Cloud)
Desired Qualifications
- Master's Degree
- 7+ years of experience in application development including Python, SQL, Spark, ETL tools, or AWS Glue
- 4+ years of experience with a public cloud (AWS, Microsoft Azure, Google Cloud)
- 4+ years experience with Distributed data/computing tools (MapReduce, Hadoop, Hive, EMR, Kafka, Spark, or MySQL)
- 4+ year experience working on real-time data and streaming applications
- 4+ years of experience with NoSQL implementation (Mongo, Cassandra)
- 4+ years of data warehousing experience (Redshift or Snowflake)
- 4+ years of experience with UNIX/Linux including basic commands and shell scripting
- 4+ years of experience with data modeling for data warehousing
- 2+ years of experience with Agile engineering practices
- Experience leveraging interactive AI tooling (Claude Code, GitHub Copilot) to accelerate software delivery
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.