Senior Data Engineer
RemoteUnited States
Job Summary
Design and maintain scalable, high-performance data pipelines using Python and AWS DynamoDB to ingest, process, and analyze vast volumes of transactional and operational data. Implement robust ETL/ELT processes for both batch and streaming workloads while optimizing DynamoDB data models and indexing strategies to meet diverse access patterns. Collaborate with data scientists and backend engineers to ensure data integrity, establish validation mechanisms for pipeline reliability, and advocate for best practices through code reviews and architectural discussions. Mentor junior team members and promote knowledge sharing across the engineering organization.
Required Qualifications
- 5+ years of professional experience in data engineering or software engineering roles focused on data-centric solutions
- Expert Python programming skills with a strong focus on data processing and pipeline development
- Deep experience designing and implementing solutions with AWS DynamoDB, including data modeling, performance tuning, and capacity planning
- Thorough understanding of cloud computing fundamentals (AWS preferred)
- Solid experience building high-scale ETL/ELT pipelines and working with streaming and batch data
- Experience working with NoSQL databases such as DynamoDB, Cassandra, or similar
- Comfortable with schema design trade-offs and query pattern optimizations in key-value and document stores
- Track record of collaborating effectively in cross-functional teams involving product, data science, and operations
- Strong verbal and written communication skills; able to explain technical concepts clearly to non-technical stakeholders
- Ability to thrive in fast-paced, evolving environments with changing requirements
- Bachelor's Degree, in lieu of a degree, demonstrating in addition to the minimum years of experience required for the role, three years of specialized training and/or progressively responsible work experience in technology for each missing year of college is required
Desired Qualifications
- Familiarity with distributed data processing frameworks; experience with Apache Spark is a significant plus but not required
- Hands-on experience with Apache Spark or other big data processing frameworks
- Familiarity with other AWS services such as Lambda, Kinesis, Glue, or Redshift
- Prior experience working in financial services or payments industry data platforms
- Exposure to containerized/cloud-native deployments (e.g., Docker, Kubernetes)
- Experience with CI/CD pipelines and infrastructure-as-code tools such as Terraform or CloudFormation
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.