Senior Data Platform Engineer
Remote · Colombia or Brazil
Job Summary
Senior Data Platform Engineer to build Databricks Lakehouse pipelines on AWS for a scientific data cloud platform in biopharma. You’ll prototyping pipelines, parsing proprietary instrument file formats, designing data models, building Python data pipelines, and shipping production-grade solutions directly with customers. Responsibilities include owning and deploying DataBricks Lakehouse pipelines on AWS, researching data acquisition strategies for scientific lab instrumentation, building file parsers for instrument outputs (.xlsx, .pdf, .txt, .raw, .fid, and vendor binaries), designing data models, unit/integration tests, and collaborating with customers to validate solutions. Requirements include 5+ years of Python and SQL, Databricks Lakehouse on AWS, AWS Redshift, AWS data stack (ECS, S3, Athena, RDS), ETL/ELT with Airflow, relational and non-relational databases, strong communication, and multi-project delivery capability. Nice-to-haves: Elasticsearch, background in scientific instrumentation or biopharma data, graduate degree in Chemistry/CS/Statistics/Public Health, and expertise parsing proprietary binary file formats from lab instruments. The role offers opportunities across global teams, continuous learning, innovation, and ethical AI standards.
Required Qualifications
- 5+ years of professional experience in Python and SQL
- Hands-on experience with Databricks Lakehouse architecture on AWS
- Strong working knowledge of AWS Redshift
- Experience across the AWS data stack: ECS, S3, Athena, RDS
- Proficiency with ETL/ELT pipeline development using Airflow and Python
- Experience with relational databases — MySQL, MariaDB, Aurora, PostgreSQL, MS SQL Server
- Familiarity with key-value and non-relational databases
- Excellent communication skills and the ability to take ownership of project delivery end-to-end
- Ability to manage multiple simultaneous projects without sacrificing quality
- Genuine curiosity and ability to learn new scientific domains and instrument ecosystems
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.