Staff Data Engineer
$185,000–$195,000 year
HybridNew York, United States
Job Summary
Own ingestion pipelines end-to-end by designing CDC patterns, Autoloader, and batch loads from operational sources into the Bronze layer. Build and maintain dbt models within a medallion architecture, ensuring testing, alerting, and Unity Catalog governance for sensitive data. Drive the active migration from GCP to AWS/Databricks, validating parity during the cutover of Airflow, Cloud Run, and BigQuery workloads while decommissioning legacy components. Establish engineering patterns for reverse ETL flows that sync curated data back into Salesforce, NetSuite, and other operational systems. Architect solutions before coding by writing ADRs, defining interfaces, and documenting deliberate build-vs-manage tradeoffs. Partner with Sales, Finance, Marketing, and Clinical teams to translate business workflows into scoped pipeline requirements. This role supports the migration of internal and external AI agents while maintaining traditional BI capabilities on a small, high-ownership team.
Required Qualifications
- 5+ years of data engineering experience, with clear evidence of seniority: owned complex domains, designed solutions from scratch, and made architectural decisions — not just executed tickets.
- Strong hands-on dbt experience: models, tests, sources, macros, documentation, CI integration, and refactoring existing work.
- Demonstrated experience with both streaming and batch ingestion patterns, including CDC pipelines, event-driven architectures, and scheduled bulk loads from operational sources.
- Experience with Databricks (Delta Lake, Delta Live Tables) or a comparable lakehouse platform.
- Solid Python for data engineering: PySpark, pipeline development, utilities, and custom tooling.
- Hands-on experience integrating with operational data sources: CRM (Salesforce), ERP (NetSuite), payments (Stripe), or similar.
- Ability to design before building: write the doc, define the interface, identify the failure modes, then implement.
- Experience with data quality, testing, and pipeline observability: dbt tests, Great Expectations, alerting, SLA tracking.
- Must be legally authorized to work in the United States and do not require sponsorship for employment visa status.
- Butterfly Network does not accept agency resumes.
- Butterfly Network is an E-Verify Company.
- Butterfly requires security adherence responsibilities from all employees: adhering to all company security policies and procedures, utilize provided company assets securely, and complete all required security awareness training programs.
- Immediately reporting any suspected or actual security incidents, including phishing attempts, malware infections, or unauthorized access, following the established incident response procedure
Desired Qualifications
- Ideally, you also have these skills/experiences/attributes (but it's ok if you don't!):
- Experience with Unity Catalog or an equivalent governance layer for column masking, row-level security, lineage, and sensitivity tags.
- Familiarity with reverse ETL or integration platforms (Mulesoft, Boomi, or equivalent).
- IaC experience at the data workload level (Terraform, Spacelift, or equivalents).
- Experience in healthcare, life sciences, or another regulated environment (HIPAA, PII/PHI classification and handling).
- Exposure to AI/ML platform patterns: vector search, RAG pipelines, or model serving data flows.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.