Staff Data Engineer
$364,000–$364,000 year
On-siteSan Jose, California, United States
Job Summary
Design and maintain high-throughput, fault-tolerant ingestion and transformation pipelines that feed training workloads at scale, defining table formats and partitioning strategies optimized for ML consumption. Build and operate the data lakehouse with instrumented data quality checks, lineage tracking, and anomaly detection to ensure models train on clean, well-governed data. Partner with ML engineers to define feature stores, dataset versioning, and experiment-to-production data contracts while integrating with tools like MLflow. Work closely with AI researchers and platform engineers to optimize query performance and unblock training runs. This Staff Data Engineer role sits within the AI Platform team at Archer, an aerospace company building an all-electric vertical takeoff and landing aircraft, focusing on software for the general aviation industry.
Required Qualifications
- 5+ years of professional data engineering experience excluding internships
- BS/MS/PhD in Computer Science, Data Engineering, Software Engineering, or a related field
- Hands-on experience building production pipelines with tools like Apache Spark, Flink, Airflow, dbt, or similar batch/streaming frameworks
- Deep proficiency with columnar formats (i.e. Parquet), open table formats (Apache Iceberg or Apache Paimon), and object storage systems (S3 or equivalent)
- Experience with high-throughput messaging systems such as Apache Pulsar or Kafka for real-time data ingestion
- Strong SQL skills
- experience with StarRocks for large-scale analytical queries and real-time analytics over the lakehouse
- Familiarity with AWS data services (S3, Glue, EMR) and containerized workloads (Docker/Kubernetes) in production environments
- Airflow/Prefect/Dagster
Desired Qualifications
- Experience with CDC (Change Data Capture) replication from transactional systems (i.e. PostgreSQL → Lakehouse via Debezium or Airbyte)
- Exposure to audio or time-series data pipelines, including preprocessing for ASR or speech model training
- Prior experience in aerospace, aviation, or other safety-critical domains where data lineage and auditability are non-negotiable
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.