Lead Consultant Data Engineer (DHS)
$138,000–$185,000 year
On-siteArlington, Virginia, United States
Job Summary
Design and build modern data solutions, including data lakes and cleansed data repositories, by developing robust, scalable, and sustainable data pipelines using batch and streaming technologies. Create and automate data processes to ensure availability and usability, writing clean, efficient, and well-documented code while monitoring data quality through automated testing frameworks. Collaborate with Architects, Product Owners, Data Scientists, and DevOps to integrate data management solutions into client environments, research acquisition sources, and identify risks with recovery plans. Maintain data repositories such as data warehouses, data lakes, and operational data stores while ensuring secure, well-structured data stores.
Required Qualifications
- 8+ years of professional experience in data engineering or related fields
- Proven ability to build robust, scalable data pipelines and production-grade ETL/ELT systems
- Strong proficiency in SQL, Python, and orchestration tools like Airflow and dbt
- 2+ years of hands-on experience with Databricks, including development, data processing, and pipeline optimization in a cloud-based environment
- Hands-on experience with big data technologies such as Spark, Kafka, and file formats like Parquet, Delta Lake, and Iceberg
- Deep experience with AWS cloud data platforms (e.g., AWS Glue, S3, Redshift, EMR, BigQuery, or Azure equivalents)
- Solid understanding of data modeling, performance optimization, and designing secure, well-structured data stores
- Familiarity with data lake and analytical architecture patterns, including Star Schema, schema-on-read, and data quality frameworks
- Experience with CI/CD, Git, infrastructure-as-code (e.g., Terraform), and NoSQL databases
- Effective communicator with experience working in Agile environments (Scrum/Kanban) and collaborating across technical and non-technical teams
- Strong problem-solving skills, a growth mindset, and a passion for learning new technologies
- Ability to hold and maintain a DHS Public Trust (requires US Citizenship)
Desired Qualifications
- One or more of the following certifications: AWS Certified Machine Learning - Specialty, Data Science Council of America (DASCA) Certifications, Databricks Certified Machine Learning Associate, Databricks Certified Developer for Apache Spark, Databricks Certified Data Engineer Associate, Databricks Certified Data Engineer Professional, AWS Certified Data Analytics - Specialty, Python or Scala Programming Certification
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.