Ab Initio Developer
HybridCharlotte, North Carolina, United States
Job Summary
Design, develop, and maintain scalable batch and near real-time data pipelines using Ab Initio, Python, PySpark, and SQL. Build and optimize Google BigQuery datasets while supporting migration initiatives from Teradata and Ab Initio to Google Cloud Platform. Develop workflow orchestration using Autosys and Cloud Composer, implement metadata management with Dataplex, and enforce enterprise data quality controls via Informatica. Monitor production pipelines, troubleshoot failures, and apply secure data engineering practices including PII protection and audit documentation. Partner with Product Owners and Architects to define technical solutions and create comprehensive technical documentation. Provide technical leadership to junior engineers while utilizing AI-assisted development tools to improve productivity.
Required Qualifications
- 4+ years of professional Data Engineering experience
- 4+ years of experience with PL/SQL and SQL, including complex query development, optimization, and troubleshooting
- Hands-on experience with Oracle, Teradata, Python, and/or Google BigQuery
- 4+ years of experience developing enterprise ETL solutions using Ab Initio, including graph development, Psets, and performance tuning
- 3+ years of experience programming in Python with hands-on PySpark development
- 3+ years of experience with ETL architecture, data warehousing concepts, dimensional modeling, and data integration best practices
- Experience building scalable batch and near real-time data processing solutions
- Strong understanding of enterprise data governance, metadata management, and data lifecycle management
- Must-have: Use AI-assisted coding tools (e.g., GitHub Copilot, Devin, or similar) to accelerate development while maintaining strong code review discipline, testing, and secure coding standards
Desired Qualifications
- Experience with Google Cloud Platform (GCP) services including: BigQuery, Dataplex, Google Cloud Composer (Apache Airflow)
- Experience migrating enterprise data platforms from on-premises environments to cloud-native architectures
- Experience with Informatica Data Quality (IDQ)
- Familiarity with Autosys scheduling and workload automation
- Experience implementing secure data engineering practices for regulated environments
- Experience using AI-assisted software development tools such as GitHub Copilot or Devin
- Experience working in Agile/Scrum environments
- Financial services or mortgage/lending industry experience is a plus
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.