Lead Software Engineer- Python / Big Data / AWS/ ETL Pipelines
On-site · Jersey City, New Jersey, United States
Job Summary
Lead Software Engineer responsible for designing and delivering secure, scalable data and analytics platforms. Key duties include building and debugging production Python code, architecting data pipelines with Databricks, Spark, and ETL/ELT processes, and advancing the technology roadmap with AWS and modern web frameworks. Will lead and mentor engineering teams, implement Agile practices, CI/CD, automated testing, and ensure data quality, governance, and lineage across platforms. Role emphasizes real-time risk insights using AI/ML and analytics, development of dashboards for valuation and risk management, and integration of data cataloging and metadata management tools. Must have hands-on experience with Python, Databricks, Spark, cloud platforms, and data warehousing, plus demonstrated leadership in delivering complex projects in a financial data context.
Required Qualifications
- 5+ years applied experience in software engineering
- Hands-on experience with Python, Databricks, Spark
- Proficiency with cloud platforms (AWS, Azure, GCP) and data warehousing (Snowflake, Databricks)
- Experience leading engineering teams and delivering complex projects
- Experience with Agile development, CI/CD, automated testing, and documentation
- Strong understanding of data pipelines, data governance, and data quality
- Experience with data cataloging and metadata management tools (e.g., AWS Glue, Collibra, Atlan)
- Demonstrated ability to design scalable, secure, and resilient systems for large-scale data environments
- Experience building production-grade data pipelines and analytics dashboards
- Knowledge of AI/ML techniques and applying them to real-time analytics
- Exposure to financial markets, risk management, or valuation control functions
- Experience with modern web stacks (React, Angular) for dashboards
- Demonstrated experience in implementing standard software engineering practices
- Ability to define technology roadmap and drive adoption of new technologies
Apply with one swipe on Sorce. We auto-fill applications and apply on your behalf — no cover letters, no 40-minute forms.
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.