Sr. Director, Software Engineering – Data Platform & MDM Integration
$185,000–$265,000 year
On-siteNew York City, New York, United States or Princeton, New Jersey, United States
Job Summary
Architect and oversee seamless, high-volume data integration between Databricks lakehouse and NeoXam MDM platform, establishing foundational data architecture for multi-billion-dollar index-linked investment products. Utilize cutting-edge Databricks features including Unity Catalog, Delta Live Tables, and Serverless compute to optimize data pipelines and maximize performance for massive-scale financial datasets. Implement Databricks Mosaic AI and agentic frameworks to automate data transformation, quality assurance, and anomaly detection across the entire conversion lifecycle. Lead, manage, and grow multiple teams of product-focused software and data engineers while mentoring their professional and technical development. Spearhead mission-critical migration of legacy assets and terabytes of data to modern AWS cloud ecosystem, breaking monolithic applications into microservice architectures. Foster a culture of cutting-edge technology adoption and continuous improvement of software engineering practices.
Required Qualifications
- Minimum 9 years of experience in software engineering and data platform development with proven track record in enterprise-scale solutions
- Minimum 7 years of people management experience leading technical teams and driving organizational growth in fast-paced environments
- Advanced expertise in cloud data platforms such as Databricks, Snowflake, or similar, with hands-on experience in data lakehouse architectures and ETL/ELT processes
- Strong experience with cloud infrastructure including AWS services (such as S3, EMR, Glue, Lambda) or equivalent cloud platforms (Azure, GCP)
- Proven leadership in large-scale data migrations and system modernization initiatives, with experience managing complex technical transformations
Desired Qualifications
- Master's degree in Computer Science, Software Engineering, or related technical discipline with 13+ years of progressive software development and data architecture experience
- Deep expertise in Master Data Management platforms such as NeoXam, Informatica MDM, or IBM InfoSphere, with proven success integrating enterprise data platforms
- Advanced AI/ML and Generative AI experience including hands-on work with machine learning frameworks (such as MLflow, Kubeflow, or SageMaker) and LLM implementation using tools like LangChain, Hugging Face, or similar platforms
- Extensive experience with Big Data technologies including distributed computing frameworks (such as Apache Spark, Hadoop, or Flink) and modern data processing architectures
- Bachelor's degree in Computer Science, Engineering, or related technical field
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.