Lead AI ML Engineer
RemoteUnited States
Job Summary
Build Navanta's retrieval and verification systems with shown queries and citations for every answer. Stand up self-hosted open-weight models serving and embeddings inside bank environments, evolving RAG to a dedicated standard. Design the MCP tool layer exposing audited read-only tools for metrics, documents, and customer 360 views. Maintain the evaluation harness including golden-question regression, groundedness metrics, and explicit "I don't know" behavior as a release gate. Implement LLM guardrails for PII redaction, prompt-injection defense, and cost limits aligned to regulatory security. Partner with data teams to ensure models select governed metrics from the semantic layer. Document architecture and guardrail controls for customer security reviews and audit readiness. Track latency, cost, and quality trade-offs across model versions. Collaborate with data and platform engineering teams while managing an on-call rotation for production reliability.
Required Qualifications
- 6–10+ years building software
- 2–3+ years shipping production LLM, RAG, or NLP systems used by real people
- A demonstrated focus on accuracy and evaluation, not just demos
- Strong Python
- Solid software-engineering fundamentals
- Comfort operating self-hosted open-weight models
- Reasoning about latency, cost, and quality trade-offs
- Bachelor's degree in computer science, mathematics, or a related technical field, or equivalent hands-on experience
- Up to 20% travel time
Desired Qualifications
- Experience in regulated or high-stakes domains where a wrong answer is costly
- Fine-tuning, adapters, and retrieval-quality optimization
- Familiarity with banking and finance terminology
- Experience in the financial services industry or a regulated, high-accuracy AI application environment
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.