Senior Software Engineer, Machine Learning Infrastructure - Generative AI
$137,100–$201,600 year
On-siteSan Francisco, California, United States
Job Summary
Design and own the open-weights model platform spanning real-time GPU serving, high-throughput batch inference, and fine-tuning pipelines for DoorDash, Wolt, and Deliveroo. Architect scalable systems for model serving, GPU autoscaling, and observability while pushing the cost and latency frontier of inference. Set technical direction for emerging capabilities like reinforcement learning and agent optimization, partnering with ML engineers and product teams to turn GenAI prototypes into durable platform primitives. Lead design across ambiguous, high-impact systems and mentor engineers as you drive the velocity of business impact from AI across the company.
Required Qualifications
- B.S., M.S., or PhD. in Computer Science or equivalent
- 6+ years of industry experience in software engineering
- Deep backend engineering fundamentals, especially in Python and distributed systems
- Track record of designing and owning production services, APIs, data pipelines, or ML infrastructure at scale
- Experience operating systems in production, including observability, debugging, reliability, incident response, and performance/cost optimization
- Deep hands-on experience with LLM inference and/or fine-tuning of open-weight models in production — serving (latency, throughput, batching, autoscaling, GPU utilization) and/or fine-tuning (SFT/DPO/LoRA)
- Demonstrated technical leadership: leading design across ambiguous, fast-moving technical areas, mentoring engineers, and turning customer use cases into reusable platform capabilities
- Proficiency in using AI coding tools (e.g., Claude Code, Codex, Cursor) in the full software development lifecycle, including designing, generating code, testing, monitoring and releasing software
Desired Qualifications
- Experience with LLM inference engines and serving frameworks (e.g., vLLM, SGLang, TensorRT-LLM) in production
- Experience with distributed/multi-node fine-tuning and training pipelines (SFT, DPO/RLHF, LoRA), including data preparation and evaluation
- GPU performance work — multi-node/distributed inference, KV-cache/memory optimization, quantization (FP8/INT8/AWQ/GPTQ), or cold-start/throughput tuning
- Experience with Kubernetes, cloud infrastructure (AWS/GCP), GPUs, serverless/elastic GPU platforms (e.g., Modal), or high-throughput batch systems
- Experience with LLM gateways, model routing, vendor abstraction, or cost attribution
- Experience building developer platforms, internal platforms, or self-serve infrastructure
- Experience building and deploying AI agents or MCP servers in production
- Experience with eval systems, LLM observability, tracing, RAG, search, or vector databases
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.