Senior Software Engineer - GPU Local AI Platforms
$224,000–$356,500 year
On-siteSeattle, Washington, United States or Austin, Texas, United States
Job Summary
Track and evaluate innovations in leading open-source LLM inference frameworks to identify performance-critical features and algorithmic improvements relevant to NVIDIA edge AI hardware. Analyze how new model architectures and inference algorithms map onto NVIDIA GPU architecture to identify mismatches, fallback paths, and optimization opportunities. Characterize multi-node inference behavior regarding collective communication primitives, topology-aware all-reduce strategies, and parallelism efficiency on edge cluster configurations. Produce performance analysis reports mapping theoretical hardware limits to observed inference throughput, latency, and utilization. Own the model validation workflow for new model releases, including architecture compatibility assessment, inference recipe development, and publication to developer recipe sites. Develop and maintain developer-facing inference recipes, ensuring accuracy as frameworks evolve and automating staleness detection. Engage with community and partners on model bring-up questions, serving as the technical point of contact for hardware-specific inference issues.
Required Qualifications
- BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience
- 12+ years of software engineering with depth in GPU computing, ML systems, or high-performance inference
- Strong Python or C++ programming, software design, and software engineering skills
- Hands-on experience with GPU kernel development or optimization (CUDA/C++, Triton, or equivalent) — you understand how thread blocks, memory hierarchy, and warp execution affect real-world performance
- Working knowledge of LLM inference internals: attention mechanisms, KV-cache management, continuous batching, quantization formats, and tensor parallelism
- Container engineering expertise: multi-architecture Docker or OCI builds, layer optimization, runtime configuration, NVIDIA Container Toolkit
- Strong analytical skills: ability to form a performance hypothesis, design an experiment, interpret results, and communicate findings clearly
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.