Senior AI Scientist
$123,500–$197,600 year
RemoteUnited States
Job Summary
Design and train large-scale transformer and hybrid foundation models, owning architecture choices across text, multimodal, and emerging paradigms. Diagnose and resolve training instabilities at scale while navigating compute versus data tradeoffs to define technical direction for next-generation models. Build models from first principles, modify transformer architectures including attention variants and Mixture-of-Experts, and experiment with loss functions for alignment. Execute large-scale distributed training using FSDP, ZeRO-3, and tensor parallelism in partnership with ML systems teams. Apply scaling laws to make informed decisions on model size, dataset size, and training duration.
Required Qualifications
- Strong fundamentals in deep learning and representation learning
- Design and modify transformer architectures, including: Attention variants RoPE, ALiBi Grouped Query Attention (GQA) Mixture‐of‐Experts (MoE)
- Build models from first principles, not just adapt pre‐existing codebases
- Own optimiser and scheduler choices, including: AdamW Lion Adafactor
- Learning‐rate and warmup schedulers
- Understand and debug: Optimiser instability Gradient pathologies
- Apply and validate scaling laws
- Navigate Chinchilla‐style compute vs data tradeoffs
- Make informed decisions about model size, dataset size, and training duration
- Design and experiment with loss functions including: Next‐token prediction Contrastive objectives RLHF, DPO, GRPO
- Understand how loss design impacts convergence, generalization, and alignment
- Design and execute large‐scale training using: FSDP ZeRO‐3 Tensor parallelism Pipeline parallelism
- Apply: Mixed precision (bf16, fp8) Gradient checkpointing
- Partner closely with ML systems teams
- Basic knowledge of information security and data privacy requirements
- Demonstrative knowledge of information security through internal training programs
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.