Speech Research Intern-2

PhD Research Intern role focusing on designing and evaluating speech-first models, including spoken language models that reason over audio and interact conversationally. Responsibilities include prototyping end-to-end speech dialogue systems, aligning speech encoders with text backbones via lightweight adapters, efficient speech tokenization and temporal compression for long-form audio, and building an evaluation harness covering ASR/ST/SLU and speech QA with streaming metrics. Projects involve prototyping a conversational SLM with a speech encoder and adapters, creating data recipes blending speech with instruction-following corpora, and shipping a minimal demo with streaming inference and logging. Required skills include PhD candidacy in CS/EE (or related), Python/PyTorch with GPU experience, knowledge of Transformers/SSMs, and experience in at least one area such as discrete speech tokens, modality alignment via adapters, or post-training/instruction tuning for speech tasks. Preferred qualifications cover experience with neural speech codecs/vocoders, multilingual or code-switching speech, robustness and safety evaluation, distributed training (FSDP/DeepSpeed) and tools like ESPnet, SpeechBrain, NVIDIA NeMo, and experience with PyTorch ecosystem tools (CUDA, torchaudio/librosa, ONNX/TensorRT). Location options include Redmond, WA or Remote with flexible scheduling; compensation is a $45 per hour rate (annualized to $93,600) with a stipend and opportunities to publish, mentor, and access GPU infrastructure.