AI Prompt & Agent Developer
$90,000–$130,000 year
On-siteSan Francisco, California, United States
Job Summary
Design and maintain production prompts powering enterprise voice AI agents for healthcare call centers, owning components like intent classification, scheduling workflows, and edge-case recovery. Analyze daily production conversations to identify failure modes, build evaluation datasets and automated testing frameworks, and conduct structured experiments to validate prompt changes using measurable outcomes. Develop human-in-the-loop onboarding workflows to configure agents for new customers while collaborating with engineering teams to improve internal tooling and dashboards. Help scale systems supporting millions of conversations by continuously improving AI behavior through data-driven iteration and production monitoring.
Required Qualifications
- 2+ years of professional experience in prompt engineering, conversational AI, or AI agent development
- Experience owning production prompt behavior for customer-facing AI systems
- Strong analytical skills with the ability to evaluate large datasets and identify behavioral improvements
- Excellent written communication and language intuition with strong attention to detail
- Familiarity with voice AI technologies, speech recognition (ASR), text-to-speech (TTS), or conversational AI platforms
- Comfortable reading Python and TypeScript code while collaborating closely with engineering teams
- Strong organizational skills with the ability to manage multiple experiments and deployments simultaneously
- Experience working in fast-paced startup environments with rapid iteration cycles
- Excellent cross-functional communication skills and customer-focused thinking
Desired Qualifications
- Experience building production AI agents serving thousands of users
- Background in linguistics, philosophy, law, cognitive science, or other language- and logic-intensive disciplines
- Experience with prompt evaluation frameworks, automated testing, or LLM optimization techniques
- Familiarity with telephony platforms such as Twilio or other voice infrastructure
- Experience working with healthcare workflows or enterprise AI products
- Strong understanding of LLM behavior, prompt engineering best practices, and AI evaluation methodologies
- Comfortable leveraging AI coding tools to accelerate development and experimentation
- Passion for building reliable, production-quality AI systems through continuous empirical improvement
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.