AI Engineer
RemoteUnited States
United StatesRemoteFull TimeStartup
Full TimeStartup
Job Summary
Build LLM-powered features, workflows, and agentic systems that solve real customer problems while investigating agent failures and developing systematic approaches to improve performance. Design evaluation frameworks, create automated benchmarks, and construct datasets to measure agent quality, reliability, and business impact. Experiment with prompting, tool use, and reasoning strategies to ship improvements to production AI systems used by customers within six months. Collaborate with customers and internal teams to understand workflows, identify automation opportunities, and establish rigorous evaluation practices across the company.
Required Qualifications
- Strong programming skills, preferably in Python
- Solid software engineering fundamentals, including testing, debugging, and system design
- Experience building applications, projects, or products using LLMs and modern AI tools
- Ability to design experiments, interpret results, and make data-driven decisions
- Strong communication skills
- Willingness to collaborate across disciplines
- Curiosity
- Ownership
- Desire to learn quickly
Desired Qualifications
- Experience building AI agents, copilots, or workflow automation systems
- Experience designing evaluations, benchmarks, or testing frameworks for AI systems
- Familiarity with OpenAI, Anthropic, Gemini, or open-source LLM ecosystems
- Experience with retrieval systems, vector databases, and RAG architectures
- Familiarity with LangGraph, OpenAI Agents SDK, MCP, or similar agent frameworks
- Experience with observability, tracing, and production monitoring for AI systems
- Exposure to cybersecurity, security operations, or developer tooling
- Open-source contributions, research projects, or personal AI products
- Experience as a software engineer, ML engineer, researcher, AI engineer, or founder
- Ability to learn quickly, work independently, and ship impactful systems
- Built and deployed AI agents
- Created evaluation frameworks for LLM applications
- Published AI projects, demos, or open-source contributions
- Developed tools that other people actively use
- Strong opinions about what makes AI systems reliable and useful
- Experience building AI products because you genuinely enjoy it
- D demonstrated ability rather than specific credentials
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.