Prompt Engineer
$139,872–$209,808 year
On-siteJersey City, New Jersey, United States
Job Summary
Prompt Engineer responsible for designing, testing, governing, and continuously improving prompts, system instructions, conversation flows, and interaction patterns for LLM applications. Ensure model outputs are accurate, grounded, safe, consistent, cost-aware, and aligned with business and compliance expectations. Own prompt patterns, system instructions, response templates, and conversation policies; manage prompt testing, versioning, evaluation, and quality workflows; conduct prompt-security and output-quality analysis for RAG-enabled assistants; collaborate with engineers to implement prompt versioning, testing, deployment, and monitoring in production; evaluate performance using metrics such as task success, groundedness, hallucination rate, completeness, safety, and user satisfaction.
Required Qualifications
- 4+ Years Strong understanding of LLM behavior, prompt design, tokenization, context windows, RAG, embeddings, and model limitations
- 4+ years Hands-on experience with OpenAI APIs, Azure OpenAI, Anthropic, LangChain, LlamaIndex, Semantic Kernel, or similar platforms
- Ability to debug LLM outputs using structured testing, error analysis, and iterative refinement
- Strong writing, analytical, communication, and stakeholder-management skills
- Understanding of prompt-security risks including prompt injection, data leakage, and hallucination
Desired Qualifications
- 4+ Years Strong understanding of LLM behavior, prompt design, tokenization, context windows, RAG, embeddings, and model limitations
- 4+ years Hands-on experience with OpenAI APIs, Azure OpenAI, Anthropic, LangChain, LlamaIndex, Semantic Kernel, or similar platforms
- Ability to debug LLM outputs using structured testing, error analysis, and iterative refinement
- Strong writing, analytical, communication, and stakeholder-management skills
- Understanding of prompt-security risks including prompt injection, data leakage, and hallucination
Hiring someone like this?
Get your role in front of qualified candidates on Sorce.