Hire Esme C. - Prompt Engineer - San Francisco

* Designed, tested, and refined LLM prompts, templates, and evaluation workflows to assess accuracy, reasoning quality, instruction adherence, and policy alignment across multiple domains. * Built and maintained golden evaluation sets and scoring rubrics to benchmark model behavior against quality, safety, and trust & safety standards. * Analyzed model outputs to identify failure modes, edge cases, hallucinations, and misclassifications, translating findings into prioritized improvement recommendations. * Categorized large volumes of textual and multimodal content with high precision and consistency, ensuring alignment with detailed evaluation guidelines and quality standards. * Evaluated both textual and audio outputs for accuracy, relevance, and adherence to prompt requirements, demonstrating comfort with multimodal materials. * Followed comprehensive rubric and style guides consistently, ensuring reliable decisions across thousands of examples and minimizing inter-annotator variability. * Provided clear, structured feedback on edge cases, ambiguous outputs, and guideline gaps to improve evaluation processes. * Promoted to Reviewer and Team Lead, owning quality governance processes, mentoring evaluators, and coordinating feedback loops and incident reviews to ensure consistent outcomes across teams.

* Designed and evaluated prompts for diverse NLP tasks, including classification, summarization, and customer support simulation, using LLMs like GPT-4 and Claude. * Iteratively refined instructions and prompt structures to improve model output quality, alignment, and task success rates. * Documented prompt behavior and edge cases to support scalable prompt libraries and evaluation frameworks. * Developed intuition for few-shot prompting, instruction tuning, and hallucination mitigation through hands-on experimentation. * Collaborated with internal reviewers to improve quality assurance and consistency in output evaluations.

* Collaborated cross-functionally with product managers, designers, and engineers to translate complex user workflows into scalable technical implementations. * Built production web features supporting high-traffic consumer platforms, ensuring reliability, accessibility (WCAG), and consistent user experience. * Participated in iterative development cycles including QA validation, product testing, and performance monitoring. * Contributed to defining feature success metrics and evaluation strategies used to measure product impact and system reliability.