Hire Nicholas P. - Prompt Engineer - San Francisco

Appointed as inaugural Prompt Engineer, driving company-wide implementation of generative AI and LLM-powered solutions; shaping strategy and infrastructure from the ground up - Subject matter expert (SME) for generative AI and LLM tooling, consulting across multiple departments to optimize implementation, improve output quality and throughput, and ensure compliance through participation in AI review committees - Designed and implemented a Retrieval-Augmented Generation (RAG) pipeline to unify company-wide knowledge sources (including Confluence, Jira, Slack, and others), from ingestion, processing, embedding, and integration into Pinecone vector database - Developed a multi-agent workflow to surface upsell and cross-sell opportunities and detect churn risk factors, leveraging customer usage metrics, performance indicators, sales call transcripts, and support case logs to support Customer Success Managers in decision-making - Architect and iterate on scalable, modular AI systems, including multi-agent orchestration frameworks that integrate various APIs, internal data streams, and tooling into functional AI applications and copilots - Built Human-in-the-Loop (HITL) feedback and testing mechanisms to generate high-quality reinforcement learning signals, continuously improving AI tool performance and reliability

* Independently designed, developed, and deployed over 25 voice features on the classic Alexa NLU platform including hero features in support of product launches across U.S. and international locales - Iteratively engineered, tested, and reviewed LLM prompting for 16 features ahead of a relaunched generative AI Alexa platform, achieving accuracy improvements of up to 72% - Established program to scale and automate API intake process to final review team, establishing minimum quality requirements and broadly communicating program across Alexa LLM developer community, resulting in successful reduction in review times and increased throughput velocity - Developed and drove adoption of testing program for LLM modeling across Alexa organization, coordinating efforts among product, engineering, data science, and tooling teams toward a single unified strategy - Independently investigated performance of all photos features on Alexa platform, identifying root causes and implementing resolution of issues among ASR, NLU, and engineering to reduce defect rate across entire feature space by 36% - Led program to improve in-vehicle communication features across automotive segment resulting in a 13% reduction in defects across 4 locales and 2 languages driven by end-to-end investigation and resolution of high impact problem areas spanning ASR misrecognitions, multi-turn disambiguation, entity resolution, and device connectivity - Implemented collection of 20 total dashboards using custom SQL queries to surface defects across communication and photos feature spaces, expediting defect reduction efforts and visualizing impact on feature performance

· Conduct analyses of user data in order to gain insights into behaviors and performance, and provide analytical support to a range of internal customers including product managers, engineers, and senior executives. · Oversee a number of data pipelines used to evaluate customer success and inform product decisions. · Offer support to Language Engineers (Computational Linguists) as needed, including authoring FST grammars for NLU feature development and expansion, and natural language annotation design.