TY - JOUR
AU - Williams, Joseph Jay
AB - Abstract: Large-Language Models like GPT-3 have the potential to enable HCI designers and researchers to create more human-like and helpful chatbots for specific applications. But evaluating the feasibility of these chatbots and designing prompts that optimize GPT-3 for a specific task is challenging. We present a case study in tackling these questions, applying GPT-3 to a brief 5-minute chatbot that anyone can talk to better manage their mood. We report a randomized factorial experiment with 945 participants on Mechanical Turk that tests three dimensions of prompt design to initialize the chatbot (identity, intent, and behaviour), and present both quantitative and qualitative analyses of conversations and user perceptions of the chatbot. We hope other HCI designers and researchers can build on this case study, for other applications of GPT-3 based chatbots to specific tasks, and build on and extend the methods we use for prompt design, and evaluation of the prompt design.
TI - Exploring The Design of Prompts For Applying GPT-3 based Chatbots: A Mental Wellbeing Case Study on Mechanical Turk
JF - Computing Research Repository
DO - 10.48550/arxiv.2209.11344
DA - 2022-09-22
UR - https://www.deepdyve.com/lp/arxiv-cornell-university/exploring-the-design-of-prompts-for-applying-gpt-3-based-chatbots-a-0HAApK5JCX
VL - 2023
IS - 2209
DP - DeepDyve
ER -