TY - JOUR
AU - McAuley, Julian
AB - Abstract:Chat models, such as ChatGPT, have shown impressive capabilities and have been rapidly adopted across numerous domains. However, these models are only accessible through a restricted API, creating barriers for new research and progress in the field. We propose a pipeline that can automatically generate a high-quality multi-turn chat corpus by leveraging ChatGPT to engage in a conversation with itself. Subsequently, we employ parameter-efficient tuning to enhance LLaMA, an open-source large language model. The resulting model, named Baize, demonstrates good performance in multi-turn dialogues with guardrails that minimize potential risks. Furthermore, we propose a new technique called Self-Distill with Feedback, to further improve the performance of the Baize models with feedback from ChatGPT. The Baize models and data are released for research purposes only at this https URL. An online demo is also available at this https URL.
TI - Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data
JF - Computing Research Repository
DO - 10.48550/arxiv.2304.01196
DA - 2023-04-03
UR - https://www.deepdyve.com/lp/arxiv-cornell-university/baize-an-open-source-chat-model-with-parameter-efficient-tuning-on-Zzhp2ateQD
VL - 2023
IS - 2304
DP - DeepDyve
ER -