TY - JOUR
AU - 
AB - Guanghui Qin and Jason Eisner Department of Computer Science, Johns Hopkins University gqin2@jhu.edu jason@cs.jhu.edu Abstract and paraphrasing based methods to automatically augment the prompt sets. Natural-language prompts have recently been Finding out what young children know is diffi- used to coax pretrained language models into cult because they can be very sensitive to the form performing other AI tasks, using a fill-in-the- of the question (Donaldson, 1978). Opinion polling blank paradigm (Petroni et al., 2019) or a is also sensitive to question design (Broughton, few-shot extrapolation paradigm (Brown et al., 1995). We observe that when we are querying 2020). For example, language models retain an LM rather than a human, we have the opportu- factual knowledge from their training corpora that can be extracted by asking them to “fill nity to tune prompts using gradient descent—the in the blank” in a sentential prompt. However, workhorse of modern NLP—so that they better where does this prompt come from? We ex- elicit the desired type of knowledge. plore the idea of learning prompts by gradi- A neural LM sees the prompt as a sequence of ent descent—either fine-tuning prompts taken continuous word vectors (Baroni et al., 2014). We from previous work, 
TI - Learning How to Ask: Querying LMs with Mixtures of Soft Prompts
JF - Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
DO - 10.18653/v1/2021.naacl-main.410
DA - 2021-01-01
UR - https://www.deepdyve.com/lp/unpaywall/learning-how-to-ask-querying-lms-with-mixtures-of-soft-prompts-DYHOOA2ufH
DP - DeepDyve
ER -