TY - JOUR
AU - 
AB - F ♦ F ♥ Suchin Gururangan Swabha Swayamdipta ♣ ♣♠ † ♣ Omer Levy Roy Schwartz Samuel R. Bowman Noah A. Smith Department of Linguistics, University of Washington, Seattle, WA, USA Language Technologies Institute, Carnegie Mellon University, Pittsburgh, PA, USA Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA Allen Institute for Artificial Intelligence, Seattle, WA, USA Center for Data Science and Department of Linguistics, New York University, New York, NY, USA {sg01,swabha,omerlevy,roysch,nasmith}@cs.washington.edu bowman@nyu.edu Abstract a premise p drawn from some corpus (e.g., image captions), and are required to generate three new Large-scale datasets for natural language in- sentences (hypotheses) based on p, according to ference are created by presenting crowd work- one of the following criteria: ers with a sentence (premise), and asking them to generate three new sentences (hypotheses) Entailment h is definitely true givenp that it entails, contradicts, or is logically neu- Neutral h might be true given p tral with respect to. We show that, in a signif- Contradiction h is definitelynot true given p icant portion of such data, this protocol leaves clues that make it possible to identify the label In this paper, we observe that hypotheses 
TI - Annotation Artifacts in Natural Language Inference Data
JF - Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers)
DO - 10.18653/v1/n18-2017
DA - 2018-01-01
UR - https://www.deepdyve.com/lp/unpaywall/annotation-artifacts-in-natural-language-inference-data-GbSdwVhRvo
DP - DeepDyve
ER -