TY - JOUR
AU - 
AB - UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation Jian Guan, Minlie Huang Department of Computer Science and Technology, Institute for Artificial Intelligence, State Key Lab of Intelligent Technology and Systems, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China j-guan19@mails.tsinghua.edu.cn, aihuang@tsinghua.edu.cn Abstract Leading Context Jack was at the bar. Despite the success of existing referenced met- Reference By Human rics (e.g., BLEU and MoverScore), they cor- He noticed a phone on the floor. He was going to take it to lost and found. But it started ringing on the way. Jack relate poorly with human judgments for open- answered it and returned it to the owner’s friends. ended text generation including story or dia- log generation because of the notorious one- Sample 1 (Reasonable, B=0.29, M=0.49, U=1.00) On the way out he noticed a phone on the floor. He asked to-many issue: there are many plausible out- around if anybody owned it. Eventually he gave it to the puts for the same input, which may differ sub- bartender. They put it into their lost and found box. stantially in literal or semantics from the lim- ited number of given references. To allevi- Sample 
TI - UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation
JF - Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
DO - 10.18653/v1/2020.emnlp-main.736
DA - 2020-01-01
UR - https://www.deepdyve.com/lp/unpaywall/union-an-unreferenced-metric-for-evaluating-open-ended-story-b0A7DM5j3T
DP - DeepDyve
ER -