no code implementations • EMNLP 2020 • Albert Webson, Zhizhong Chen, Carsten Eickhoff, Ellie Pavlick
In politics, neologisms are frequently invented for partisan objectives.
no code implementations • ACL (NALOMA, IWCS) 2021 • Aaron Traylor, Ellie Pavlick, Roman Feiman
In modern natural language processing pipelines, it is common practice to “pretrain” a generative language model on a large corpus of text, and then to “finetune” the created representations by continuing to train them on a discriminative textual inference task.
no code implementations • EMNLP 2021 • Roma Patel, Ellie Pavlick
People use language in subtle and nuanced ways to convey their beliefs.
no code implementations • 31 Mar 2022 • Tian Yun, Usha Bhalla, Ellie Pavlick, Chen Sun
Our study reveals that state-of-the-art VL pretrained models learn primitive concepts that are highly useful as visual descriptors, as demonstrated by their strong performance on fine-grained visual recognition tasks, but those concepts struggle to provide interpretable compositional derivations, which highlights limitations of existing VL models.
1 code implementation • 10 Nov 2021 • Babak Hemmatian, Sheridan Feucht, Rachel Avram, Alexander Wey, Muskaan Garg, Kate Spitalnic, Carsten Eickhoff, Ellie Pavlick, Bjorn Sandstede, Steven Sloman
We present a novel corpus of 445 human- and computer-generated documents, comprising about 27, 000 clauses, annotated for semantic clause types and coherence relations that allow for nuanced comparison of artificial and natural discourse modes.
no code implementations • ICLR 2022 • Roma Patel, Ellie Pavlick
A fundamental criticism of text-only language models (LMs) is their lack of grounding---that is, the ability to tie a word for which they have learned a representation, to its actual use in the world.
1 code implementation • Findings (EMNLP) 2021 • Tian Yun, Chen Sun, Ellie Pavlick
Linguistic representations derived from text alone have been criticized for their lack of grounding, i. e., connecting words to their meanings in the physical world.
1 code implementation • EMNLP 2021 • Jason Wei, Dan Garrette, Tal Linzen, Ellie Pavlick
Pre-trained language models perform well on a variety of linguistic tasks that require symbolic reasoning, raising the question of whether such models implicitly represent abstract symbols and rules.
no code implementations • CoNLL (EMNLP) 2021 • Mostafa Abdou, Artur Kulmizev, Daniel Hershcovich, Stella Frank, Ellie Pavlick, Anders Søgaard
Pretrained language models have been shown to encode relational information, such as the relations between entities or concepts in knowledge-bases -- (Paris, Capital, France).
1 code implementation • 2 Sep 2021 • Albert Webson, Ellie Pavlick
We find that models learn just as fast with many prompts that are intentionally irrelevant or even pathologically misleading as they do with instructively "good" prompts.
no code implementations • ACL 2021 • Aaron Traylor, Roman Feiman, Ellie Pavlick
A current open question in natural language processing is to what extent language models, which are trained with access only to the form of language, are able to capture the meaning of language.
1 code implementation • ICLR 2022 • Thibault Sellam, Steve Yadlowsky, Jason Wei, Naomi Saphra, Alexander D'Amour, Tal Linzen, Jasmijn Bastings, Iulia Turc, Jacob Eisenstein, Dipanjan Das, Ian Tenney, Ellie Pavlick
Experiments with pre-trained models such as BERT are often based on a single checkpoint.
no code implementations • ACL 2021 • Najoung Kim, Ellie Pavlick, Burcu Karagol Ayan, Deepak Ramachandran
Through a user preference study, we demonstrate that the oracle behavior of our proposed system that provides responses based on presupposition failure is preferred over the oracle behavior of existing QA systems.
no code implementations • ICLR 2021 • Charles Lovering, Rohan Jha, Tal Linzen, Ellie Pavlick
In this work, we test the hypothesis that the extent to which a feature influences a model's decisions can be predicted using a combination of two factors: The feature's "extractability" after pre-training (measured using information-theoretic probing techniques), and the "evidence" available during fine-tuning (defined as the feature's co-occurrence rate with the label).
1 code implementation • 4 Dec 2020 • Kaiyu Zheng, Deniz Bayazit, Rebecca Mathew, Ellie Pavlick, Stefanie Tellex
We propose SLOOP (Spatial Language Object-Oriented POMDP), a new framework for partially observable decision making with a probabilistic observation model for spatial language.
1 code implementation • Joint Conference on Lexical and Computational Semantics 2020 • Dylan Ebert, Ellie Pavlick
We introduce a new dataset for training and evaluating grounded language models.
no code implementations • 12 Oct 2020 • Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, Slav Petrov
Pre-trained models have revolutionized natural language understanding.
no code implementations • 10 Oct 2020 • Charles Lovering, Ellie Pavlick
When communicating, people behave consistently across conversational roles: People understand the words they say and are able to produce the words they hear.
1 code implementation • 6 Oct 2020 • Albert Webson, Zhizhong Chen, Carsten Eickhoff, Ellie Pavlick
In politics, neologisms are frequently invented for partisan objectives.
no code implementations • ACL 2020 • Yonatan Belinkov, Sebastian Gehrmann, Ellie Pavlick
While deep learning has transformed the natural language processing (NLP) field and impacted the larger computational linguistics community, the rise of neural networks is stained by their opaque nature: It is challenging to interpret the inner workings of neural network models, and explicate their behavior.
1 code implementation • 23 Jun 2020 • Thao Nguyen, Nakul Gopalan, Roma Patel, Matt Corsaro, Ellie Pavlick, Stefanie Tellex
The model takes in a language command containing a verb, for example "Hand me something to cut," and RGB images of candidate objects and selects the object that best satisfies the task specified by the verb.
no code implementations • 30 Apr 2020 • Rohan Jha, Charles Lovering, Ellie Pavlick
Neural models often exploit superficial features to achieve good performance, rather than deriving more general features.
no code implementations • EMNLP (BlackboxNLP) 2020 • Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, Ian Tenney
While there has been much recent work studying how linguistic information is encoded in pre-trained sentence representations, comparatively little is understood about how these models change when adapted to solve downstream tasks.
no code implementations • IJCNLP 2019 • Alexis Ross, Ellie Pavlick
In natural language inference (NLI), contexts are considered veridical if they allow us to infer that their underlying propositions make true claims about the real world.
no code implementations • 25 Sep 2019 • Charles Lovering, Ellie Pavlick
When communicating, humans rely on internally-consistent language representations.
no code implementations • WS 2019 • Dylan Ebert, Ellie Pavlick
The fields of cognitive science and philosophy have proposed many different theories for how humans represent {``}concepts{''}.
1 code implementation • 28 May 2019 • Yoonseon Oh, Roma Patel, Thao Nguyen, Baichuan Huang, Ellie Pavlick, Stefanie Tellex
Often times, we specify tasks for a robot using temporal language that can also span different levels of abstraction.
2 code implementations • ICLR 2019 • Ian Tenney, Patrick Xia, Berlin Chen, Alex Wang, Adam Poliak, R. Thomas McCoy, Najoung Kim, Benjamin Van Durme, Samuel R. Bowman, Dipanjan Das, Ellie Pavlick
The jiant toolkit for general-purpose text understanding models
1 code implementation • ACL 2019 • Ian Tenney, Dipanjan Das, Ellie Pavlick
Pre-trained text encoders have rapidly advanced the state of the art on many NLP tasks.
no code implementations • ICLR 2019 • Samuel R. Bowman, Ellie Pavlick, Edouard Grave, Benjamin Van Durme, Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen
Work on the problem of contextualized word representation—the development of reusable neural network components for sentence understanding—has recently seen a surge of progress centered on the unsupervised pretraining task of language modeling with methods like ELMo (Peters et al., 2018).
no code implementations • SEMEVAL 2019 • Najoung Kim, Roma Patel, Adam Poliak, Alex Wang, Patrick Xia, R. Thomas McCoy, Ian Tenney, Alexis Ross, Tal Linzen, Benjamin Van Durme, Samuel R. Bowman, Ellie Pavlick
Our results show that pretraining on language modeling performs the best on average across our probing tasks, supporting its widespread use for pretraining state-of-the-art NLP models, and CCG supertagging and NLI pretraining perform comparably.
no code implementations • TACL 2019 • Ellie Pavlick, Tom Kwiatkowski
We analyze human{'}s disagreements about the validity of natural language inferences.
5 code implementations • ACL 2019 • R. Thomas McCoy, Ellie Pavlick, Tal Linzen
We find that models trained on MNLI, including BERT, a state-of-the-art model, perform very poorly on HANS, suggesting that they have indeed adopted these heuristics.
no code implementations • ACL 2019 • Alex Wang, Jan Hula, Patrick Xia, Raghavendra Pappagari, R. Thomas McCoy, Roma Patel, Najoung Kim, Ian Tenney, Yinghui Huang, Katherin Yu, Shuning Jin, Berlin Chen, Benjamin Van Durme, Edouard Grave, Ellie Pavlick, Samuel R. Bowman
Natural language understanding has recently seen a surge of progress with the use of sentence encoders like ELMo (Peters et al., 2018a) and BERT (Devlin et al., 2019) which are pretrained on variants of language modeling.
no code implementations • EMNLP 2018 • Anne Cocos, Skyler Wharton, Ellie Pavlick, Marianna Apidianaki, Chris Callison-Burch
Adjectives like {``}warm{''}, {``}hot{''}, and {``}scalding{''} all describe temperature but differ in intensity.
no code implementations • EMNLP 2018 • Manaal Faruqui, Ellie Pavlick, Ian Tenney, Dipanjan Das
We release a corpus of 43 million atomic edits across 8 languages.
no code implementations • EMNLP (ACL) 2018 • Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, Benjamin Van Durme
We present a large-scale collection of diverse natural language inference (NLI) datasets that help provide insight into how well a sentence representation captures distinct types of reasoning.
no code implementations • ACL 2017 • Ellie Pavlick, Marius Pa{\c{s}}ca
We present a method for populating fine-grained classes (e. g., {``}1950s American jazz musicians{''}) with instances (e. g., Charles Mingus ).
no code implementations • TACL 2016 • Ellie Pavlick, Joel Tetreault
This paper presents an empirical study of linguistic formality.
1 code implementation • TACL 2016 • Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, Chris Callison-Burch
Most recent sentence simplification systems use basic machine translation models to learn lexical and syntactic paraphrases from a manually simplified parallel corpus.
Ranked #7 on
Text Simplification
on TurkCorpus
no code implementations • TACL 2014 • Ellie Pavlick, Matt Post, Ann Irvine, Dmitry Kachaev, Chris Callison-Burch
We present a large scale study of the languages spoken by bilingual workers on Mechanical Turk (MTurk).