Search Results for author: Nelson F. Liu

Found 15 papers, 5 papers with code

Making Heads and Tails of Models with Marginal Calibration for Sparse Tagsets

1 code implementation Findings (EMNLP) 2021 Michael Kranzlein, Nelson F. Liu, Nathan Schneider

For interpreting the behavior of a probabilistic model, it is useful to measure a model's calibration--the extent to which it produces reliable confidence scores.


Identifying the Limits of Cross-Domain Knowledge Transfer for Pretrained Models

1 code implementation RepL4NLP (ACL) 2022 Zhengxuan Wu, Nelson F. Liu, Christopher Potts

There is growing evidence that pretrained language models improve task-specific fine-tuning not just for the languages seen in pretraining, but also for new languages and even non-linguistic data.

Pretrained Language Models Transfer Learning

Can Small and Synthetic Benchmarks Drive Modeling Innovation? A Retrospective Study of Question Answering Modeling Approaches

no code implementations1 Feb 2021 Nelson F. Liu, Tony Lee, Robin Jia, Percy Liang

While large, natural datasets are necessary for training accurate systems, are they necessary for driving modeling innovation?

Question Answering

Evaluating NLP Models via Contrast Sets

no code implementations1 Oct 2020 Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, A. Zhang, Ben Zhou

Unfortunately, when a dataset has systematic gaps (e. g., annotation artifacts), these evaluations are misleading: a model can learn simple decision rules that perform well on the test set but do not capture a dataset's intended capabilities.

Reading Comprehension Sentiment Analysis

Lexical Semantic Recognition

2 code implementations ACL (MWE) 2021 Nelson F. Liu, Daniel Hershcovich, Michael Kranzlein, Nathan Schneider

In lexical semantics, full-sentence segmentation and segment labeling of various phenomena are generally treated separately, despite their interdependence.

Sentence segmentation

Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets

no code implementations NAACL 2019 Nelson F. Liu, Roy Schwartz, Noah A. Smith

Several datasets have recently been constructed to expose brittleness in models trained on existing benchmarks.

Linguistic Knowledge and Transferability of Contextual Representations

no code implementations NAACL 2019 Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, Noah A. Smith

Contextual word representations derived from large-scale neural language models are successful across a diverse set of NLP tasks, suggesting that they encode useful and transferable features of language.

Language Modelling

Discovering Phonesthemes with Sparse Regularization

no code implementations WS 2018 Nelson F. Liu, Gina-Anne Levow, Noah A. Smith

We introduce a simple method for extracting non-arbitrary form-meaning representations from a collection of semantic vectors.

LSTMs Exploit Linguistic Attributes of Data

no code implementations WS 2018 Nelson F. Liu, Omer Levy, Roy Schwartz, Chenhao Tan, Noah A. Smith

While recurrent neural networks have found success in a variety of natural language processing applications, they are general models of sequential data.

Crowdsourcing Multiple Choice Science Questions

no code implementations WS 2017 Johannes Welbl, Nelson F. Liu, Matt Gardner

With this method we have assembled SciQ, a dataset of 13. 7K multiple choice science exam questions (Dataset available at http://allenai. org/data. html).

Multiple-choice Question Generation

Cannot find the paper you are looking for? You can Submit a new open access paper.