Search Results for author: John Hewitt

Found 20 papers, 13 papers with code

Automatic Construction of Morphologically Motivated Translation Models for Highly Inflected, Low-Resource Languages

1 code implementation AMTA 2016 John Hewitt, Matt Post, David Yarowsky

Statistical Machine Translation (SMT) of highly inflected, low-resource languages suffers from the problem of low bitext availability, which is exacerbated by large inflectional paradigms.

Machine Translation Translation

Lost in the Middle: How Language Models Use Long Contexts

1 code implementation6 Jul 2023 Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang

While recent language models have the ability to take long contexts as input, relatively little is known about how well they use longer context.

Language Modelling Question Answering +1

Backpack Language Models

no code implementations26 May 2023 John Hewitt, John Thickstun, Christopher D. Manning, Percy Liang

We can interpret a sense vector by inspecting its (non-contextual, linear) projection onto the output space, and intervene on these interpretable hooks to change the model's behavior in predictable ways.

Language Modelling Text Generation +1

JamPatoisNLI: A Jamaican Patois Natural Language Inference Dataset

no code implementations7 Dec 2022 Ruth-Ann Armstrong, John Hewitt, Christopher Manning

While our work, along with previous work, shows that transfer from these models to low-resource languages that are unrelated to languages in their training set is not very effective, we would expect stronger results from transfer to creoles.

Cross-Lingual Transfer Few-Shot Learning +1

Truncation Sampling as Language Model Desmoothing

1 code implementation27 Oct 2022 John Hewitt, Christopher D. Manning, Percy Liang

In this light, truncation algorithms aim to perform desmoothing, estimating a subset of the support of the true distribution.

Language Modelling

Ensembles and Cocktails: Robust Finetuning for Natural Language Generation

no code implementations29 Sep 2021 John Hewitt, Xiang Lisa Li, Sang Michael Xie, Benjamin Newman, Percy Liang

When finetuning a pretrained language model for natural language generation tasks, one is currently faced with a tradeoff.

Language Modelling Text Generation

Conditional probing: measuring usable information beyond a baseline

1 code implementation EMNLP 2021 John Hewitt, Kawin Ethayarajh, Percy Liang, Christopher D. Manning

Probing experiments investigate the extent to which neural representations make properties -- like part-of-speech -- predictable.

Word Embeddings

On the Opportunities and Risks of Foundation Models

3 code implementations16 Aug 2021 Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora Demszky, Chris Donahue, Moussa Doumbouya, Esin Durmus, Stefano Ermon, John Etchemendy, Kawin Ethayarajh, Li Fei-Fei, Chelsea Finn, Trevor Gale, Lauren Gillespie, Karan Goel, Noah Goodman, Shelby Grossman, Neel Guha, Tatsunori Hashimoto, Peter Henderson, John Hewitt, Daniel E. Ho, Jenny Hong, Kyle Hsu, Jing Huang, Thomas Icard, Saahil Jain, Dan Jurafsky, Pratyusha Kalluri, Siddharth Karamcheti, Geoff Keeling, Fereshte Khani, Omar Khattab, Pang Wei Koh, Mark Krass, Ranjay Krishna, Rohith Kuditipudi, Ananya Kumar, Faisal Ladhak, Mina Lee, Tony Lee, Jure Leskovec, Isabelle Levent, Xiang Lisa Li, Xuechen Li, Tengyu Ma, Ali Malik, Christopher D. Manning, Suvir Mirchandani, Eric Mitchell, Zanele Munyikwa, Suraj Nair, Avanika Narayan, Deepak Narayanan, Ben Newman, Allen Nie, Juan Carlos Niebles, Hamed Nilforoshan, Julian Nyarko, Giray Ogut, Laurel Orr, Isabel Papadimitriou, Joon Sung Park, Chris Piech, Eva Portelance, Christopher Potts, aditi raghunathan, Rob Reich, Hongyu Ren, Frieda Rong, Yusuf Roohani, Camilo Ruiz, Jack Ryan, Christopher Ré, Dorsa Sadigh, Shiori Sagawa, Keshav Santhanam, Andy Shih, Krishnan Srinivasan, Alex Tamkin, Rohan Taori, Armin W. Thomas, Florian Tramèr, Rose E. Wang, William Wang, Bohan Wu, Jiajun Wu, Yuhuai Wu, Sang Michael Xie, Michihiro Yasunaga, Jiaxuan You, Matei Zaharia, Michael Zhang, Tianyi Zhang, Xikun Zhang, Yuhui Zhang, Lucia Zheng, Kaitlyn Zhou, Percy Liang

AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.

Transfer Learning

Refining Targeted Syntactic Evaluation of Language Models

1 code implementation NAACL 2021 Benjamin Newman, Kai-Siang Ang, Julia Gong, John Hewitt

Targeted syntactic evaluation of subject-verb number agreement in English (TSE) evaluates language models' syntactic knowledge using hand-crafted minimal pairs of sentences that differ only in the main verb's conjugation.

Probing artificial neural networks: insights from neuroscience

no code implementations16 Apr 2021 Anna A. Ivanova, John Hewitt, Noga Zaslavsky

A major challenge in both neuroscience and machine learning is the development of useful tools for understanding complex information processing systems.

BIG-bench Machine Learning

The EOS Decision and Length Extrapolation

1 code implementation EMNLP (BlackboxNLP) 2020 Benjamin Newman, John Hewitt, Percy Liang, Christopher D. Manning

Extrapolation to unseen sequence lengths is a challenge for neural generative models of language.

Finding Universal Grammatical Relations in Multilingual BERT

1 code implementation ACL 2020 Ethan A. Chi, John Hewitt, Christopher D. Manning

Recent work has found evidence that Multilingual BERT (mBERT), a transformer-based multilingual masked language model, is capable of zero-shot cross-lingual transfer, suggesting that some aspects of its representations are shared cross-lingually.

Language Modelling Zero-Shot Cross-Lingual Transfer

Designing and Interpreting Probes with Control Tasks

1 code implementation IJCNLP 2019 John Hewitt, Percy Liang

The selectivity of a probe puts linguistic task accuracy in context with the probe's capacity to memorize from word types.

Part-Of-Speech Tagging

A Structural Probe for Finding Syntax in Word Representations

1 code implementation NAACL 2019 John Hewitt, Christopher D. Manning

Recent work has improved our ability to detect linguistic knowledge in word representations.

Simple, Fast, Accurate Intent Classification and Slot Labeling for Goal-Oriented Dialogue Systems

no code implementations WS 2019 Arshit Gupta, John Hewitt, Katrin Kirchhoff

With the advent of conversational assistants, like Amazon Alexa, Google Now, etc., dialogue systems are gaining a lot of traction, especially in industrial setting.

General Classification Goal-Oriented Dialogue Systems +3

A Distributional and Orthographic Aggregation Model for English Derivational Morphology

1 code implementation ACL 2018 Daniel Deutsch, John Hewitt, Dan Roth

Modeling derivational morphology to generate words with particular semantics is useful in many text generation tasks, such as machine translation or abstractive question answering.

abstractive question answering Machine Translation +3

XNMT: The eXtensible Neural Machine Translation Toolkit

1 code implementation WS 2018 Graham Neubig, Matthias Sperber, Xinyi Wang, Matthieu Felix, Austin Matthews, Sarguna Padmanabhan, Ye Qi, Devendra Singh Sachan, Philip Arthur, Pierre Godard, John Hewitt, Rachid Riad, Liming Wang

In this paper we describe the design of XNMT and its experiment configuration system, and demonstrate its utility on the tasks of machine translation, speech recognition, and multi-tasked machine translation/parsing.

Machine Translation NMT +3

Cannot find the paper you are looking for? You can Submit a new open access paper.