Search Results for author: Jason Eisner

Found 89 papers, 28 papers with code

Online Semantic Parsing for Latency Reduction in Task-Oriented Dialogue

no code implementations ACL 2022 Jiawei Zhou, Jason Eisner, Michael Newman, Emmanouil Antonios Platanios, Sam Thomson

Standard conversational semantic parsing maps a complete user utterance into an executable program, after which the program is executed to respond to the user.

Machine Translation Semantic Parsing +1

Learning to Retrieve Iteratively for In-Context Learning

no code implementations20 Jun 2024 Yunmo Chen, Tongfei Chen, Harsh Jhamtani, Patrick Xia, Richard Shin, Jason Eisner, Benjamin Van Durme

We introduce iterative retrieval, a novel framework that empowers retrievers to make iterative decisions through policy optimization.

Combinatorial Optimization In-Context Learning +2

LLMs in the Imaginarium: Tool Learning through Simulated Trial and Error

1 code implementation7 Mar 2024 Boshi Wang, Hao Fang, Jason Eisner, Benjamin Van Durme, Yu Su

We find that existing LLMs, including GPT-4 and open-source LLMs specifically fine-tuned for tool use, only reach a correctness rate in the range of 30% to 60%, far from reliable use in practice.

Continual Learning In-Context Learning

Principled Gradient-based Markov Chain Monte Carlo for Text Generation

no code implementations29 Dec 2023 Li Du, Afra Amini, Lucas Torroba Hennigen, Xinyan Velocity Yu, Jason Eisner, Holden Lee, Ryan Cotterell

Recent papers have demonstrated the possibility of energy-based text generation by adapting gradient-based sampling algorithms, a paradigm of MCMC algorithms that promises fast convergence.

Language Modelling Text Generation

Do Androids Know They're Only Dreaming of Electric Sheep?

no code implementations28 Dec 2023 Sky CH-Wang, Benjamin Van Durme, Jason Eisner, Chris Kedzie

We design probes trained on the internal representations of a transformer language model to predict its hallucinatory behavior on three grounded generation tasks.

Hallucination Hallucination Evaluation +1

Structure-Aware Path Inference for Neural Finite State Transducers

no code implementations21 Dec 2023 Weiting Tan, Chu-Cheng Lin, Jason Eisner

In this paper, we focus on the resulting challenge of imputing the latent alignment path that explains a given pair of input and output strings (e. g., during training).

A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia

1 code implementation4 Dec 2023 Giovanni Monea, Maxime Peyrard, Martin Josifoski, Vishrav Chaudhary, Jason Eisner, Emre Kiciman, Hamid Palangi, Barun Patra, Robert West

We present a novel method to study grounding abilities using Fakepedia, a novel dataset of counterfactual texts constructed to clash with a model's internal parametric knowledge.

counterfactual Language Modelling +1

Interpreting User Requests in the Context of Natural Language Standing Instructions

1 code implementation16 Nov 2023 Nikita Moghe, Patrick Xia, Jacob Andreas, Jason Eisner, Benjamin Van Durme, Harsh Jhamtani

Users of natural language interfaces, generally powered by Large Language Models (LLMs), often must repeat their preferences each time they make a similar request.

Toward Interactive Dictation

no code implementations8 Jul 2023 Belinda Z. Li, Jason Eisner, Adam Pauls, Sam Thomson

Voice dictation is an increasingly important text input modality.

Efficient Semiring-Weighted Earley Parsing

1 code implementation6 Jul 2023 Andreas Opedal, Ran Zmigrod, Tim Vieira, Ryan Cotterell, Jason Eisner

This paper provides a reference description, in the form of a deduction system, of Earley's (1970) context-free parsing algorithm with various speed-ups.

Sentence

Decision-Oriented Dialogue for Human-AI Collaboration

1 code implementation31 May 2023 Jessy Lin, Nicholas Tomlin, Jacob Andreas, Jason Eisner

We describe a class of tasks called decision-oriented dialogues, in which AI assistants such as large language models (LMs) must collaborate with one or more humans via natural language to help them make complex decisions.

Autoregressive Modeling with Lookahead Attention

no code implementations20 May 2023 Li Du, Hongyuan Mei, Jason Eisner

To predict the next token, autoregressive models ordinarily examine the past.

Morphological Inflection

Algorithms for Acyclic Weighted Finite-State Automata with Failure Arcs

1 code implementation17 Jan 2023 Anej Svete, Benjamin Dayan, Tim Vieira, Ryan Cotterell, Jason Eisner

The pathsum in ordinary acyclic WFSAs is efficiently computed by the backward algorithm in time $O(|E|)$, where $E$ is the set of transitions.

A Measure-Theoretic Characterization of Tight Language Models

no code implementations20 Dec 2022 Li Du, Lucas Torroba Hennigen, Tiago Pimentel, Clara Meister, Jason Eisner, Ryan Cotterell

Language modeling, a central task in natural language processing, involves estimating a probability distribution over strings.

Language Modelling

Contrastive Decoding: Open-ended Text Generation as Optimization

2 code implementations27 Oct 2022 Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, Mike Lewis

We propose contrastive decoding (CD), a reliable decoding approach that optimizes a contrastive objective subject to a plausibility constraint.

Language Modelling Text Generation

On the Intersection of Context-Free and Regular Languages

1 code implementation14 Sep 2022 Clemente Pasti, Andreas Opedal, Tiago Pimentel, Tim Vieira, Jason Eisner, Ryan Cotterell

It shows, by a simple construction, that the intersection of a context-free language and a regular language is itself context-free.

BenchCLAMP: A Benchmark for Evaluating Language Models on Syntactic and Semantic Parsing

1 code implementation NeurIPS 2023 Subhro Roy, Sam Thomson, Tongfei Chen, Richard Shin, Adam Pauls, Jason Eisner, Benjamin Van Durme

We introduce BenchCLAMP, a Benchmark to evaluate Constrained LAnguage Model Parsing, that includes context-free grammars for seven semantic parsing datasets and two syntactic parsing datasets with varied output representations, as well as a constrained decoding interface to generate only valid outputs covered by these grammars.

Decoder Language Modelling +3

Non-Programmers Can Label Programs Indirectly via Active Examples: A Case Study with Text-to-SQL

1 code implementation25 May 2022 Ruiqi Zhong, Charlie Snell, Dan Klein, Jason Eisner

We introduce APEL, a framework in which non-programmers select among candidate programs generated by a seed semantic parser (e. g., Codex).

Bayesian Inference Text-To-SQL

When More Data Hurts: A Troubling Quirk in Developing Broad-Coverage Natural Language Understanding Systems

1 code implementation24 May 2022 Elias Stengel-Eskin, Emmanouil Antonios Platanios, Adam Pauls, Sam Thomson, Hao Fang, Benjamin Van Durme, Jason Eisner, Yu Su

Rejecting class imbalance as the sole culprit, we reveal that the trend is closely associated with an effect we call source signal dilution, where strong lexical cues for the new symbol become diluted as the training dataset grows.

Intent Recognition Natural Language Understanding +1

Transformer Embeddings of Irregularly Spaced Events and Their Participants

1 code implementation ICLR 2022 Chenghao Yang, Hongyuan Mei, Jason Eisner

The neural Hawkes process (Mei & Eisner, 2017) is a generative model of irregularly spaced sequences of discrete events.

Searching for More Efficient Dynamic Programs

no code implementations Findings (EMNLP) 2021 Tim Vieira, Ryan Cotterell, Jason Eisner

To this end, we describe a set of program transformations, a simple metric for assessing the efficiency of a transformed program, and a heuristic search procedure to improve this metric.

Learning How to Ask: Querying LMs with Mixtures of Soft Prompts

2 code implementations NAACL 2021 Guanghui Qin, Jason Eisner

We explore the idea of learning prompts by gradient descent -- either fine-tuning prompts taken from previous work, or starting from random initialization.

Language Modelling

Limitations of Autoregressive Models and Their Alternatives

no code implementations NAACL 2021 Chu-Cheng Lin, Aaron Jaech, Xin Li, Matthew R. Gormley, Jason Eisner

Standard autoregressive language models perform only polynomial-time computation to compute the probability of the next symbol.

Language Modelling

Evaluation of Logic Programs with Built-Ins and Aggregation: A Calculus for Bag Relations

1 code implementation20 Oct 2020 Matthew Francis-Landau, Tim Vieira, Jason Eisner

We present a scheme for translating logic programs, which may use aggregation and arithmetic, into algebraic expressions that denote bag relations over ground terms of the Herbrand universe.

Programming Languages Symbolic Computation

Neural Datalog Through Time: Informed Temporal Modeling via Logical Specification

1 code implementation ICML 2020 Hongyuan Mei, Guanghui Qin, Minjie Xu, Jason Eisner

Learning how to predict future events from patterns of past events is difficult when the set of possible event types is large.

A Corpus for Large-Scale Phonetic Typology

no code implementations ACL 2020 Elizabeth Salesky, Eleanor Chodroff, Tiago Pimentel, Matthew Wiesner, Ryan Cotterell, Alan W. black, Jason Eisner

A major hurdle in data-driven research on typology is having sufficient data in many languages to draw meaningful conclusions.

Spelling-Aware Construction of Macaronic Texts for Teaching Foreign-Language Vocabulary

no code implementations IJCNLP 2019 Adithya Renduchintala, Philipp Koehn, Jason Eisner

We present a machine foreign-language teacher that modifies text in a student{'}s native language (L1) by replacing some word tokens with glosses in a foreign language (L2), in such a way that the student can acquire L2 vocabulary simply by reading the resulting macaronic text.

Language Modelling

Specializing Word Embeddings (for Parsing) by Information Bottleneck

1 code implementation IJCNLP 2019 Xiang Lisa Li, Jason Eisner

Pre-trained word embeddings like ELMo and BERT contain rich syntactic and semantic information, resulting in state-of-the-art performance on various tasks.

Dimensionality Reduction POS +2

Simple Construction of Mixed-Language Texts for Vocabulary Learning

no code implementations WS 2019 Adithya Renduchintala, Philipp Koehn, Jason Eisner

We accomplish this by modifying a cloze language model to incrementally learn new vocabulary items, and use this language model as a proxy for the word guessing and learning ability of real students.

Language Modelling

A Generative Model for Punctuation in Dependency Trees

no code implementations TACL 2019 Xiang Lisa Li, Dingquan Wang, Jason Eisner

When the tree's yield is rendered as a written sentence, a string rewriting mechanism transduces the underlying marks into "surface" marks, which are part of the observed (surface) string but should not be regarded as part of the tree.

Punctuation Restoration Sentence

What Kind of Language Is Hard to Language-Model?

no code implementations ACL 2019 Sabrina J. Mielke, Ryan Cotterell, Kyle Gorman, Brian Roark, Jason Eisner

Trying to answer the question of what features difficult languages have in common, we try and fail to reproduce our earlier (Cotterell et al., 2018) observation about morphological complexity and instead reveal far simpler statistics of the data that seem to drive complexity in a much larger sample.

Language Modelling Sentence

Neural Finite-State Transducers: Beyond Rational Relations

no code implementations NAACL 2019 Chu-Cheng Lin, Hao Zhu, Matthew R. Gormley, Jason Eisner

We introduce neural finite state transducers (NFSTs), a family of string transduction models defining joint and conditional probability distributions over pairs of strings.

Imputing Missing Events in Continuous-Time Event Streams

2 code implementations14 May 2019 Hongyuan Mei, Guanghui Qin, Jason Eisner

On held-out incomplete sequences, our method is effective at inferring the ground-truth unobserved events, with particle smoothing consistently improving upon particle filtering.

The CoNLL--SIGMORPHON 2018 Shared Task: Universal Morphological Reinflection

no code implementations CONLL 2018 Ryan Cotterell, Christo Kirov, John Sylak-Glassman, Géraldine Walther, Ekaterina Vylomova, Arya D. McCarthy, Katharina Kann, Sabrina J. Mielke, Garrett Nicolai, Miikka Silfverberg, David Yarowsky, Jason Eisner, Mans Hulden

Apart from extending the number of languages involved in earlier supervised tasks of generating inflected forms, this year the shared task also featured a new second task which asked participants to inflect words in sentential context, similar to a cloze task.

LEMMA Task 2

Inference of unobserved event streams with neural Hawkes particle smoothing

no code implementations27 Sep 2018 Hongyuan Mei, Guanghui Qin, Jason Eisner

Particle smoothing is an extension of particle filtering in which proposed events are conditioned on the future as well as the past.

Decoder

Are All Languages Equally Hard to Language-Model?

no code implementations NAACL 2018 Ryan Cotterell, Sabrina J. Mielke, Jason Eisner, Brian Roark

For general modeling methods applied to diverse languages, a natural question is: how well should we expect our models to work on languages with differing typological profiles?

Language Modelling

Neural Particle Smoothing for Sampling from Conditional Sequence Models

no code implementations NAACL 2018 Chu-Cheng Lin, Jason Eisner

We introduce neural particle smoothing, a sequential Monte Carlo method for sampling annotations of an input string from a given probability model.

On the Diachronic Stability of Irregularity in Inflectional Morphology

no code implementations23 Apr 2018 Ryan Cotterell, Christo Kirov, Mans Hulden, Jason Eisner

Many languages' inflectional morphological systems are replete with irregulars, i. e., words that do not seem to follow standard inflectional rules.

Relation

Spell Once, Summon Anywhere: A Two-Level Open-Vocabulary Language Model

1 code implementation23 Apr 2018 Sabrina J. Mielke, Jason Eisner

By invoking the second RNN to generate spellings for novel words in context, we obtain an open-vocabulary language model.

Language Modelling Sentence +1

Surface Statistics of an Unknown Language Indicate How to Parse It

no code implementations TACL 2018 Dingquan Wang, Jason Eisner

We show experimentally across multiple languages: (1) Features computed from the unparsed corpus improve parsing accuracy.

Dependency Parsing POS

Fine-Grained Prediction of Syntactic Typology: Discovering Latent Structure with Supervised Learning

no code implementations TACL 2017 Dingquan Wang, Jason Eisner

We show how to predict the basic word-order facts of a novel language given only a corpus of part-of-speech (POS) sequences.

POS

The Galactic Dependencies Treebanks: Getting More Data by Synthesizing New Languages

1 code implementation TACL 2016 Dingquan Wang, Jason Eisner

We release Galactic Dependencies 1. 0---a large set of synthetic languages not found on Earth, but annotated in Universal Dependencies format.

Diversity

Knowledge Tracing in Sequential Learning of Inflected Vocabulary

no code implementations CONLL 2017 Adithya Renduchintala, Philipp Koehn, Jason Eisner

We present a feature-rich knowledge tracing method that captures a student{'}s acquisition and retention of knowledge during a foreign language phrase learning task.

Knowledge Tracing Structured Prediction

Approximation-Aware Dependency Parsing by Belief Propagation

no code implementations TACL 2015 Matthew R. Gormley, Mark Dredze, Jason Eisner

We show how to adjust the model parameters to compensate for the errors introduced by this approximation, by following the gradient of the actual loss on training data.

Dependency Parsing

Modeling Word Forms Using Latent Underlying Morphs and Phonology

no code implementations TACL 2015 Ryan Cotterell, Nanyun Peng, Jason Eisner

Given some surface word types of a concatenative language along with the abstract morpheme sequences that they express, we show how to recover consistent underlying forms for these morphemes, together with the (stochastic) phonology that maps each concatenation of underlying forms to a surface form.

Learned Prioritization for Trading Off Accuracy and Speed

no code implementations NeurIPS 2012 Jiarong Jiang, Adam Teichert, Jason Eisner, Hal Daume

Users want natural language processing (NLP) systems to be both fast and accurate, but quality often comes at the cost of speed.

Imitation Learning Reinforcement Learning

Imitation Learning by Coaching

no code implementations NeurIPS 2012 He He, Jason Eisner, Hal Daume

However, it is important to note that these guarantees depend on how well the policy we found can imitate the oracle on the training data.

feature selection Imitation Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.