Search Results for author:
Found 14 papers, 6 papers with code
Conditional Poisson Stochastic Beams
Beam search is the default decoding strategy for many sequence generation tasks in NLP.
Variational Best-of-N Alignment
To the extent this fine-tuning is successful and we end up with a good approximation, we have reduced the inference cost by a factor of N. Our experiments on a controlled generation task suggest that while variational BoN is not as effective as BoN in aligning language models, it is close to BoN performance as vBoN appears more often on the Pareto frontier of reward and KL divergence compared to models trained with KL-constrained RL objective.
The Role of $n$-gram Smoothing in the Age of Neural Networks
For nearly three decades, language models derived from the $n$-gram assumption held the state of the art on the task.
Direct Preference Optimization with an Offset
DPO, as originally formulated, relies on binary preference data and fine-tunes a language model to increase the likelihood of a preferred response over a dispreferred response.
Principled Gradient-based Markov Chain Monte Carlo for Text Generation
Recent papers have demonstrated the possibility of energy-based text generation by adapting gradient-based sampling algorithms, a paradigm of MCMC algorithms that promises fast convergence.
Assessing Large Language Models on Climate Information
As Large Language Models (LLMs) rise in popularity, it is necessary to assess their capability in critically relevant domains.
Which Spurious Correlations Impact Reasoning in NLI Models? A Visual Interactive Diagnosis through Data-Constrained Counterfactuals
We present a human-in-the-loop dashboard tailored to diagnosing potential spurious features that NLI models rely on for predictions.
Hexatagging: Projective Dependency Parsing as Tagging
We introduce a novel dependency parser, the hexatagger, that constructs dependency trees by tagging the words in a sentence with elements from a finite set of possible tags.
Structured Voronoi Sampling
In this paper, we take an important step toward building a principled approach for sampling from language models with gradient-based methods.
Linear-Time Modeling of Linguistic Structure: An Order-Theoretic Perspective
We show that these exhaustive comparisons can be avoided, and, moreover, the complexity of such tasks can be reduced to linear by casting the relation between tokens as a partial order over the string.
In-Context Probing: Toward Building Robust Classifiers via Probing Large Language Models
However, the effectiveness of in-context learning is dependent on the provided context, and the performance on a downstream task can vary considerably, depending on the instruction.
On Parsing as Tagging
There have been many proposals to reduce constituency parsing to tagging in the literature.
Naturalistic Causal Probing for Morpho-Syntax
Probing has become a go-to methodology for interpreting and analyzing deep neural models in natural language processing.
Conditional Poisson Stochastic Beam Search
In this work, we propose a new method for turning beam search into a stochastic process: Conditional Poisson stochastic beam search.
