Search Results for author:
Found 33 papers, 20 papers with code
Conditional Poisson Stochastic Beams
Beam search is the default decoding strategy for many sequence generation tasks in NLP.
Efficient Sampling of Dependency Structure
In this paper, we adapt two spanning tree sampling algorithms to faithfully sample dependency trees from a graph subject to the root constraint.
From Language Models over Tokens to Language Models over Characters
Modern language models are internally -- and mathematically -- distributions over token strings rather than \emph{character} strings, posing numerous challenges for programmers building user applications on top of them.
On the Proper Treatment of Tokenization in Psycholinguistics
The paper argues that token-level language models should be (approximately) marginalized into character-level language models before they are used in psycholinguistic studies to compute the surprisal of a region of interest; then, the marginalized character-level language model can be used to compute the surprisal of an arbitrary character substring, which we term a focal area, that the experimenter may wish to use as a predictor.
The Foundations of Tokenization: Statistical and Computational Concerns
The present paper contributes to addressing this theoretical gap by proposing a unified formal framework for representing and analyzing tokenizer models.
Variational Best-of-N Alignment
To the extent this fine-tuning is successful and we end up with a good approximation, we have reduced the inference cost by a factor of N. Our experiments on a controlled generation task suggest that while variational BoN is not as effective as BoN in aligning language models, it is close to BoN performance as vBoN appears more often on the Pareto frontier of reward and KL divergence compared to models trained with KL-constrained RL objective.
Direct Preference Optimization with an Offset
DPO, as originally formulated, relies on binary preference data and fine-tunes a language model to increase the likelihood of a preferred response over a dispreferred response.
An Exploration of Left-Corner Transformations
The left-corner transformation (Rosenkrantz and Lewis, 1970) is used to remove left recursion from context-free grammars, which is an important step towards making the grammar parsable top-down with simple techniques.
Efficient Algorithms for Recognizing Weighted Tree-Adjoining Languages
From these, we also immediately obtain stringsum and allsum algorithms for TAG, LIG, PAA, and EPDA.
Efficient Semiring-Weighted Earley Parsing
This paper provides a reference description, in the form of a deduction system, of Earley's (1970) context-free parsing algorithm with various speed-ups.
A Formal Perspective on Byte-Pair Encoding
Via submodular functions, we prove that the iterative greedy version is a $\frac{1}{{\sigma(\boldsymbol{\mu}^\star)}}(1-e^{-{\sigma(\boldsymbol{\mu}^\star)}})$-approximation of an optimal merge sequence, where ${\sigma(\boldsymbol{\mu}^\star)}$ is the total backward curvature with respect to the optimal merge sequence $\boldsymbol{\mu}^\star$.
Algorithms for Acyclic Weighted Finite-State Automata with Failure Arcs
The pathsum in ordinary acyclic WFSAs is efficiently computed by the backward algorithm in time $O(|E|)$, where $E$ is the set of transitions.
Algorithms for Weighted Pushdown Automata
Weighted pushdown automata (WPDAs) are at the core of many natural language processing tasks, like syntax-based statistical machine translation and transition-based dependency parsing.
On the Intersection of Context-Free and Regular Languages
It shows, by a simple construction, that the intersection of a context-free language and a regular language is itself context-free.
Exact Paired-Permutation Testing for Structured Test Statistics
However, practitioners rely on Monte Carlo approximation to perform this test due to a lack of a suitable exact algorithm.
Conditional Poisson Stochastic Beam Search
In this work, we propose a new method for turning beam search into a stochastic process: Conditional Poisson stochastic beam search.
Searching for More Efficient Dynamic Programs
To this end, we describe a set of program transformations, a simple metric for assessing the efficiency of a transformed program, and a heuristic search procedure to improve this metric.
Efficient Sampling of Dependency Structures
Colbourn (1996)'s sampling algorithm has a running time of $\mathcal{O}(N^3)$, which is often greater than the mean hitting time of a directed graph.
On Finding the K-best Non-projective Dependency Trees
Furthermore, we present a novel extension of the algorithm for decoding the K-best dependency trees of a graph which are subject to a root constraint.
Higher-order Derivatives of Weighted Finite-state Machines
In the case of second-order derivatives, our scheme runs in the optimal $\mathcal{O}(A^2 N^4)$ time where $A$ is the alphabet size and $N$ is the number of states.
On Finding the $K$-best Non-projective Dependency Trees
Furthermore, we present a novel extension of the algorithm for decoding the $K$-best dependency trees of a graph which are subject to a root constraint.
Evaluation of Logic Programs with Built-Ins and Aggregation: A Calculus for Bag Relations
We present a scheme for translating logic programs, which may use aggregation and arithmetic, into algebraic expressions that denote bag relations over ground terms of the Herbrand universe.
Programming Languages Symbolic Computation
Please Mind the Root: Decoding Arborescences for Dependency Parsing
The connection between dependency trees and spanning trees is exploited by the NLP community to train and to decode graph-based dependency parsers.
If beam search is the answer, what was the question?
This implies that the MAP objective alone does not express the properties we desire in text, which merits the question: if beam search is the answer, what was the question?
Efficient Computation of Expectations under Spanning Tree Distributions
We propose unified algorithms for the important cases of first-order expectations and second-order expectations in edge-factored, non-projective spanning-tree models.
Best-First Beam Search
Decoding for many NLP tasks requires an effective heuristic algorithm for approximating exact search since the problem of searching the full output space is often intractable, or impractical in many settings.
The Universal Decompositional Semantics Dataset and Decomp Toolkit
We present the Universal Decompositional Semantics (UDS) dataset (v1. 0), which is bundled with the Decomp toolkit (v0. 1).
Learning to Prune: Exploring the Frontier of Fast and Accurate Parsing
Pruning hypotheses during dynamic programming is commonly used to speed up inference in settings such as parsing.
Reasoning about Quantities in Natural Language
In order to address these quantitative reasoning problems we first develop a computational approach which we show to successfully recognize and normalize textual expressions of quantities.
