Search Results for author:
Found 36 papers, 27 papers with code
The Era of Semantic Decoding
We refer to these orchestrated interactions among semantic processors, optimizing and searching in semantic space, as semantic decoding algorithms.
Symbolic Autoencoding for Self-Supervised Sequence Learning
Traditional language models, adept at next-token prediction in text sequences, often struggle with transduction tasks between distinct symbolic systems, particularly when parallel data is scarce.
Evaluating Language Model Agency through Negotiations
We introduce an approach to evaluate language model (LM) agency using negotiation games.
A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia
Yet the mechanisms underlying this contextual grounding remain unknown, especially in situations where contextual information contradicts factual knowledge stored in the parameters, which LLMs also excel at recalling.
Fly-Swat or Cannon? Cost-Effective Language Model Choice via Meta-Modeling
For a wide variety of tasks, inputs can be phrased as natural language prompts for an LM, from whose output the solution can then be extracted.
Flows: Building Blocks of Reasoning and Collaborating AI
To support rapid and rigorous research, we introduce the aiFlows library embodying Flows.
Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning
In this work, we demonstrate that formal grammars can describe the output space for a much wider range of tasks and argue that GCD can serve as a unified framework for structured NLP tasks in general.
REFINER: Reasoning Feedback on Intermediate Representations
Language models (LMs) have recently shown remarkable performance on reasoning tasks by explicitly generating intermediate inferences, e. g., chain-of-thought prompting.
Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extraction
This work shows that useful data can be synthetically generated even for tasks that cannot be solved directly by LLMs: for problems with structured outputs, it is possible to prompt an LLM to perform the task in the reverse direction, by generating plausible input text for a target output structure.
Language Model Decoding as Likelihood-Utility Alignment
Specifically, by analyzing the correlation between the likelihood and the utility of predictions across a diverse set of tasks, we provide empirical evidence supporting the proposed taxonomy and a set of principles to structure reasoning when choosing a decoding algorithm.
Distribution inference risks: Identifying and mitigating sources of leakage
We identify three sources of leakage: (1) memorizing specific information about the $\mathbb{E}[Y|X]$ (expected label given the feature values) of interest to the adversary, (2) wrong inductive bias of the model, and (3) finiteness of the training data.
The Glass Ceiling of Automatic Evaluation in Natural Language Generation
Automatic evaluation metrics capable of replacing human judgments are critical to allowing fast development of new methods.
Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learning
Underspecification refers to the existence of multiple models that are indistinguishable in their in-domain accuracy, even though they differ in other desirable properties such as out-of-distribution (OOD) performance.
Descartes: Generating Short Descriptions of Wikipedia Articles
Wikipedia is one of the richest knowledge sources on the Web today.
On the Context-Free Ambiguity of Emoji
We find that with 30 annotations per emoji, 16 emojis (1. 2%) are completely unambiguous, whereas 55 emojis (4. 3%) are so ambiguous that their descriptions are indistinguishable from randomly chosen descriptions.
GenIE: Generative Information Extraction
Structured and grounded representation of text is typically formalized by closed information extraction, the problem of extracting an exhaustive set of (subject, relation, object) triplets that are consistent with a predefined set of entities and relations from a knowledge base schema.
Better than Average: Paired Evaluation of NLP Systems
Evaluation in NLP is usually done by comparing the scores of competing systems independently averaged over a common set of test instances.
Invariant Language Modeling
In particular, we adapt a game-theoretic formulation of IRM (IRM-games) to language models, where the invariance emerges from a specific training schedule in which all the environments compete to optimize their own environment-specific loss by updating subsets of the model in a round-robin fashion.
Laughing Heads: Can Transformers Detect What Makes a Sentence Funny?
We make progress in both respects by training and analyzing transformer-based humor recognition models on a recently introduced dataset consisting of minimal pairs of aligned sentences, one serious, the other humorous.
KLearn: Background Knowledge Inference from Summarization Data
The goal of text summarization is to compress documents to the relevant information while excluding background information already known to the receiver.
Experts and authorities receive disproportionate attention on Twitter during the COVID-19 crisis
Timely access to accurate information is crucial during the COVID-19 pandemic.
Social and Information Networks
A Ladder of Causal Distances
Causal discovery, the task of automatically constructing a causal model from data, is of major significance across the sciences.
On the Limitations of Cross-lingual Encoders as Exposed by Reference-Free Machine Translation Evaluation
We systematically investigate a range of metrics based on state-of-the-art cross-lingual semantic representations obtained with pretrained M-BERT and LASER.
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance
A robust evaluation metric has a profound impact on the development of text generation systems.
Studying Summarization Evaluation Metrics in the Appropriate Scoring Range
In summarization, automatic evaluation metrics are usually compared based on their ability to correlate with human judgments.
Objective Function Learning to Match Human Judgements for Optimization-Based Summarization
Supervised summarization systems usually rely on supervision at the sentence or n-gram level provided by automatic metrics like ROUGE, which act as noisy proxies for human judgments.
Concatenated Power Mean Word Embeddings as Universal Cross-Lingual Sentence Representations
Here, we generalize the concept of average word embeddings to power mean word embeddings.
Live Blog Corpus for Summarization
Live blogs are an increasingly popular news format to cover breaking news and live events in online journalism.
A Simple Theoretical Model of Importance for Summarization
Research on summarization has mainly been driven by empirical approaches, crafting systems to perform well on standard datasets with the notion of information Importance remaining latent.
Learning to Score System Summaries for Better Content Selection Evaluation.
The evaluation of summaries is a challenging but crucial task of the summarization field.
Supervised Learning of Automatic Pyramid for Optimization-Based Multi-Document Summarization
We present a new supervised framework that learns to estimate automatic Pyramid scores and uses them for optimization-based extractive multi-document summarization.
A Principled Framework for Evaluating Summarizers: Comparing Models of Summary Quality against Human Judgments
We present a new framework for evaluating extractive summarizers, which is based on a principled representation as optimization problem.
LSDSem 2017: Exploring Data Generation Methods for the Story Cloze Test
The Story Cloze test is a recent effort in providing a common test scenario for text understanding systems.
The Next Step for Multi-Document Summarization: A Heterogeneous Multi-Genre Corpus Built with a Novel Construction Approach
In a detailed analysis, we show that our new corpus is significantly different from the homogeneous corpora commonly used, and that it is heterogeneous along several dimensions.
A General Optimization Framework for Multi-Document Summarization Using Genetic Algorithms and Swarm Intelligence
Extracting summaries via integer linear programming and submodularity are popular and successful techniques in extractive multi-document summarization.
