Search Results for author:
Found 18 papers, 14 papers with code
Auditing Prompt Caching in Language Model APIs
We detect global cache sharing across users in seven API providers, including OpenAI, resulting in potential privacy leakage about users' prompts.
s1: Simple test-time scaling
After supervised finetuning the Qwen2. 5-32B-Instruct language model on s1K and equipping it with budget forcing, our model s1-32B exceeds o1-preview on competition math questions by up to 27% (MATH and AIME24).
Ranked #4 on
Mathematical Reasoning
on AIME24
AutoBencher: Creating Salient, Novel, Difficult Datasets for Language Models
In this paper, we present three desiderata for a good benchmark for language models: (i) salience (e. g., knowledge about World War II is more salient than a random day in history), (ii) novelty (i. e., the benchmark reveals new trends in model rankings not shown by previous benchmarks), and (iii) difficulty (i. e., the benchmark should be difficult for existing models, leaving headroom for future improvement).
Few-Shot Recalibration of Language Models
Recent work has uncovered promising ways to extract well-calibrated confidence estimates from language models (LMs), where the model's confidence score reflects how likely it is to be correct.
On the Learnability of Watermarks for Language Models
Watermarking of language model outputs enables statistical detection of model-generated text, which can mitigate harms and misuses of language models.
Benchmarking and Improving Generator-Validator Consistency of Language Models
To improve the consistency of LMs, we propose to finetune on the filtered generator and validator responses that are GV-consistent, and call this approach consistency fine-tuning.
Learning to Compress Prompts with Gist Tokens
Prompting is the primary way to utilize the multitask capabilities of language models (LMs), but prompts occupy valuable space in the input context window, and repeatedly encoding the same prompt is computationally inefficient.
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
Retrieval-augmented in-context learning has emerged as a powerful approach for addressing knowledge-intensive tasks using frozen language models (LM) and retrieval models (RM).
Evaluating Human-Language Model Interaction
To evaluate human-LM interaction, we develop a new framework, Human-AI Language-based Interaction Evaluation (HALIE), that defines the components of interactive systems and dimensions to consider when designing evaluation metrics.
Contrastive Decoding: Open-ended Text Generation as Optimization
We propose contrastive decoding (CD), a reliable decoding approach that optimizes a contrastive objective subject to a plausibility constraint.
Diffusion-LM Improves Controllable Text Generation
Controlling the behavior of language models (LMs) without re-training is a major open problem in natural language generation.
Ensembles and Cocktails: Robust Finetuning for Natural Language Generation
When finetuning a pretrained language model for natural language generation tasks, one is currently faced with a tradeoff.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Fine-tuning is the de facto way to leverage large pretrained language models to perform downstream tasks.
Decoding Methods for Neural Narrative Generation
Narrative generation is an open-ended NLP task in which a model generates a story given a prompt.
Posterior Control of Blackbox Generation
In this work, we consider augmenting neural generation models with discrete control states learned through a structured latent-variable approach.
Specializing Word Embeddings (for Parsing) by Information Bottleneck
Pre-trained word embeddings like ELMo and BERT contain rich syntactic and semantic information, resulting in state-of-the-art performance on various tasks.
A Generative Model for Punctuation in Dependency Trees
When the tree's yield is rendered as a written sentence, a string rewriting mechanism transduces the underlying marks into "surface" marks, which are part of the observed (surface) string but should not be regarded as part of the tree.
