Search Results for author:
Found 14 papers, 10 papers with code
ChartMuseum: Testing Visual Reasoning Capabilities of Large Vision-Language Models
Unlike prior chart understanding benchmarks -- where frontier models perform similarly and near saturation -- our benchmark exposes a substantial gap between model and human performance, while effectively differentiating model capabilities: although humans achieve 93% accuracy, the best-performing model Gemini-2. 5-Pro attains only 63. 0%, and the leading open-source LVLM Qwen2. 5-VL-72B-Instruct achieves only 38. 5%.
KRISTEVA: Close Reading as a Novel Task for Benchmarking Interpretive Reasoning
Each year, tens of millions of essays are written and graded in college-level English courses.
RankAlign: A Ranking View of the Generator-Validator Gap in Large Language Models
Although large language models (LLMs) have become generally more capable and accurate across many tasks, some fundamental sources of unreliability remain in their behavior.
Parameterized Synthetic Text Generation with SimpleStories
We present SimpleStories, a large synthetic story dataset in simple language, consisting of 2 million samples each in English and Japanese.
Characterizing the Role of Similarity in the Property Inferences of Language Models
Property inheritance -- a phenomenon where novel properties are projected from higher level categories (e. g., birds) to lower level ones (e. g., sparrows) -- provides a unique window into how humans organize and deploy conceptual knowledge.
To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoning
Chain-of-thought (CoT) via prompting is the de facto method for eliciting reasoning capabilities from large language models (LLMs).
Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways
We pretrained our masked language models with three ingredients: an initial pretraining with music data, training on shorter sequences before training on longer ones, and masking specific tokens to target some of the BLiMP subtasks.
X-PARADE: Cross-Lingual Textual Entailment and Information Divergence across Paragraphs
Aligned paragraphs are sourced from Wikipedia pages in different languages, reflecting real information divergences observed in the wild.
WiCE: Real-World Entailment for Claims in Wikipedia
Textual entailment models are increasingly applied in settings like fact-checking, presupposition verification in question answering, or summary evaluation.
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Data augmentation is an important component in the robustness evaluation of models in natural language processing (NLP) and in enhancing the diversity of the data they are trained on.
Reusable Templates and Guides For Documenting Datasets and Models for Natural Language Processing and Generation: A Case Study of the HuggingFace and GEM Data and Model Cards
Developing documentation guidelines and easy-to-use templates for datasets and models is a challenging task, especially given the variety of backgrounds, skills, and incentives of the people involved in the building of natural language processing (NLP) tools.
The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics
We introduce GEM, a living benchmark for natural language Generation (NLG), its Evaluation, and Metrics.
Ranked #1 on
Extreme Summarization
on GEM-XSum
Abstractive Text Summarization
Cross-Lingual Abstractive Summarization
+5
Leveraging WordNet Paths for Neural Hypernym Prediction
We formulate the problem of hypernym prediction as a sequence generation task, where the sequences are taxonomy paths in WordNet.
Transfer Learning for Entity Recognition of Novel Classes
Our results empirically demonstrate when each of the published approaches tends to do well.
Entity Extraction using GAN
Named Entity Recognition (NER)
+2
