Search Results for author:
Found 6 papers, 5 papers with code
One Thousand and One Pairs: A "novel" challenge for long-context language models
Synthetic long-context LLM benchmarks (e. g., "needle-in-the-haystack") test only surface-level retrieval capabilities, but how well can long-context LLMs retrieve, synthesize, and reason over information across book-length inputs?
Iteratively Prompting Multimodal LLMs to Reproduce Natural and AI-Generated Images
With the digital imagery landscape rapidly evolving, image stocks and AI-generated image marketplaces have become central to visual media.
Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature
Using Par3, we discover that expert literary translators prefer reference human translations over machine-translated paragraphs at a rate of 84%, while state-of-the-art automatic MT metrics do not correlate with those preferences.
DEMETR: Diagnosing Evaluation Metrics for Translation
While machine translation evaluation metrics based on string overlap (e. g., BLEU) have their limitations, their computations are transparent: the BLEU score assigned to a particular candidate translation can be traced back to the presence or absence of certain words.
ChapterBreak: A Challenge Dataset for Long-Range Language Models
While numerous architectures for long-range language models (LRLMs) have recently been proposed, a meaningful evaluation of their discourse-level language understanding capabilities has not yet followed.
RELIC: Retrieving Evidence for Literary Claims
Humanities scholars commonly provide evidence for claims that they make about a work of literature (e. g., a novel) in the form of quotations from the work.
