Search Results for author:
Found 60 papers, 34 papers with code
Do Zombies Understand? A Choose-Your-Own-Adventure Exploration of Machine Cognition
Recent advances in LLMs have sparked a debate on whether they understand text.
Leveraging Collection-Wide Similarities for Unsupervised Document Structure Extraction
Document collections of various domains, e. g., legal, medical, or financial, often share some underlying collection-wide structure, which captures information that can aid both human users and structure-aware models.
K-QA: A Real-World Medical Q&A Benchmark
Ensuring the accuracy of responses provided by large language models (LLMs) is crucial, particularly in clinical settings where incorrect information may directly impact patient health.
State of What Art? A Call for Multi-Prompt LLM Evaluation
Recent advances in large language models (LLMs) have led to the development of various evaluation benchmarks.
Exploring the Impact of Training Data Distribution and Subword Tokenization on Gender Bias in Machine Translation
We study the effect of tokenization on gender bias in machine translation, an aspect that has been largely overlooked in previous works.
Instructed to Bias: Instruction-Tuned Language Models Exhibit Emergent Cognitive Bias
Recent studies show that instruction tuning (IT) and reinforcement learning from human feedback (RLHF) improve the abilities of large language models (LMs) dramatically.
Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents
Recent work has shown that infusing layout features into language models (LMs) improves processing of visually-rich documents such as scientific papers.
Comparing Humans and Models on a Similar Scale: Towards Cognitive Gender Bias Evaluation in Coreference Resolution
We approach this question through the lens of the dual-process theory for human decision-making.
Schema-Driven Information Extraction from Heterogeneous Tables
We use this collection of annotated tables to evaluate the ability of open-source and API-based language models to extract information from tables covering diverse domains and data formats.
Ranked #1 on
Attribute Extraction
on SWDE
The Perfect Victim: Computational Analysis of Judicial Attitudes towards Victims of Sexual Violence
Second, we curate a manually annotated dataset for judicial assessments of victim's credibility in the Hebrew language, as well as a model that can extract credibility labels from court cases.
Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images
We introduce WHOOPS!, a new dataset and benchmark for visual commonsense.
Ranked #1 on
Image-to-Text Retrieval
on WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images
(using extra training data)
Evaluating and Improving the Coreference Capabilities of Machine Translation Models
Machine translation (MT) requires a wide range of linguistic capabilities, which current end-to-end models are expected to learn implicitly by observing aligned sentences in bilingual corpora.
A Large-Scale Multilingual Study of Visual Constraints on Linguistic Selection of Descriptions
We study the relation between visual input and linguistic choices by training classifiers to predict the probability of expressing a property from raw images, and find evidence supporting the claim that linguistic properties are constrained by visual context across languages.
VASR: Visual Analogies of Situation Recognition
We leverage situation recognition annotations and the CLIP model to generate a large set of 500k candidate analogies.
Ranked #1 on
Visual Reasoning
on VASR
"Covid vaccine is against Covid but Oxford vaccine is made at Oxford!" Semantic Interpretation of Proper Noun Compounds
Proper noun compounds, e. g., "Covid vaccine", convey information in a succinct manner (a "Covid vaccine" is a "vaccine that immunizes against the Covid disease").
You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Models
Our method outperforms standard training methods in low-resource languages and retrains performance on high-resource languages while using the same amount of data.
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models
While vision-and-language models perform well on tasks such as visual question answering, they struggle when it comes to basic human commonsense reasoning skills.
Ranked #1 on
Common Sense Reasoning
on WinoGAViL
A Computational Acquisition Model for Multimodal Word Categorization
Recent advances in self-supervised modeling of text and images open new opportunities for computational models of child language acquisition, which is believed to rely heavily on cross-modal signals.
A Balanced Data Approach for Evaluating Cross-Lingual Transfer: Mapping the Linguistic Blood Bank
We show that the choice of pretraining languages affects downstream cross-lingual transfer for BERT-based models.
On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations
Recent work has shown that deep learning models in NLP are highly sensitive to low-level correlations between simple features and specific output labels, leading to overfitting and lack of generalization.
Automated Extraction of Sentencing Decisions from Court Cases in the Hebrew Language
We present the task of Automated Punishment Extraction (APE) in sentencing decisions from criminal court cases in Hebrew.
Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach
We present models which complete missing text given transliterations of ancient Mesopotamian documents, originally written on cuneiform clay tablets (2500 BCE - 100 CE).
Collecting a Large-Scale Gender Bias Dataset for Coreference Resolution and Machine Translation
We manually verify the quality of our corpus and use it to evaluate gender bias in various coreference resolution and machine translation models.
Data Efficient Masked Language Modeling for Vision and Language
We investigate a range of alternative masking strategies specific to the cross-modal setting that address these shortcomings, aiming for better fusion of text and image in the learned representation.
Realistic Evaluation Principles for Cross-document Coreference Resolution
We point out that common evaluation practices for cross-document coreference resolution have been unrealistically permissive in their assumed settings, yielding inflated results.
coreference-resolution
Cross Document Coreference Resolution
Cross-document Coreference Resolution over Predicted Mentions
Here, we introduce the first end-to-end model for CD coreference resolution from raw text, which extends the prominent model for within-document coreference to the CD setting.
coreference-resolution
Cross Document Coreference Resolution
Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA
Recent works have shown that supervised models often exploit data artifacts to achieve good test scores while their performance severely degrades on samples outside their training distribution.
Process-Level Representation of Scientific Protocols with Interactive Annotation
We develop Process Execution Graphs (PEG), a document-level representation of real-world wet lab biochemistry protocols, addressing challenges such as cross-sentence relations, long-range coreference, grounding, and implicit arguments.
GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation
While often assumed a gold standard, effective human evaluation of text generation remains an important, open area for research.
Gender Coreference and Bias Evaluation at WMT 2020
Our work presents the largest evidence for the phenomenon in more than 19 systems submitted to the WMT over four diverse target languages: Czech, German, Polish, and Russian.
MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics
Posing reading comprehension as a generation problem provides a great deal of flexibility, allowing for open-ended questions with few restrictions on possible answers.
Streamlining Cross-Document Coreference Resolution: Evaluation and Modeling
Recent evaluation protocols for Cross-document (CD) coreference resolution have often been inconsistent or lenient, leading to incomparable results across works and overestimation of performance.
coreference-resolution
Cross Document Coreference Resolution
+2
Active Learning for Coreference Resolution using Discrete Annotation
We improve upon pairwise annotation for active learning in coreference resolution, by asking annotators to identify mention antecedents if a presented mention pair is deemed not coreferent.
The Right Tool for the Job: Matching Model and Instance Complexities
Our method presents a favorable speed/accuracy tradeoff in almost all cases, producing models which are up to five times faster than the state of the art, while preserving their accuracy.
Ecological Semantics: Programming Environments for Situated Language Understanding
Large-scale natural language understanding (NLU) systems have made impressive progress: they can be applied flexibly across a variety of tasks, and employ minimal structural assumptions.
Controlled Crowdsourcing for High-Quality QA-SRL Annotation
Question-answer driven Semantic Role Labeling (QA-SRL) was proposed as an attractive open and natural flavour of SRL, potentially attainable from laymen.
Y'all should read this! Identifying Plurality in Second-Person Personal Pronouns in English Texts
Distinguishing between singular and plural {``}you{''} in English is a challenging task which has potential for downstream applications, such as machine translation or coreference resolution.
Evaluating Question Answering Evaluation
Our study suggests that while current metrics may be suitable for existing QA datasets, they limit the complexity of QA datasets that can be created.
Yall should read this! Identifying Plurality in Second-Person Personal Pronouns in English Texts
Distinguishing between singular and plural "you" in English is a challenging task which has potential for downstream applications, such as machine translation or coreference resolution.
On the Limits of Learning to Actively Learn Semantic Representations
We conclude that the current applicability of LTAL for improving data efficiency in learning semantic meaning representations is limited.
Learning Semantic Representations
Natural Language Understanding
Evaluating Gender Bias in Machine Translation
We present the first challenge set and evaluation protocol for the analysis of gender bias in machine translation (MT).
SemEval-2019 Task 10: Math Question Answering
Systems were evaluated based on the percentage of correctly answered questions.
DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs
We introduce a new English reading comprehension benchmark, DROP, which requires Discrete Reasoning Over the content of Paragraphs.
Ranked #14 on
Question Answering
on DROP Test
Semantics as a Foreign Language
We propose a novel approach to semantic dependency parsing (SDP) by casting the task as an instance of multi-lingual machine translation, where each semantic representation is a different foreign dialect.
Spot the Odd Man Out: Exploring the Associative Power of Lexical Resources
We propose Odd-Man-Out, a novel task which aims to test different properties of word representations.
Supervised Open Information Extraction
We present data and methods that enable a supervised learning approach to Open Information Extraction (Open IE).
Crowdsourcing Question-Answer Meaning Representations
We introduce Question-Answer Meaning Representations (QAMRs), which represent the predicate-argument structure of a sentence as a set of question-answer pairs.
Acquiring Predicate Paraphrases from News Tweets
We present a simple method for ever-growing extraction of predicate paraphrases from news headlines in Twitter.
Integrating Deep Linguistic Features in Factuality Prediction over Unified Datasets
Previous models for the assessment of commitment towards a predicate in a sentence (also known as factuality prediction) were trained and tested against a specific annotated dataset, subsequently limiting the generality of their results.
A Consolidated Open Knowledge Representation for Multiple Texts
We propose to move from Open Information Extraction (OIE) ahead to Open Knowledge Representation (OKR), aiming to represent information conveyed jointly in a set of texts in an open text-based manner.
Recognizing Mentions of Adverse Drug Reaction in Social Media Using Knowledge-Infused Recurrent Models
Recognizing mentions of Adverse Drug Reactions (ADR) in social media is challenging: ADR mentions are context-dependent and include long, varied and unconventional descriptions as compared to more formal medical symptom terminology.
Modeling Extractive Sentence Intersection via Subtree Entailment
Sentence intersection captures the semantic overlap of two texts, generalizing over paradigms such as textual entailment and semantic text similarity.
Abstractive Text Summarization
Natural Language Inference
+2
Getting More Out Of Syntax with PropS
Semantic NLP applications often rely on dependency trees to recognize major elements of the proposition structure of sentences.
Ranked #27 on
Open Information Extraction
on CaRB
