Search Results for author:
Found 30 papers, 21 papers with code
Crowdsourcing Beyond Annotation: Case Studies in Benchmark Data Collection
Even though it is such a fundamental tool in NLP, crowdsourcing use is largely guided by common practices and the personal experience of researchers.
Interpretable by Design Visual Question Answering
Model interpretability has long been a hard problem for the AI community especially in the multimodal setting, where vision and language need to be aligned and reasoned at the same time.
AmbiCoref: Evaluating Human and Model Sensitivity to Ambiguous Coreference
To this end, we construct AmbiCoref, a diagnostic corpus of minimal sentence pairs with ambiguous and unambiguous referents.
Language in a Bottle: Language Model Guided Concept Bottlenecks for Interpretable Image Classification
Overall, LaBo demonstrates that inherently interpretable models can be widely applied at similar, or better, performance than black box approaches.
Visualizing the Obvious: A Concreteness-based Ensemble Model for Noun Property Prediction
We propose to extract these properties from images and use them in an ensemble model, in order to complement the information that is extracted from language models.
Cascading Biases: Investigating the Effect of Heuristic Annotation Strategies on Data and Models
Cognitive psychologists have documented that humans use cognitive heuristics, or mental shortcuts, to make quick decisions while expending less effort.
Iconary: A Pictionary-Based Game for Testing Multimodal Communication with Drawings and Text
We investigate these challenges in the context of Iconary, a collaborative game of drawing and guessing based on Pictionary, that poses a novel challenge for the research community.
Induce, Edit, Retrieve:Language Grounded Multimodal Schema for Instructional Video Retrieval
Schemata are structured representations of complex tasks that can aid artificial intelligence by allowing models to break down complex tasks into intermediate steps.
Visual Goal-Step Inference using wikiHow
Understanding what sequence of steps are needed to complete a goal can help artificial intelligence systems reason about human activities.
Ranked #1 on
VGSI
on wikiHow-image
Visual Semantic Role Labeling for Video Understanding
We propose a new framework for understanding and representing related salient events in a video using visual semantic role labeling.
Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles
We evaluate performance on synthetic datasets, and four datasets built to penalize models that exploit known biases on textual entailment, visual question answering, and image recognition tasks.
What Does BERT with Vision Look At?
Pre-trained visually grounded language models such as ViLBERT, LXMERT, and UNITER have achieved significant performance improvement on vision-and-language tasks but what they learn during pre-training remains unclear.
RoboTHOR: An Open Simulation-to-Real Embodied AI Platform
We argue that interactive and embodied visual AI has reached a stage of development similar to visual recognition prior to the advent of these ecosystems.
Grounded Situation Recognition
We introduce Grounded Situation Recognition (GSR), a task that requires producing structured semantic summaries of images describing: the primary activity, entities engaged in the activity with their roles (e. g. agent, tool), and bounding-box groundings of entities.
Ranked #5 on
Situation Recognition
on imSitu
Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases
Our method has two stages: we (1) train a naive model that makes predictions exclusively based on dataset biases, and (2) train a robust model as part of an ensemble with the naive one in order to encourage it to focus on other patterns in the data that are more likely to generalize.
Ranked #5 on
Visual Question Answering (VQA)
on VQA-CP
VisualBERT: A Simple and Performant Baseline for Vision and Language
We propose VisualBERT, a simple and flexible framework for modeling a broad range of vision-and-language tasks.
Ranked #1 on
Visual Reasoning
on NLVR
Gender Bias in Contextualized Word Embeddings
In this paper, we quantify, analyze and mitigate gender bias exhibited in ELMo's contextualized word vectors.
Balanced Datasets Are Not Enough: Estimating and Mitigating Gender Bias in Deep Image Representations
In this work, we present a framework to measure and mitigate intrinsic biases with respect to protected variables --such as gender-- in visual recognition tasks.
QuAC: Question Answering in Context
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).
A Qualitative Comparison of CoQA, SQuAD 2.0 and QuAC
We compare three new datasets for question answering: SQuAD 2. 0, QuAC, and CoQA, along several of their new features: (1) unanswerable questions, (2) multi-turn interactions, and (3) abstractive answers.
Ranked #3 on
Question Answering
on CoQA
QuAC : Question Answering in Context
We present QuAC, a dataset for Question Answering in Context that contains 14K information-seeking QA dialogs (100K questions in total).
Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods
We introduce a new benchmark, WinoBias, for coreference resolution focused on gender bias.
Neural Motifs: Scene Graph Parsing with Global Context
We then introduce Stacked Motif Networks, a new architecture designed to capture higher order motifs in scene graphs that further improves over our strong baseline by an average 7. 1% relative gain.
Ranked #6 on
Panoptic Scene Graph Generation
on PSG Dataset
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Language is increasingly being used to define rich visual recognition problems with supporting image collections sourced from the web.
Neural AMR: Sequence-to-Sequence Models for Parsing and Generation
Sequence-to-sequence models have shown strong performance across a broad range of applications.
Ranked #6 on
AMR Parsing
on LDC2015E86
Commonly Uncommon: Semantic Sparsity in Situation Recognition
Semantic sparsity is a common challenge in structured visual classification problems; when the output space is complex, the vast majority of the possible predictions are rarely, if ever, seen in the training set.
Ranked #10 on
Grounded Situation Recognition
on SWiG
Situation Recognition: Visual Semantic Role Labeling for Image Understanding
This paper introduces situation recognition, the problem of producing a concise summary of the situation an image depicts including: (1) the main activity (e. g., clipping), (2) the participating actors, objects, substances, and locations (e. g., man, shears, sheep, wool, and field) and most importantly (3) the roles these participants play in the activity (e. g., the man is clipping, the shears are his tool, the wool is being clipped from the sheep, and the clipping is in a field).
Ranked #11 on
Grounded Situation Recognition
on SWiG
