Search Results for author:
Found 28 papers, 8 papers with code
CREPE: Can Vision-Language Foundation Models Reason Compositionally?
To measure systematicity, CREPE consists of three test datasets.
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Prior work has identified a resilient phenomenon that threatens the performance of human-AI decision-making teams: overreliance, when people agree with an AI, even when it is incorrect.
ELIGN: Expectation Alignment as a Multi-Agent Intrinsic Reward
These results identify tasks where expectation alignment is a more useful strategy than curiosity-driven exploration for multi-agent coordination, enabling agents to do zero-shot coordination.
Measuring Compositional Consistency for Video Question Answering
Recent video question answering benchmarks indicate that state-of-the-art models struggle to answer compositional questions.
AGQA 2.0: An Updated Benchmark for Compositional Spatio-Temporal Reasoning
Prior benchmarks have analyzed models' answers to questions about videos in order to measure visual compositional reasoning.
Visual Intelligence through Human Interaction
Over the last decade, Computer Vision, the branch of Artificial Intelligence aimed at understanding the visual world, has evolved from simply recognizing objects in images to describing pictures, answering questions about images, aiding robots maneuver around physical spaces and even generating novel visual content.
On the Opportunities and Risks of Foundation Models
AI is undergoing a paradigm shift with the rise of models (e. g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks.
Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering
Active learning promises to alleviate the massive data needs of supervised machine learning: it has successfully improved sample efficiency by an order of magnitude on traditional tasks like topic classification and object recognition.
AGQA: A Benchmark for Compositional Spatio-Temporal Reasoning
AGQA contains $192M$ unbalanced question answer pairs for $9. 6K$ videos.
Determining Question-Answer Plausibility in Crowdsourced Datasets Using Multi-Task Learning
Datasets extracted from social networks and online forums are often prone to the pitfalls of natural language, namely the presence of unstructured and noisy data.
Conceptual Metaphors Impact Perceptions of Human-AI Collaboration
In a third study, we assess effects of metaphor choices on potential users' desire to try out the system and find that users are drawn to systems that project higher competence and warmth.
Action Genome: Actions as Composition of Spatio-temporal Scene Graphs
Next, by decomposing and learning the temporal changes in visual relationships that result in an action, we demonstrate the utility of a hierarchical event decomposition by enabling few-shot action recognition, achieving 42. 7% mAP using as few as 10 examples.
Deep Bayesian Active Learning for Multiple Correct Outputs
The assumption that these tasks always have exactly one correct answer has resulted in the creation of numerous uncertainty-based measurements, such as entropy and least confidence, which operate over a model's outputs.
Learning Predicates as Functions to Enable Few-shot Scene Graph Prediction
We introduce the first scene graph prediction model that supports few-shot learning of predicates.
Scene Graph Prediction with Limited Labels
All scene graph models to date are limited to training on a small set of visual relationships that have thousands of training labels each.
Ranked #1 on
Scene Graph Detection
on VRD
HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Models
We construct Human eYe Perceptual Evaluation (HYPE) a human benchmark that is (1) grounded in psychophysics research in perception, (2) reliable across different sets of randomly sampled outputs from a model, (3) able to produce separable model performances, and (4) efficient in cost and time.
Information Maximizing Visual Question Generation
We build a model that maximizes mutual information between the image, the expected answer and the generated question.
HYPE: Human-eYe Perceptual Evaluation of Generative Models
The second, HYPE-Infinity, measures human error rate on fake and real images with no time constraints, maintaining stability and drastically reducing time and cost.
The ActivityNet Large-Scale Activity Recognition Challenge 2018 Summary
The guest tasks focused on complementary aspects of the activity recognition problem at large scale and involved three challenging and recently compiled datasets: the Kinetics-600 dataset from Google DeepMind, the AVA dataset from Berkeley and Google, and the Moments in Time dataset from MIT and IBM Research.
Referring Relationships
We formulate the cyclic condition between the entities in a relationship by modelling predicates that connect the entities as shifts in attention from one entity to another.
Dense-Captioning Events in Videos
We also introduce ActivityNet Captions, a large-scale benchmark for dense-captioning events.
A Hierarchical Approach for Generating Descriptive Image Paragraphs
Recent progress on image captioning has made it possible to generate novel sentences describing images in natural language, but compressing an image into a single sentence can describe visual content in only coarse detail.
A Glimpse Far into the Future: Understanding Long-term Crowd Worker Quality
Microtask crowdsourcing is increasingly critical to the creation of extremely large datasets.
Visual Relationship Detection with Language Priors
We improve on prior work by leveraging language priors from semantic word embeddings to finetune the likelihood of a predicted relationship.
Ranked #2 on
Scene Graph Generation
on VRD
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Despite progress in perceptual tasks such as image classification, computers still perform poorly on cognitive tasks such as image description and question answering.
Embracing Error to Enable Rapid Crowdsourcing
Microtask crowdsourcing has enabled dataset advances in social science and machine learning, but existing crowdsourcing schemes are too expensive to scale up with the expanding volume of data.
Image Retrieval Using Scene Graphs
We introduce a novel dataset of 5, 000 human-generated scene graphs grounded to images and use this dataset to evaluate our method for image retrieval.
