Search Results for author:
Found 52 papers, 28 papers with code
CornerNet-Lite: Efficient Keypoint Based Object Detection
Together these two variants address the two critical use cases in efficient object detection: improving efficiency without sacrificing accuracy, and improving accuracy at real-time efficiency.
Ranked #137 on
Object Detection
on COCO minival
Remember the Past: Distilling Datasets into Addressable Memories for Neural Networks
We propose an algorithm that compresses the critical information of a large dataset into compact addressable memories.
Vision-Language Dataset Distillation
In this work, we design the first vision-language dataset distillation method, building on the idea of trajectory matching.
ImageNet Large Scale Visual Recognition Challenge
The ImageNet Large Scale Visual Recognition Challenge is a benchmark in object category classification and detection on hundreds of object categories and millions of images.
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Machine learning models are known to perpetuate and even amplify the biases present in the data.
End-to-end Learning of Action Detection from Frame Glimpses in Videos
In this work we introduce a fully end-to-end approach for action detection in videos that learns to directly predict the temporal bounds of actions.
Ranked #9 on
Temporal Action Localization
on THUMOS’14
(mAP IOU@0.2 metric)
What Actions are Needed for Understanding Human Actions in Videos?
We present the many kinds of information that will be needed to achieve substantial gains in activity understanding: objects, verbs, intent, and sequential reasoning.
Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation
We design a simple but surprisingly effective visual recognition benchmark for studying bias mitigation.
Ranked #1 on
Out-of-Distribution Generalization
on UrbanCars
Fair Attribute Classification through Latent Space De-biasing
Fairness in visual recognition is becoming a prominent and critical topic of discussion as recognition systems are deployed at scale in the real world.
SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition
Understanding the spatial relations between objects in images is a surprisingly challenging task.
What's in a Question: Using Visual Questions as a Form of Supervision
For instance, the question "what is the breed of the dog?"
What's the Point: Semantic Segmentation with Point Supervision
The semantic image segmentation task presents a trade-off between test time accuracy and training-time annotation cost.
Multi-Query Video Retrieval
Retrieving target videos based on text descriptions is a task of great practical value and has received increasing attention over the past few years.
[Re] Don't Judge an Object by Its Context: Learning to Overcome Contextual Bias
The implementation of most (7 of 10) methods was straightforward, especially after we received additional details from the original authors.
HIVE: Evaluating the Human Interpretability of Visual Explanations
As AI technology is increasingly applied to high-impact, high-risk domains, there have been a number of new methods aimed at making AI models more human interpretable.
Point and Ask: Incorporating Pointing into Visual Question Answering
Concretely, we (1) introduce and motivate point-input questions as an extension of VQA, (2) define three novel classes of questions within this space, and (3) for each class, introduce both a benchmark dataset and a series of baseline models to handle its unique challenges.
Directional Bias Amplification
We introduce and analyze a new, decoupled metric for measuring bias amplification, $\text{BiasAmp}_{\rightarrow}$ (Directional Bias Amplification).
Towards Unique and Informative Captioning of Images
We find that modern captioning systems return higher likelihoods for incorrect distractor sentences compared to ground truth captions, and that evaluation metrics like SPICE can be 'topped' using simple captioning systems relying on object detectors.
Understanding and Evaluating Racial Biases in Image Captioning
Image captioning is an important task for benchmarking visual reasoning and for enabling accessibility for people with vision impairments.
Overlooked factors in concept-based explanations: Dataset choice, concept learnability, and human capability
Second, we find that concepts in the probe dataset are often less salient and harder to learn than the classes they claim to explain, calling into question the correctness of the explanations.
CARETS: A Consistency And Robustness Evaluative Test Suite for VQA
We introduce CARETS, a systematic test suite to measure consistency and robustness of modern VQA models through a series of six fine-grained capability tests.
SiRi: A Simple Selective Retraining Mechanism for Transformer-based Visual Grounding
Particularly, SiRi conveys a significant principle to the research of visual grounding, i. e., a better initialized vision-language encoder would help the model converge to a better local minimum, advancing the performance accordingly.
Overwriting Pretrained Bias with Finetuning Data
Transfer learning is beneficial by allowing the expressive features of models pretrained on large-scale datasets to be finetuned for the target task of smaller, more domain-specific datasets.
Efficient, Self-Supervised Human Pose Estimation with Inductive Prior Tuning
We (1) analyze the relationship between reconstruction quality and pose estimation accuracy, (2) develop a model pipeline that outperforms the baseline which inspired our work, using less than one-third the amount of training data, and (3) offer a new metric suitable for self-supervised settings that measures the consistency of predicted body part length proportions.
Predictive-Corrective Networks for Action Detection
While deep feature learning has revolutionized techniques for static-image understanding, the same does not quite hold for video processing.
Every Moment Counts: Dense Detailed Labeling of Actions in Complex Videos
Every moment counts in action recognition.
Ranked #7 on
Action Detection
on Multi-THUMOS
Learning to Learn from Noisy Web Videos
Our method uses Q-learning to learn a data labeling policy on a small labeled training dataset, and then uses this to automatically label noisy web data for new visual concepts.
Crowdsourcing in Computer Vision
Computer vision systems require large amounts of manually annotated data to properly learn challenging visual concepts.
Much Ado About Time: Exhaustive Annotation of Temporal Data
We conclude that the optimal strategy is to ask as many questions as possible in a HIT (up to 52 binary questions after watching a 30-second video clip in our experiments).
Joint calibration of Ensemble of Exemplar SVMs
We formulate joint calibration as a constrained optimization problem and devise an efficient optimization algorithm to find its global optimum.
Best of Both Worlds: Human-Machine Collaboration for Object Annotation
This paper brings together the latest advancements in object detection and in crowd engineering into a principled framework for accurately and efficiently localizing objects in images.
Human uncertainty makes classification more robust
We then show that, while contemporary classifiers fail to exhibit human-like uncertainty on their own, explicit training on our dataset closes this gap, supports improved generalization to increasingly out-of-training-distribution test datasets, and confers robustness to adversarial attacks.
Compositional Temporal Visual Grounding of Natural Language Event Descriptions
Temporal grounding entails establishing a correspondence between natural language event descriptions and their visual depictions.
Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy
Computer vision technology is being used by many but remains representative of only a few.
Take the Scenic Route: Improving Generalization in Vision-and-Language Navigation
In the Vision-and-Language Navigation (VLN) task, an agent with egocentric vision navigates to a destination given natural language instructions.
Evolving Graphical Planner: Contextual Global Planning for Vision-and-Language Navigation
The ability to perform effective planning is crucial for building an instruction-following agent.
A Technical and Normative Investigation of Social Bias Amplification
The conversation around the fairness of machine learning models is growing and evolving.
Towards Intersectionality in Machine Learning: Including More Identities, Handling Underrepresentation, and Performing Evaluation
In this work, we grapple with questions that arise along three stages of the machine learning pipeline when incorporating intersectionality as multiple demographic attributes: (1) which demographic attributes to include as dataset labels, (2) how to handle the progressively smaller size of subgroups during model training, and (3) how to move beyond existing evaluation metrics when benchmarking model fairness for more subgroups.
ELUDE: Generating interpretable explanations via a decomposition into labelled and unlabelled features
Specifically, we develop a novel explanation framework ELUDE (Explanation via Labelled and Unlabelled DEcomposition) that decomposes a model's prediction into two parts: one that is explainable through a linear combination of the semantic attributes, and another that is dependent on the set of uninterpretable features.
Gender Artifacts in Visual Datasets
Gender biases are known to exist within large-scale visual datasets and can be reflected or even amplified in downstream models.
Predicting Word Learning in Children from the Performance of Computer Vision Systems
The performance of the computer vision systems is correlated with human judgments of the concreteness of words, which are in turn a predictor of children's word learning, suggesting that these models are capturing the relationship between words and visual phenomena.
"Help Me Help the AI": Understanding How Explainability Can Support Human-AI Interaction
Despite the proliferation of explainable AI (XAI) methods, little is understood about end-users' explainability needs and behaviors around XAI explanations.
UFO: A unified method for controlling Understandability and Faithfulness Objectives in concept-based explanations for CNNs
In this work, we propose UFO, a unified method for controlling Understandability and Faithfulness Objectives in concept-based explanations.
Humans, AI, and Context: Understanding End-Users' Trust in a Real-World Computer Vision Application
Trust is an important factor in people's interactions with AI systems.
Art and the science of generative AI: A deeper dive
A new class of tools, colloquially called generative AI, can produce high-quality artistic media for visual arts, concept art, music, fiction, literature, video, and animation.
ICON$^2$: Reliably Benchmarking Predictive Inequity in Object Detection
As computer vision systems are being increasingly deployed at scale in high-stakes applications like autonomous driving, concerns about social bias in these systems are rising.
ImageNet-OOD: Deciphering Modern Out-of-Distribution Detection Algorithms
Through comprehensive experiments, we show that OOD detectors are more sensitive to covariate shift than to semantic shift, and the benefits of recent OOD detection algorithms on semantic shift detection is minimal.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Unseen Image Synthesis with Diffusion Models
While the current trend in the generative field is scaling up towards larger models and more training data for generalized domain representations, we go the opposite direction in this work by synthesizing unseen domain images without additional training.
DETER: Detecting Edited Regions for Deterring Generative Manipulations
While face swapping and attribute editing are performed on similar face regions such as eyes and nose, the inpainting operation can be performed on random image regions, removing the spurious correlations of previous datasets.
Analyzing the Roles of Language and Vision in Learning from Limited Data
Does language help make sense of the visual world?
