Search Results for author:
Found 80 papers, 43 papers with code
Improving Personalized Search with Regularized Low-Rank Parameter Updates
To evaluate how well general knowledge is preserved in a finetuned representation, we introduce a metric that measures image retrieval accuracy based on captions generated by a vision language model (VLM).
Contrastive Flow Matching
We introduce Contrastive Flow Matching, an extension to the flow matching objective that explicitly enforces uniqueness across all conditional flows, enhancing condition separation.
Gaze-LLE: Gaze Target Estimation via Large-Scale Learned Encoders
We address the problem of gaze target estimation, which aims to predict where a person is looking in a scene.
Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors
Text-to-image diffusion models have impactful applications in art, design, and entertainment, yet these technologies also pose significant risks by enabling the creation and dissemination of misinformation.
EgoMimic: Scaling Imitation Learning via Egocentric Video
The scale and diversity of demonstration data required for imitation learning is a significant challenge.
Model merging with SVD to tie the Knots
We study this phenomenon and observe that the weights of LoRA finetuned models showcase a lower degree of alignment compared to their fully-finetuned counterparts.
We're Not Using Videos Effectively: An Updated Domain Adaptive Video Segmentation Baseline
While the vast majority of prior work has studied this as a frame-level Image-DAS problem, a few Video-DAS works have sought to additionally leverage the temporal signal present in adjacent frames.
AUGCAL: Improving Sim2Real Adaptation by Uncertainty Calibration on Augmented Synthetic Images
Synthetic data (SIM) drawn from simulators have emerged as a popular alternative for training models where acquiring annotated real-world images is difficult.
SkyScenes: A Synthetic Dataset for Aerial Scene Understanding
Real-world aerial scene understanding is limited by a lack of datasets that contain densely annotated images curated under a diverse set of conditions.
Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives
We present Ego-Exo4D, a diverse, large-scale multimodal multiview video dataset and benchmark challenge.
Window Attention is Bugged: How not to Interpolate Position Embeddings
To fix it, we introduce a simple absolute window position embedding strategy, which solves the bug outright in Hiera and allows us to increase both speed and performance of the model in ViTDet.
Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks
Battle of the Backbones (BoB) makes this choice easier by benchmarking a diverse suite of pretrained models, including vision-language models, those trained via self-supervised learning, and the Stable Diffusion backbone, across a diverse set of computer vision tasks ranging from classification to object detection to OOD generalization and more.
FACTS: First Amplify Correlations and Then Slice to Discover Bias
Computer vision datasets frequently contain spurious correlations between task-relevant labels and (easy to learn) latent task-irrelevant attributes (e. g. context).
Memory in Plain Sight: Surveying the Uncanny Resemblances of Associative Memories and Diffusion Models
The generative process of Diffusion Models (DMs) has recently set state-of-the-art on many AI generation benchmarks.
ICON$^2$: Reliably Benchmarking Predictive Inequity in Object Detection
As computer vision systems are being increasingly deployed at scale in high-stakes applications like autonomous driving, concerns about social bias in these systems are rising.
Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistles
Modern hierarchical vision transformers have added several vision-specific components in the pursuit of supervised classification performance.
Ranked #1 on
Image Classification
on iNaturalist 2019
(using extra training data)
LANCE: Stress-testing Visual Models by Generating Language-guided Counterfactual Images
We propose an automated algorithm to stress-test a trained visual model by generating language-guided counterfactual test images (LANCE).
ZipIt! Merging Models from Different Tasks without Training
While this works for models trained on the same task, we find that this fails to account for the differences in models trained on disjoint tasks.
Benchmarking Low-Shot Robustness to Natural Distribution Shifts
Robustness to natural distribution shifts has seen remarkable progress thanks to recent pre-training strategies combined with better fine-tuning methods.
Token Merging for Fast Stable Diffusion
In the process, we speed up image generation by up to 2x and reduce memory consumption by up to 5. 6x.
Synthetic-to-Real Domain Adaptation for Action Recognition: A Dataset and Baseline Performances
The dataset is composed of both real and synthetic videos from seven gesture classes, and is intended to support the study of synthetic-to-real domain shift for video-based action recognition.
Bridging the Sim2Real gap with CARE: Supervised Detection Adaptation with Conditional Alignment and Reweighting
Sim2Real domain adaptation (DA) research focuses on the constrained setting of adapting from a labeled synthetic source domain to an unlabeled or sparsely labeled real target domain.
Mitigating Bias in Visual Transformers via Targeted Alignment
We visualize the feature space of the transformer self-attention modules and discover that a significant portion of the bias is encoded in the query matrix.
PASTA: Proportional Amplitude Spectrum Training Augmentation for Syn-to-Real Domain Generalization
Synthetic data offers the promise of cheap and bountiful training data for settings where labeled real-world data is scarce.
Signed Binary Weight Networks
Efficient inference of Deep Neural Networks (DNNs) is essential to making AI ubiquitous.
Structure-Encoding Auxiliary Tasks for Improved Visual Representation in Vision-and-Language Navigation
In Vision-and-Language Navigation (VLN), researchers typically take an image encoder pre-trained on ImageNet without fine-tuning on the environments that the agent will be trained or tested on.
Token Merging: Your ViT But Faster
Off-the-shelf, ToMe can 2x the throughput of state-of-the-art ViT-L @ 512 and ViT-H @ 518 models on images and 2. 2x the throughput of ViT-L on video with only a 0. 2-0. 3% accuracy drop in each case.
Ranked #13 on
Efficient ViTs
on ImageNet-1K (with DeiT-S)
Hydra Attention: Efficient Attention with Many Heads
While transformers have begun to dominate many tasks in vision, applying them to large images is still computationally difficult.
ZSON: Zero-Shot Object-Goal Navigation using Multimodal Goal Embeddings
We present a scalable approach for learning open-world object-goal navigation (ObjectNav) -- the task of asking a virtual robot (agent) to find any instance of an object in an unexplored environment (e. g., "find a sink").
Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency
Inspired by the design of recent SSL approaches based on learning from partial image inputs generated via masking or cropping -- either by learning to predict the missing pixels, or learning representational invariances to such augmentations -- we propose PACMAC, a simple two-stage adaptation algorithm for self-supervised ViTs.
Can domain adaptation make object recognition work for everyone?
Despite the rapid progress in deep visual recognition, modern computer vision datasets significantly overrepresent the developed world and models trained on such datasets underperform on images from unseen geographies.
VisCUIT: Visual Auditor for Bias in CNN Image Classifier
CNN image classifiers are widely used, thanks to their efficiency and accuracy.
Concept Evolution in Deep Learning Training: A Unified Interpretation Framework and Discoveries
We present ConceptEvo, a unified interpretation framework for deep neural networks (DNNs) that reveals the inception and evolution of learned concepts during training.
UDIS: Unsupervised Discovery of Bias in Deep Visual Recognition Models
Deep learning models have been shown to learn spurious correlations from data that sometimes lead to systematic failures for certain subpopulations.
AUGCO: Augmentation Consistency-guided Self-training for Source-free Domain Adaptive Semantic Segmentation
Most modern approaches for domain adaptive semantic segmentation rely on continued access to source data during adaptation, which may be infeasible due to computational or privacy constraints.
RobustNav: Towards Benchmarking Robustness in Embodied Navigation
As an attempt towards assessing the robustness of embodied navigation agents, we propose RobustNav, a framework to quantify the performance of embodied navigation agents when exposed to a wide variety of visual - affecting RGB inputs - and dynamics - affecting transition dynamics - corruptions.
SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation
Many existing approaches for unsupervised domain adaptation (UDA) focus on adapting under only data distribution shift and offer limited success under additional cross-domain label distribution shift.
Ranked #18 on
Domain Adaptation
on Office-Home
Temporal Action Detection with Multi-level Supervision
We extensively benchmark against the baselines for SSAD and OSAD on our created data splits in THUMOS14 and ActivityNet1. 2, and demonstrate the effectiveness of the proposed UFA and IB methods.
Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings
Generalizing deep neural networks to new target domains is critical to their real-world utility.
Auxiliary Task Reweighting for Minimum-data Learning
By adjusting the auxiliary task weights to minimize the divergence between the surrogate prior and the true prior of the main task, we obtain a more accurate prior estimation, achieving the goal of minimizing the required amount of training data for the main task and avoiding a costly grid search.
Integrating Egocentric Localization for More Realistic Point-Goal Navigation Agents
This enables a seamless adaption to changing dynamics (a different robot or floor type) by simply re-calibrating the visual odometry model -- circumventing the expense of re-training of the navigation policy.
Ranked #5 on
Robot Navigation
on Habitat 2020 Point Nav test-std
Learning to Balance Specificity and Invariance for In and Out of Domain Generalization
For domain generalization, the goal is to learn from a set of source domains to produce a single model that will best generalize to an unseen target domain.
Ranked #35 on
Domain Generalization
on DomainNet
Likelihood Landscapes: A Unifying Principle Behind Many Adversarial Defenses
Convolutional Neural Networks have been shown to be vulnerable to adversarial examples, which are known to locate in subspaces close to where normal data lies but are not naturally occurring and of low probability.
TIDE: A General Toolbox for Identifying Object Detection Errors
We introduce TIDE, a framework and associated toolbox for analyzing the sources of error in object detection and instance segmentation algorithms.
Analyzing Visual Representations in Embodied Navigation Tasks
Recent advances in deep reinforcement learning require a large amount of training data and generally result in representations that are often over specialized to the target task.
Representation Learning Through Latent Canonicalizations
We seek to learn a representation on a large annotated data source that generalizes to a target domain using limited new supervision.
Insights on Visual Representations for Embodied Navigation Tasks
Surprisingly, we find that slight differences in task have no measurable effect on the visual representation for both SqueezeNet and ResNet architectures.
Instance adaptive adversarial training: Improved accuracy tradeoffs in neural nets
Adversarial training is by far the most successful strategy for improving robustness of neural networks to adversarial attacks.
Robust Learning with Jacobian Regularization
Design of reliable systems must guarantee stability against input perturbations.
SplitNet: Sim2Sim and Task2Task Transfer for Embodied Visual Navigation
We propose SplitNet, a method for decoupling visual perception and policy learning.
Predictive Inequity in Object Detection
In this work, we investigate whether state-of-the-art object detection systems have equitable predictive performance on pedestrians with different skin tones.
Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptation
In this paper, we present a new large-scale benchmark called Syn2Real, which consists of a synthetic domain rendered from 3D object models and two real-image domains containing the same object categories.
Algorithms and Theory for Multiple-Source Adaptation
This work includes a number of novel contributions for the multiple-source adaptation problem.
Label Efficient Learning of Transferable Representations acrosss Domains and Tasks
We propose a framework that learns a representation transferable across different domains and tasks in a data efficient manner.
Label Efficient Learning of Transferable Representations across Domains and Tasks
We propose a framework that learns a representation transferable across different domains and tasks in a label efficient manner.
Multiple-Source Adaptation for Regression Problems
We present a detailed theoretical analysis of the problem of multiple-source adaptation in the general stochastic scenario, extending known results that assume a single target labeling function.
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Domain adaptation is critical for success in new, unseen environments.
VisDA: The Visual Domain Adaptation Challenge
We present the 2017 Visual Domain Adaptation (VisDA) dataset and challenge, a large-scale testbed for unsupervised domain adaptation across visual domains.
Fine-grained Recognition in the Wild: A Multi-Task Domain Adaptation Approach
While fine-grained object recognition is an important problem in computer vision, current models are unlikely to accurately classify objects in the wild.
Inferring and Executing Programs for Visual Reasoning
Existing methods for visual reasoning attempt to directly map inputs to outputs using black-box architectures without explicitly modeling the underlying reasoning processes.
Ranked #5 on
Visual Question Answering (VQA)
on CLEVR-Humans
Adversarial Discriminative Domain Adaptation
Adversarial learning methods are a promising approach to training robust deep networks, and can generate complex samples across diverse domains.
FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation
In this paper, we introduce the first domain adaptive semantic segmentation method, proposing an unsupervised adversarial approach to pixel prediction problems.
Ranked #2 on
Image-to-Image Translation
on SYNTHIA Fall-to-Winter
Clockwork Convnets for Video Semantic Segmentation
Recent years have seen tremendous progress in still-image segmentation; however the na\"ive application of these state-of-the-art algorithms to every video frame requires considerable computation and ignores the temporal continuity inherent in video.
Learning With Side Information Through Modality Hallucination
Thus, our method transfers information commonly extracted from depth training data to a network which can extract that information from the RGB counterpart.
Fine-to-coarse Knowledge Transfer For Low-Res Image Classification
We address the difficult problem of distinguishing fine-grained object categories in low resolution images.
Adapting Deep Visuomotor Representations with Weak Pairwise Constraints
We propose a novel, more powerful combination of both distribution and pairwise image alignment, and remove the requirement for expensive annotation by using weakly aligned pairs of images in the source and target domains.
Quantification in-the-wild: data-sets and baselines
Quantification is the task of estimating the class-distribution of a data-set.
Spatial Semantic Regularisation for Large Scale Object Detection
Our approach proves to be especially useful in large scale settings with thousands of classes, where spatial and semantic interactions are very frequent and only weakly supervised detectors can be built due to a lack of bounding box annotations.
Simultaneous Deep Transfer Across Domains and Tasks
Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias.
Cross Modal Distillation for Supervision Transfer
In this work we propose a technique that transfers supervision between images from different modalities.
Deep Domain Confusion: Maximizing for Domain Invariance
Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias on a standard benchmark.
Ranked #6 on
Domain Adaptation
on Office-Caltech
Detector Discovery in the Wild: Joint Multiple Instance and Representation Learning
We develop methods for detector learning which exploit joint training over both weak and strong labels and which transfer learned perceptual representations from strongly-labeled auxiliary tasks.
LSDA: Large Scale Detection Through Adaptation
A major challenge in scaling object detection is the difficulty of obtaining labeled images for large numbers of categories.
Continuous Manifold Based Adaptation for Evolving Visual Domains
The classic domain adaptation paradigm considers the world to be separated into stationary domains with clear boundaries between them.
One-Shot Adaptation of Supervised Deep Convolutional Models
In other words, are deep CNNs trained on large amounts of labeled data as susceptible to dataset bias as previous methods have been shown to be?
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
We evaluate whether features extracted from the activation of a deep convolutional network trained in a fully supervised fashion on a large, fixed set of object recognition tasks can be re-purposed to novel generic tasks.
Towards Adapting ImageNet to Reality: Scalable Domain Adaptation with Implicit Low-rank Transformations
Images seen during test time are often not from the same distribution as images used for learning.
Semi-supervised Domain Adaptation with Instance Constraints
Most successful object classification and detection methods rely on classifiers trained on large labeled datasets.
Efficient Learning of Domain-invariant Image Representations
We present an algorithm that learns representations which explicitly compensate for domain mismatch and which can be efficiently realized as linear classifiers.
