Search Results for author:
Found 195 papers, 100 papers with code
Is Large-Scale Pretraining the Secret to Good Domain Generalization?
We show that all evaluated DG methods struggle on DomainBed-OOP, while recent methods excel on DomainBed-IP.
KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models
This paper investigates visual analogical reasoning in large multimodal models (LMMs) compared to human adults and children.
Tell Me What's Next: Textual Foresight for Generic UI Representations
Mobile app user interfaces (UIs) are rich with action, text, structure, and image content that can be utilized to learn generic UI representations for tasks like automating user commands, summarizing content, and evaluating the accessibility of user interfaces.
SLANT: Spurious Logo ANalysis Toolkit
We uncover various seemingly harmless logos that VL models correlate 1) with negative human adjectives 2) with the concept of `harmlessness'; causing models to misclassify harmful online content as harmless, and 3) with user-provided object concepts; causing lower recognition accuracy on ImageNet zero-shot classification.
An Introduction to Vision-Language Modeling
Then, we present and discuss approaches to evaluate VLMs.
Concept Arithmetics for Circumventing Concept Inhibition in Diffusion Models
We use compositional property of diffusion models, which allows to leverage multiple prompts in a single image generation.
Koala: Key frame-conditioned long video-LLM
Long video question answering is a challenging task that involves recognizing short-term activities and reasoning about their fine-grained relationships.
Vision-LLMs Can Fool Themselves with Self-Generated Typographic Attacks
Furthermore, prior work's Typographic attacks against CLIP randomly sample a misleading class from a predefined set of categories.
SynCDR : Training Cross Domain Retrieval Models with Synthetic Data
In cross-domain retrieval, a model is required to identify images from the same semantic category across two visual domains.
CLAMP: Contrastive LAnguage Model Prompt-tuning
Our results show that LLMs can, indeed, achieve good image classification performance when adapted this way.
Learning to Compose SuperWeights for Neural Parameter Allocation Search
To address this, we generate layer weights by learning to compose sets of SuperWeights, which represent a group of trainable parameters.
Lasagna: Layered Score Distillation for Disentangled Object Relighting
Although generative editing methods now enable some forms of image editing, relighting is still beyond today's capabilities; existing methods struggle to keep other aspects of the image -- colors, shapes, and textures -- consistent after the edit.
Video Frame Interpolation with Many-to-many Splatting and Spatial Selective Refinement
In this work, we first propose a fully differentiable Many-to-Many (M2M) splatting framework to interpolate frames efficiently.
Socratis: Are large multimodal models emotionally aware?
We further see that current captioning metrics based on large vision-language models also fail to correlate with human preferences.
Label Budget Allocation in Multi-Task Learning
How should the label budget (i. e. the amount of money spent on labeling) be allocated among different tasks to achieve optimal multi-task performance?
From Fake to Real: Pretraining on Balanced Synthetic Images to Prevent Spurious Correlations in Image Recognition
By training on real and synthetic data separately, FFR does not expose the model to the statistical differences between real and synthetic data and thus avoids the issue of bias toward the pair $(B, G)$.
DualCoOp++: Fast and Effective Adaptation to Multi-Label Recognition with Limited Annotations
Previous works have focused on learning the alignment between textual and visual spaces to compensate for limited image labels, yet may suffer from reduced accuracy due to the scarcity of high-quality multi-label annotations.
Multiscale Video Pretraining for Long-Term Activity Forecasting
To alleviate this issue, we propose Multiscale Video Pretraining (MVP), a novel self-supervised pretraining approach that learns robust representations for forecasting by learning to predict contextualized representations of future video clips over multiple timescales.
Hardwiring ViT Patch Selectivity into CNNs using Patch Mixing
We hypothesize that this power to ignore out-of-context information (which we name $\textit{patch selectivity}$), while integrating in-context information in a non-local manner in early layers, allows ViTs to more easily handle occlusion.
WikiWeb2M: A Page-Level Multimodal Wikipedia Dataset
Webpages have been a rich resource for language and vision-language tasks.
A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding
Webpages have been a rich, scalable resource for vision-language and language only tasks.
ERM++: An Improved Baseline for Domain Generalization
We also explore the relationship between DG performance and similarity to pre-training data, and find that similarity to pre-training data distributions is an important driver of performance, but that ERM++ with stronger initializations can deliver strong performance even on dissimilar datasets. Code is released at https://github. com/piotr-teterwak/erm_plusplus.
DIME-FM: DIstilling Multimodal and Efficient Foundation Models
We transfer the knowledge from the pre-trained CLIP-ViTL/14 model to a ViT-B/32 model, with only 40M public images and 28. 4M unpaired public sentences.
Language-Guided Audio-Visual Source Separation via Trimodal Consistency
We propose a self-supervised approach for learning to perform audio source separation in videos based on natural language queries, using only unlabeled video and audio pairs as training data.
VisDA 2022 Challenge: Domain Adaptation for Industrial Waste Sorting
To test the abilities of computer vision models on this task, we present the VisDA 2022 Challenge on Domain Adaptation for Industrial Waste Sorting.
Mind the Backbone: Minimizing Backbone Distortion for Robust Object Detection
We propose to use Relative Gradient Norm (RGN) as a way to measure the vulnerability of a backbone to feature distortion, and show that high RGN is indeed correlated with lower OOD performance.
MaskSketch: Unpaired Structure-guided Masked Image Generation
We show that intermediate self-attention maps of a masked generative transformer encode important structural information of the input image, such as scene layout and object shape, and we propose a novel sampling method based on this observation to enable structure-guided generation.
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval
Existing methods rely on supervised learning of CIR models using labeled triplets consisting of the query image, text specification, and the target image.
Ranked #1 on
Zero-shot Image Retrieval
on ImageNet-R
Anchored Learning for On-the-Fly Adaptation -- Extended Technical Report
This study presents "anchor critics", a novel strategy for enhancing the robustness of reinforcement learning (RL) agents in crossing the sim-to-real gap.
DIME-FM : DIstilling Multimodal and Efficient Foundation Models
In this paper, we introduce a new distillation mechanism (DIME-FM) that allows us to transfer the knowledge contained in large VLFMs to smaller, customized foundation models using a relatively small amount of inexpensive, unpaired images and sentences.
Finding Differences Between Transformers and ConvNets Using Counterfactual Simulation Testing
One shortcoming of this is the fact that these deep neural networks cannot be easily evaluated for robustness issues with respect to specific scene variations.
Exploring Consistency in Cross-Domain Transformer for Domain Adaptive Semantic Segmentation
We propose to perform adaptation on attention maps with cross-domain attention layers that share features between the source and the target domains.
Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark
For example, asking a model to generate a varying number of the same object to measure its ability to count or providing a text prompt with several objects that each have a different attribute to identify its ability to match objects and attributes correctly.
Bias Mimicking: A Simple Sampling Approach for Bias Mitigation
Using this notion, BM, through a novel training procedure, ensures that the model is exposed to the entire distribution per epoch without repeating samples.
Ani-GIFs: A benchmark dataset for domain generalization of action recognition from GIFs
Deep learning models perform remarkably well for the same task under the assumption that data is always coming from the same distribution.
FETA: Towards Specializing Foundation Models for Expert Task Applications
However, as we show in this paper, FMs still have poor out-of-the-box performance on expert tasks (e. g. retrieval of car manuals technical illustrations from language queries), data for which is either unseen or belonging to a long-tail part of the data distribution of the huge datasets used for FM pre-training.
Ranked #1 on
Image-to-Text Retrieval
on FETA Car-Manuals
NewsStories: Illustrating articles with visual summaries
Thus, we explore a novel setting where the goal is to learn a self-supervised visual-language representation that is robust to varying text length and the number of images.
DualCoOp: Fast Adaptation to Multi-Label Recognition with Limited Annotations
Solving multi-label recognition (MLR) for images in the low-label regime is a challenging task with many real-world applications.
Prefix Conditioning Unifies Language and Label Supervision
In experiments, we show that this simple technique improves the performance in zero-shot image recognition accuracy and robustness to the image-level distribution shift.
Temporal Relevance Analysis for Video Action Models
In this paper, we provide a deep analysis of temporal modeling for action recognition, an important but underexplored problem in the literature.
Many-to-many Splatting for Efficient Video Frame Interpolation
Motion-based video frame interpolation commonly relies on optical flow to warp pixels from the inputs to the desired interpolation instant.
Ranked #1 on
Video Frame Interpolation
on Xiph-4K (Crop)
A Unified Framework for Domain Adaptive Pose Estimation
In this paper, we investigate the problem of domain adaptive 2D pose estimation that transfers knowledge learned on a synthetic source domain to a target domain without supervision.
A Broad Study of Pre-training for Domain Generalization and Adaptation
In this paper, we provide a broad study and in-depth analysis of pre-training for domain adaptation and generalization, namely: network architectures, size, pre-training loss, and datasets.
The Abduction of Sherlock Holmes: A Dataset for Visual Abductive Reasoning
We present Sherlock, an annotated corpus of 103K images for testing machine capacity for abductive reasoning beyond literal image contents.
A Dataset for Interactive Vision-Language Navigation with Unknown Command Feasibility
To study VLN with unknown command feasibility, we introduce a new dataset Mobile app Tasks with Iterative Feedback (MoTIF), where the goal is to complete a natural language command in a mobile app.
Explaining Reinforcement Learning Policies through Counterfactual Trajectories
In order for humans to confidently decide where to employ RL agents for real-world tasks, a human developer must validate that the agent will perform well at test-time.
Task2Sim: Towards Effective Pre-Training and Transfer From Synthetic Data
It is thus better to tailor synthetic pre-training data to a specific downstream task, for best performance.
Extending the WILDS Benchmark for Unsupervised Adaptation
Unlabeled data can be a powerful point of leverage for mitigating these distribution shifts, as it is frequently much more available than labeled data and can often be obtained from distributions beyond the source distribution as well.
Unsupervised Domain Generalization by Learning a Bridge Across Domains
The ability to generalize learned representations across significantly different visual domains, such as between real photos, clipart, paintings, and sketches, is a fundamental capacity of the human visual system.
Learning to Detect Every Thing in an Open World
LDET leads to significant improvements on many datasets in the open-world instance segmentation task, outperforming baselines on cross-category generalization on COCO, as well as cross-dataset evaluation on UVO and Cityscapes.
Look at What I’m Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos
Key to our approach is the ability to learn to spatially localize interactions with self-supervision on a large corpus of videos with accompanying transcribed narrations.
OpenMatch: Open-Set Semi-supervised Learning with Open-set Consistency Regularization
\ours achieves state-of-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.
Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Data
It is thus better to tailor synthetic pre-training data to a specific downstream task, for best performance.
Disentangled Unsupervised Image Translation via Restricted Information Flow
Unsupervised image-to-image translation methods aim to map images from one domain into plausible examples from another domain while preserving structures shared across two domains.
Contrast and Mix: Temporal Contrastive Video Domain Adaptation with Background Mixing
Unsupervised domain adaptation which aims to adapt models trained on a labeled source domain to a completely unlabeled target domain has attracted much attention in recent years.
Ranked #2 on
Unsupervised Domain Adaptation
on UCF-HMDB
Look at What I'm Doing: Self-Supervised Spatial Grounding of Narrations in Instructional Videos
Key to our approach is the ability to learn to spatially localize interactions with self-supervision on a large corpus of videos with accompanying transcribed narrations.
Multi-Critic Actor Learning: Teaching RL Policies to Act with Style
MultiCriticAL is tested in the context of multi-style learning, a special case of MTRL where agents are trained to behave with different distinct behavior styles, and yields up to 45% performance gains over the single-critic baselines and even successfully learns behavior styles in cases where single-critic approaches may simply fail to learn.
MixtureEnsembles: Leveraging Parameter Sharing for Efficient Ensembles
We improve on these methods with MixtureEnsembles, which learns to factorize ensemble members with shared parameters by constructing each layer with a linear combination of templates.
Pyramid Mini-Batching for Optimal Transport
Optimal transport theory provides a useful tool to measure the differences between two distributions.
Learning Cross-modal Contrastive Features for Video Domain Adaptation
Learning transferable and domain adaptive feature representations from videos is important for video-relevant tasks such as action recognition.
Tune it the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Density
Unsupervised domain adaptation (UDA) methods can dramatically improve generalization on unlabeled target domains.
Dynamic Network Quantization for Efficient Video Inference
Deep convolutional networks have recently achieved great success in video recognition, yet their practical realization remains a challenge due to the large amount of computational resources required to achieve robust recognition.
MetaPose: Fast 3D Pose from Multiple Views without 3D Supervision
In the era of deep learning, human pose estimation from multiple cameras with unknown calibration has received little attention to date.
Ranked #1 on
3D Human Pose Estimation
on SkiPose
VisDA-2021 Competition Universal Domain Adaptation to Improve Performance on Out-of-Distribution Data
Progress in machine learning is typically measured by training and testing a model on the same distribution of data, i. e., the same domain.
Separating Skills and Concepts for Novel Visual Question Answering
Generalization to out-of-distribution data has been a problem for Visual Question Answering (VQA) models.
ZeroWaste Dataset: Towards Deformable Object Segmentation in Cluttered Scenes
Recyclable waste detection poses a unique computer vision challenge as it requires detection of highly deformable and often translucent objects in cluttered scenes without the kind of context information usually present in human-centric datasets.
OpenMatch: Open-set Consistency Regularization for Semi-supervised Learning with Outliers
OpenMatch achieves state-of-the-art performance on three datasets, and even outperforms a fully supervised model in detecting outliers unseen in unlabeled data on CIFAR10.
AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognition
Specifically, given a video segment, a multi-modal policy network is used to decide what modalities should be used for processing by the recognition model, with the goal of improving both accuracy and efficiency.
Detector-Free Weakly Supervised Grounding by Separation
In this work, we focus on the task of Detector-Free WSG (DF-WSG) to solve WSG without relying on a pre-trained detector.
Ranked #1 on
Phrase Grounding
on Visual Genome
Mobile App Tasks with Iterative Feedback (MoTIF): Addressing Task Feasibility in Interactive Visual Environments
In recent years, vision-language research has shifted to study tasks which require more complex reasoning, such as interactive question answering, visual common sense reasoning, and question-answer plausibility prediction.
OVANet: One-vs-All Network for Universal Domain Adaptation
In this paper, we propose a method to learn the threshold using source samples and to adapt it to the target domain.
Ranked #6 on
Universal Domain Adaptation
on DomainNet
Evaluation of Correctness in Unsupervised Many-to-Many Image Translation
Given an input image from a source domain and a guidance image from a target domain, unsupervised many-to-many image-to-image (UMMI2I) translation methods seek to generate a plausible example from the target domain that preserves domain-invariant information of the input source image and inherits the domain-specific information from the guidance image.
Improved Techniques for Quantizing Deep Networks with Adaptive Bit-Widths
Second, to effectively transfer knowledge, we develop a dynamic block swapping method by randomly replacing the blocks in the lower-precision student network with the corresponding blocks in the higher-precision teacher network.
Honey, I Shrunk The Actor: A Case Study on Preserving Performance with Smaller Actors in Actor-Critic RL
Actors and critics in actor-critic reinforcement learning algorithms are functionally separate, yet they often use the same network architectures.
VA-RED$^2$: Video Adaptive Redundancy Reduction
An inherent property of real-world videos is the high correlation of information across frames which can translate into redundancy in either temporal or spatial feature maps of the models, or both.
AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognition
Temporal modelling is the key for efficient video action recognition.
Semi-Supervised Action Recognition with Temporal Contrastive Learning
We approach this problem by learning a two-pathway temporal contrastive model using unlabeled videos at two different speeds leveraging the fact that changing video speed does not change an action.
Surprisingly Simple Semi-Supervised Domain Adaptation with Pretraining and Consistency
With our Pretraining and Consistency (PAC) approach, we achieve state of the art target accuracy on this semi-supervised domain adaptation task, surpassing multiple adversarial domain alignment methods, across multiple datasets.
Semi-supervised Domain Adaptation
Unsupervised Domain Adaptation
CDS: Cross-Domain Self-Supervised Pre-Training
We present a two-stage pre-training approach that improves the generalization ability of standard single-domain pre-training.
Regularizing Action Policies for Smooth Control with Reinforcement Learning
A critical problem with the practical utility of controllers trained with deep Reinforcement Learning (RL) is the notable lack of smoothness in the actions learned by the RL policies.
Fine-grained Angular Contrastive Learning with Coarse Labels
A very practical example of C2FS is when the target classes are sub-classes of the training classes.
Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation
Partial domain adaptation which assumes that the unknown target label space is a subset of the source label space has attracted much attention in computer vision.
Ranked #1 on
Partial Domain Adaptation
on Office-31
Uncertainty-Aware Learning for Zero-Shot Semantic Segmentation
Recently, most ZSS methods focus on learning the visual-semantic correspondence to transfer knowledge from seen classes to unseen classes at the pixel level.
Learning from Lexical Perturbations for Consistent Visual Question Answering
Existing Visual Question Answering (VQA) models are often fragile and sensitive to input variations.
Temporal Action Detection with Multi-level Supervision
We extensively benchmark against the baselines for SSAD and OSAD on our created data splits in THUMOS14 and ActivityNet1. 2, and demonstrate the effectiveness of the proposed UFA and IB methods.
Auxiliary Task Reweighting for Minimum-data Learning
By adjusting the auxiliary task weights to minimize the divergence between the surrogate prior and the true prior of the main task, we obtain a more accurate prior estimation, achieving the goal of minimizing the required amount of training data for the main task and avoiding a costly grid search.
Active Domain Adaptation via Clustering Uncertainty-weighted Embeddings
Generalizing deep neural networks to new target domains is critical to their real-world utility.
Detecting Cross-Modal Inconsistency to Defend Against Neural Fake News
In addition to the valuable insights gleaned from our user study experiments, we provide a relatively effective approach based on detecting visual-semantic inconsistencies, which will serve as an effective first line of defense and a useful reference for future work in defending against machine-generated disinformation.
Self-supervised Visual Attribute Learning for Fashion Compatibility
Our approach consists of three self-supervised tasks designed to capture different concepts that are neglected in prior work that we can select from depending on the needs of our downstream tasks.
AR-Net: Adaptive Frame Resolution for Efficient Action Recognition
Specifically, given a video frame, a policy network is used to decide what input resolution should be used for processing by the action recognition model, with the goal of improving both accuracy and efficiency.
Domain2Vec: Domain Embedding for Unsupervised Domain Adaptation
Extensive experiments are conducted to demonstrate the power of our new datasets in benchmarking state-of-the-art multi-source domain adaptation methods, as well as the advantage of our proposed model.
COCO-FUNIT: Few-Shot Unsupervised Image Translation with a Content Conditioned Style Encoder
Unsupervised image-to-image translation intends to learn a mapping of an image in a given domain to an analogous image in a different domain, without explicit supervision of the mapping.
Real-time Semantic Segmentation with Fast Attention
The proposed architecture relies on our fast spatial attention, which is a simple yet efficient modification of the popular self-attention mechanism and captures the same rich spatial context at a small fraction of the computational cost, by changing the order of operations.
Ranked #32 on
Semantic Segmentation
on DensePASS
Neural Parameter Allocation Search
We introduce Neural Parameter Allocation Search (NPAS), a novel task where the goal is to train a neural network given an arbitrary, fixed parameter budget.
Black-box Explanation of Object Detectors via Saliency Maps
We propose D-RISE, a method for generating visual explanations for the predictions of object detectors.
Learning visual servo policies via planner cloning
Learning control policies for visual servoing in novel environments is an important problem.
Learning to Scale Multilingual Representations for Vision-Language Tasks
Current multilingual vision-language models either require a large number of additional parameters for each supported language, or suffer performance degradation as languages are added.
Spatio-Temporal Action Detection with Multi-Object Interaction
Spatio-temporal action detection in videos requires localizing the action both spatially and temporally in the form of an "action tube".
Revisiting Few-shot Activity Detection with Class Similarity Control
In this paper, we present a conceptually simple and general yet novel framework for few-shot temporal activity detection based on proposal regression which detects the start and end time of the activities in untrimmed videos.
Log-Likelihood Ratio Minimizing Flows: Towards Robust and Quantifiable Neural Distribution Alignment
We show that, under certain assumptions, this combination yields a deep neural likelihood-based minimization objective that attains a known lower bound upon convergence.
Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels
We show that when labeled source examples are limited, existing methods often fail to learn discriminative features applicable for both source and target domains.
Explainable Deep Classification Models for Domain Generalization
Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision.
Universal Domain Adaptation through Self Supervision
While some methods address target settings with either partial or open-set categories, they assume that the particular setting is known a priori.
A Broader Study of Cross-Domain Few-Shot Learning
Extensive experiments on the proposed benchmark are performed to evaluate state-of-art meta-learning approaches, transfer learning approaches, and newer methods for cross-domain few-shot learning.
Ranked #3 on
Cross-Domain Few-Shot
on Plantae
cross-domain few-shot learning
Few-Shot Image Classification
+1
AdaShare: Learning What To Share For Efficient Deep Multi-Task Learning
Multi-task learning is an open and challenging problem in computer vision.
Ranked #112 on
Semantic Segmentation
on NYU Depth v2
Federated Adversarial Domain Adaptation
In this work, we present a principled approach to the problem of federated domain adaptation, which aims to align the representations learned among the different nodes with the data distribution of the target node.
Class-imbalanced Domain Adaptation: An Empirical Odyssey
Unsupervised domain adaptation is a promising way to generalize deep models to novel domains.
LoGAN: Latent Graph Co-Attention Network for Weakly-Supervised Video Moment Retrieval
However, while such approaches tend to focus on identifying relationships between elements of the video and language modalities, there is less emphasis on modeling relational context between video frames given the semantic context of the query.
Generalized Domain Adaptation with Covariate and Label Shift CO-ALignment
In this paper, we explore the task of Generalized Domain Adaptation (GDA): How to transfer knowledge across different domains in the presence of both covariate and label shift?
wMAN: WEAKLY-SUPERVISED MOMENT ALIGNMENT NETWORK FOR TEXT-BASED VIDEO SEGMENT RETRIEVAL
Given a video and a sentence, the goal of weakly-supervised video moment retrieval is to locate the video segment which is described by the sentence without having access to temporal annotations during training.
MULE: Multimodal Universal Language Embedding
In this paper, we present a modular approach which can easily be incorporated into existing vision-language methods in order to support many languages.
Learning Similarity Conditions Without Explicit Supervision
Many real-world tasks require models to compare images along multiple similarity conditions (e. g. similarity in color, category or shape).
Language Features Matter: Effective Language Representations for Vision-Language Tasks
Shouldn't language and vision features be treated equally in vision-language (VL) tasks?
Adversarial Self-Defense for Cycle-Consistent GANs
The goal of unsupervised image-to-image translation is to map images from one domain to another without the ground truth correspondence between the two domains.
Weakly-supervised Compositional FeatureAggregation for Few-shot Recognition
Learning from a few examples is a challenging task for machine learning.
Two-Stream Region Convolutional 3D Network for Temporal Activity Detection
We address the problem of temporal activity detection in continuous, untrimmed video streams.
Ranked #4 on
Action Recognition
on THUMOS’14
Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation
The actual grounding can connect language to the environment through multiple modalities, e. g. "stop at the door" might ground into visual objects, while "turn right" might rely only on the geometric structure of a route.
Why do These Match? Explaining the Behavior of Image Similarity Models
Explaining a deep learning model can help users understand its behavior and allow researchers to discern its shortcomings.
Language-Conditioned Graph Networks for Relational Reasoning
E. g., conditioning on the "on" relationship to the plate, the object "mug" gathers messages from the object "plate" to update its representation to "mug on the plate", which can be easily consumed by a simple classifier for answer prediction.
Ranked #3 on
Referring Expression Comprehension
on CLEVR-Ref+
Domain Agnostic Learning with Disentangled Representations
Unsupervised model transfer has the potential to greatly improve the generalizability of deep models to novel domains.
Ranked #4 on
Multi-target Domain Adaptation
on DomainNet
Semi-supervised Domain Adaptation via Minimax Entropy
Contemporary domain adaptation methods are very effective at aligning feature distributions of source and target domains without any target supervision.
Cross-Domain Image Manipulation by Demonstration
In this work we propose a model that can manipulate individual visual attributes of objects in a real scene using examples of how respective attribute manipulations affect the output of a simulation.
Similarity R-C3D for Few-shot Temporal Activity Detection
In this paper, we present a conceptually simple and general yet novel framework for few-shot temporal activity detection which detects the start and end time of the few-shot input activities in an untrimmed video.
Strong-Weak Distribution Alignment for Adaptive Object Detection
This motivates us to propose a novel method for detector adaptation based on strong local alignment and weak global alignment.
Ranked #2 on
Unsupervised Domain Adaptation
on SIM10K to BDD100K
Guided Zoom: Questioning Network Evidence for Fine-grained Classification
We propose Guided Zoom, an approach that utilizes spatial grounding of a model's decision to make more informed predictions.
Moment Matching for Multi-Source Domain Adaptation
Conventional unsupervised domain adaptation (UDA) assumes that training data are sampled from a single domain.
TwoStreamVAN: Improving Motion Modeling in Video Generation
Video generation is an inherently challenging task, as it requires modeling realistic temporal dynamics as well as spatial content.
SPLAT: Semantic Pixel-Level Adaptation Transforms for Detection
Without dense labels, as is the case when only detection labels are available in the source, transformations are learned using CycleGAN alignment.
Revisiting Image-Language Networks for Open-ended Phrase Detection
Most existing work that grounds natural language phrases in images starts with the assumption that the phrase in question is relevant to the image.
Toward Driving Scene Understanding: A Dataset for Learning Driver Behavior and Causal Reasoning
We present the Honda Research Institute Driving Dataset (HDD), a challenging dataset to enable research on learning driver behavior in real-life environments.
Exploiting Environmental Variation to Improve Policy Robustness in Reinforcement Learning
We propose a novel method to exploit this observation to develop robust actor policies, by automatically developing a sampling curriculum over environment settings to use in training.
Object Hallucination in Image Captioning
Despite continuously improving performance, contemporary image captioning models are prone to "hallucinating" objects that are not actually in a scene.
Adapting control policies from simulation to reality using a pairwise loss
This paper proposes an approach to domain transfer based on a pairwise loss function that helps transfer control policies learned in simulation onto a real robot.
Explainable Neural Computation via Stack Neural Module Networks
In complex inferential tasks like question answering, machine learning models must confront two challenges: the need to implement a compositional reasoning process, and, in many applications, the need for this reasoning process to be interpretable to assist users in both development and prediction.
Ranked #14 on
Referring Expression Comprehension
on Talk2Car
Women also Snowboard: Overcoming Bias in Captioning Models (Extended Abstract)
Most machine learning methods are known to capture and exploit biases of the training data.
Syn2Real: A New Benchmark forSynthetic-to-Real Visual Domain Adaptation
In this paper, we present a new large-scale benchmark called Syn2Real, which consists of a synthetic domain rendered from 3D object models and two real-image domains containing the same object categories.
RISE: Randomized Input Sampling for Explanation of Black-box Models
We compare our approach to state-of-the-art importance extraction methods using both an automatic deletion/insertion metric and a pointing metric based on human-annotated object segments.
Explainable Artificial Intelligence (XAI)
Feature Importance
+5
Unsupervised Video-to-Video Translation
Unsupervised image-to-image translation is a recently proposed task of translating an image to a different style or domain given only unpaired image examples at training time.
Speaker-Follower Models for Vision-and-Language Navigation
We use this speaker model to (1) synthesize new instructions for data augmentation and to (2) implement pragmatic reasoning, which evaluates how well candidate action sequences explain an instruction.
Hierarchical Reinforcement Learning with Hindsight
Reinforcement Learning (RL) algorithms can suffer from poor sample efficiency when rewards are delayed and sparse.
Hierarchical Reinforcement Learning
reinforcement-learning
+2
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
To capture the inherent structures present in both text and video, we introduce a multilevel model that integrates vision and language features earlier and more tightly than prior work.
Women also Snowboard: Overcoming Bias in Captioning Models
We introduce a new Equalizer model that ensures equal gender probability when gender evidence is occluded in a scene and confident predictions when gender evidence is present.
Joint Event Detection and Description in Continuous Video Streams
In order to explicitly model temporal relationships between visual events and their captions in a single video, we also propose a two-level hierarchical captioning module that keeps track of context.
Contextual Multi-Scale Region Convolutional 3D Network for Activity Detection
In this paper, we propose the contextual multi-scale region convolutional 3D network (CMS-RC3D) for activity detection.
Learning Multi-Level Hierarchies with Hindsight
Hierarchical agents have the potential to solve sequential decision making tasks with greater sample efficiency than their non-hierarchical counterparts because hierarchical agents can break down tasks into sets of subtasks that only require short sequences of decisions.
CyCADA: Cycle-Consistent Adversarial Domain Adaptation
Domain adaptation is critical for success in new, unseen environments.
Adversarial Dropout Regularization
However, a drawback of this approach is that the critic simply labels the generated features as in-domain or not, without considering the boundaries between classes.
Ranked #2 on
Synthetic-to-Real Translation
on Syn2Real-C
VisDA: The Visual Domain Adaptation Challenge
We present the 2017 Visual Domain Adaptation (VisDA) dataset and challenge, a large-scale testbed for unsupervised domain adaptation across visual domains.
Stable Distribution Alignment Using the Dual of the Adversarial Distance
Our empirical results suggest that using the dual formulation for the restricted family of linear discriminators results in a more stable convergence to a desirable solution when compared with the performance of a primal min-max GAN-like objective and an MMD objective under the same restrictions.
Grasp Pose Detection in Point Clouds
Many grasp detection methods achieve grasp success rates (grasp successes as a fraction of the total number of grasp attempts) between 75% and 95% for novel objects presented in isolation or in light clutter.
Robotics
Learning a visuomotor controller for real world robotic grasping using simulated depth images
This paper proposes an approach to learning a closed-loop controller for robotic grasping that dynamically guides the gripper to the object.
Learning to Reason: End-to-End Module Networks for Visual Question Answering
Natural language questions are inherently compositional, and many are most easily answered by reasoning about their decomposition into modular sub-problems.
Ranked #44 on
Visual Question Answering (VQA)
on VQA v2 test-dev
R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
We address the problem of activity detection in continuous, untrimmed video streams.
Ranked #1 on
Action Recognition In Videos
on THUMOS’14
Adversarial Discriminative Domain Adaptation
Adversarial learning methods are a promising approach to training robust deep networks, and can generate complex samples across diverse domains.
Ranked #3 on
Unsupervised Image-To-Image Translation
on S