Search Results for author:
Found 43 papers, 12 papers with code
JoIN: Joint GANs Inversion for Intrinsic Image Decomposition
In this work, we propose to solve ill-posed inverse imaging problems using a bank of Generative Adversarial Networks (GAN) as a prior and apply our method to the case of Intrinsic Image Decomposition for faces and materials.
One-Shot Stylization for Full-Body Human Images
Since body shape deformation is an essential component of an art character's style, we incorporate a novel skeleton deformation module to reshape the pose of the input person and modify the DiOr pose-guided person generator to be more robust to the rescaled poses falling outside the distribution of the realistic poses that the generator is originally trained on.
Robust Online Video Instance Segmentation with Track Queries
We propose a fully online transformer-based video instance segmentation model that performs comparably to top offline methods on the YouTube-VIS 2019 benchmark and considerably outperforms them on UVO and OVIS.
Ranked #10 on
Video Instance Segmentation
on OVIS validation
MultiStyleGAN: Multiple One-shot Image Stylizations using a Single GAN
Recent approaches for one-shot stylization such as JoJoGAN fine-tune a pre-trained StyleGAN2 generator on a single style reference image.
Transfer of Representations to Video Label Propagation: Implementation Factors Matter
This work studies feature representations for dense label propagation in video, with a focus on recently proposed methods that learn video correspondence using self-supervised signals such as colorization or temporal cycle consistency.
Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents
We show that the emergent communication can be grounded to the agent observations and the spatial structure of the 3D environment.
Multi-Object Tracking with Hallucinated and Unlabeled Videos
Next, to tackle harder tracking cases, we mine hard examples across an unlabeled pool of real videos with a tracker trained on our hallucinated video data.
Dressing in Order: Recurrent Person Image Generation for Pose Transfer, Virtual Try-on and Outfit Editing
We propose a flexible person generation framework called Dressing in Order (DiOr), which supports 2D pose transfer, virtual try-on, and several fashion editing tasks.
Ranked #1 on
Pose Transfer
on Deep-Fashion
GridToPix: Training Embodied Agents with Minimal Supervision
While deep reinforcement learning (RL) promises freedom from hand-labeled data, great successes, especially for Embodied AI, require significant work to create supervision via carefully shaped rewards.
Bridging the Imitation Gap by Adaptive Insubordination
However, we show that when the teaching agent makes decisions with access to privileged information that is unavailable to the student, this information is marginalized during imitation learning, resulting in an "imitation gap" and, potentially, poor results.
A Cordial Sync: Going Beyond Marginal Policies for Multi-Agent Embodied Tasks
Autonomous agents must learn to collaborate.
Memory-Efficient Incremental Learning Through Feature Adaptation
We assume that the model is updated incrementally for new classes as new data becomes available sequentially. This requires adapting the previously stored feature vectors to the updated feature space without having access to the corresponding original training images.
Contextual Translation Embedding for Visual Relationship Detection and Scene Graph Generation
The previous VTransE model maps entities and predicates into a low-dimensional embedding vector space where the predicate is interpreted as a translation vector between the embedded features of the bounding box regions of the subject and the object.
Two Body Problem: Collaborative Visual Task Completion
Collaboration is a necessary skill to perform tasks that are beyond one agent's capabilities.
Revisiting Image-Language Networks for Open-ended Phrase Detection
Most existing work that grounds natural language phrases in images starts with the assumption that the phrase in question is relevant to the image.
Out of the Box: Reasoning with Graph Convolution Nets for Factual Visual Question Answering
Given a question-image pair, deep network techniques have been employed to successively reduce the large set of facts until one of the two entities of the final remaining fact is predicted as the answer.
Two can play this Game: Visual Dialog with Discriminative Question Generation and Answering
In addition, for the first time on the visual dialog dataset, we assess the performance of a system asking questions, and demonstrate how visual dialog can be generated from discriminative question generation and question answering.
Ranked #7 on
Visual Dialog
on VisDial v0.9 val
Piggyback: Adapting a Single Network to Multiple Tasks by Learning to Mask Weights
This work presents a method for adapting a single, fixed deep neural network to multiple tasks without affecting performance on already learned tasks.
Conditional Image-Text Embedding Networks
This paper presents an approach for grounding phrases in images which jointly learns multiple text-conditioned embeddings in a single end-to-end model.
Diverse and Accurate Image Description Using a Variational Auto-Encoder with an Additive Gaussian Encoding Space
This paper explores image caption generation using conditional variational auto-encoders (CVAEs).
PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning
This paper presents a method for adding multiple tasks to a single deep neural network while avoiding catastrophic forgetting.
Ranked #5 on
Continual Learning
on CUBS (Fine-grained 6 Tasks)
Enhancing Video Summarization via Vision-Language Embedding
This paper addresses video summarization, or the problem of distilling a raw video into a shorter form while still capturing the original story.
Learning Two-Branch Neural Networks for Image-Text Matching Tasks
Image-language matching tasks have recently attracted a lot of attention in the computer vision field.
Recurrent Models for Situation Recognition
This work proposes Recurrent Neural Network (RNN) models to predict structured 'image situations' -- actions and noun entities fulfilling semantic roles related to the action.
Ranked #9 on
Situation Recognition
on imSitu
Grounded Situation Recognition
Human-Object Interaction Detection
+1
Phrase Localization and Visual Relationship Detection with Comprehensive Image-Language Cues
This paper presents a framework for localization or grounding of phrases in images using a large collection of linguistic and visual cues.
Combining Multiple Cues for Visual Madlibs Question Answering
This paper presents an approach for answering fill-in-the-blank multiple choice questions from the Visual Madlibs dataset.
Solving Visual Madlibs with Multiple Cues
This paper focuses on answering fill-in-the-blank style multiple choice questions from the Visual Madlibs dataset.
Learning Models for Actions and Person-Object Interactions with Transfer to Question Answering
This paper proposes deep convolutional network models that utilize local and global context to make human activity label predictions in still images, achieving state-of-the-art performance on two recent datasets with hundreds of labels each.
Ranked #6 on
Human-Object Interaction Detection
on HICO
General Classification
Human-Object Interaction Detection
+4
Adaptive Object Detection Using Adjacency and Zoom Prediction
Compared to methods based on fixed anchor locations, our approach naturally adapts to cases where object instances are sparse and small.
Learning Informative Edge Maps for Indoor Scene Layout Prediction
We learn to predict 'informative edge' probability maps using two recent methods that exploit local and global context, respectively: structured edge detection forests, and a fully convolutional network for pixelwise labeling.
Where to Buy It: Matching Street Clothing Photos in Online Shops
In this paper, we define a new task, Exact Street to Shop, where our goal is to match a real-world example of a garment item to the same item in an online shop.
Learning Deep Structure-Preserving Image-Text Embeddings
This paper proposes a method for learning joint embeddings of images and text using a two-branch neural network with multiple layers of linear projections followed by nonlinearities.
Ranked #14 on
Image Retrieval
on Flickr30K 1K test
Active Object Localization with Deep Reinforcement Learning
We present an active detection model for localizing objects in scenes.
Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models
The Flickr30k dataset has become a standard benchmark for sentence-based image description.
Ranked #16 on
Image Retrieval
on Flickr30K 1K test
Training Deeper Convolutional Networks with Deep Supervision
One of the most promising ways of improving the performance of deep convolutional neural networks is by increasing the number of convolutional layers.
Scene Parsing with Object Instances and Occlusion Ordering
This work proposes a method to interpret a scene by assigning a semantic label at every pixel and inferring the spatial extent of individual object instances together with their occlusion relationships.
Multi-scale Orderless Pooling of Deep Convolutional Activation Features
Deep convolutional neural networks (CNN) have shown their promise as a universal representation for recognition.
Learning Binary Codes for High-Dimensional Data Using Bilinear Projections
Recent advances in visual recognition indicate that to achieve good retrieval and classification accuracy on largescale datasets like ImageNet, extremely high-dimensional visual descriptors, e. g., Fisher Vectors, are needed.
Finding Things: Image Parsing with Regions and Per-Exemplar Detectors
This paper presents a system for image parsing, or labeling each pixel in an image with its semantic category, aimed at achieving broad coverage across hundreds of object categories, many of them sparsely sampled.
A Multi-View Embedding Space for Modeling Internet Images, Tags, and their Semantics
This paper investigates the problem of modeling Internet images and associated text or tags for tasks such as image-to-image search, tag-to-image search, and image-to-tag search (image annotation).
Angular Quantization-based Binary Codes for Fast Similarity Search
Such data typically arises in a large number of vision and text applications where counts or frequencies are used as features.
Locality-sensitive binary codes from shift-invariant kernels
This paper addresses the problem of designing binary codes for high-dimensional data such that vectors that are similar in the original space map to similar binary strings.
Near-minimax recursive density estimation on the binary hypercube
This paper describes a recursive estimation procedure for multivariate binary densities using orthogonal expansions.
