Search Results for author:
Found 28 papers, 13 papers with code
Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers
We present Region-aware Open-vocabulary Vision Transformers (RO-ViT) - a contrastive image-text pretraining recipe to bridge the gap between image-level pretraining and open-vocabulary object detection.
Ranked #5 on
Zero-Shot Cross-Modal Retrieval
on Flickr30k
Detection-Oriented Image-Text Pretraining for Open-Vocabulary Detection
We present a new open-vocabulary detection approach based on detection-oriented image-text pretraining to bridge the gap between image-level pretraining and open-vocabulary object detection.
Ranked #1 on
Open Vocabulary Object Detection
on LVIS v1.0
DeepLab2: A TensorFlow Library for Deep Labeling
DeepLab2 is a TensorFlow library for deep labeling, aiming to provide a state-of-the-art and easy-to-use TensorFlow codebase for general dense pixel prediction problems in computer vision.
Deep Video Inpainting
Video inpainting aims to fill spatio-temporal holes with plausible content in a video.
Ranked #7 on
Video Inpainting
on DAVIS
Video Panoptic Segmentation
In this paper, we propose and explore a new video extension of this task, called video panoptic segmentation.
Ranked #7 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
Learning Open-World Object Proposals without Learning to Classify
In this paper, we identify that the problem is that the binary classifiers in existing proposal methods tend to overfit to the training categories.
Ranked #2 on
Open World Object Detection
on COCO VOC to non-VOC
Neural Human Performer: Learning Generalizable Radiance Fields for Human Performance Rendering
To tackle this, we propose Neural Human Performer, a novel approach that learns generalizable neural radiance fields based on a parametric human body model for robust performance capture.
Ranked #3 on
Generalizable Novel View Synthesis
on ZJU-MoCap
CMT-DeepLab: Clustering Mask Transformers for Panoptic Segmentation
We propose Clustering Mask Transformer (CMT-DeepLab), a transformer-based framework for panoptic segmentation designed around clustering.
Ranked #6 on
Panoptic Segmentation
on COCO test-dev
Rotationally-Temporally Consistent Novel View Synthesis of Human Performance Video
Second, we introduce a novel loss to explicitly enforce consistency across generated views both in space and in time.
Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence
Blind video decaptioning is a problem of automatically removing text overlays and inpainting the occluded parts in videos without any input masks.
LinkNet: Relational Embedding for Scene Graph
In this paper, we present a method that improves scene graph generation by explicitly modeling inter-dependency among the entire object instances.
Many-to-Many Spoken Language Translation via Unified Speech and Text Representation Learning with Unit-to-Unit Translation
A single pre-trained model with UTUT can be employed for diverse multilingual speech- and text-related tasks, such as Speech-to-Speech Translation (STS), multilingual Text-to-Speech Synthesis (TTS), and Text-to-Speech Translation (TTST).
Discriminative Feature Learning for Unsupervised Video Summarization
The proposed variance loss allows a network to predict output scores for each frame with high discrepancy which enables effective feature learning and significantly improves model performance.
Ranked #3 on
Unsupervised Video Summarization
on SumMe
Supervised Video Summarization
Unsupervised Video Summarization
Learning Image Representations by Completing Damaged Jigsaw Puzzles
The recovery of the aforementioned damage pushes the network to obtain robust and general-purpose representations.
Two-Phase Learning for Weakly Supervised Object Localization
Weakly supervised semantic segmentation and localiza- tion have a problem of focusing only on the most important parts of an image since they use only image-level annota- tions.
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
Self-supervised tasks such as colorization, inpainting and zigsaw puzzle have been utilized for visual representation learning for still images, when the number of labeled images is limited or absent at all.
Ranked #42 on
Self-Supervised Action Recognition
on HMDB51
Align-and-Attend Network for Globally and Locally Coherent Video Inpainting
Our video inpainting network consists of two stages.
Preserving Semantic and Temporal Consistency for Unpaired Video-to-Video Translation
In this paper, we investigate the problem of unpaired video-to-video translation.
Hide-and-Tell: Learning to Bridge Photo Streams for Visual Storytelling
In this paper, we propose to explicitly learn to imagine a storyline that bridges the visual gap.
Ranked #7 on
Visual Storytelling
on VIST
The Devil is in the Boundary: Exploiting Boundary Representation for Basis-based Instance Segmentation
In this work, we propose Boundary Basis based Instance Segmentation(B2Inst) to learn a global boundary representation that can complement existing global-mask-based methods that are often lacking high-frequency details.
Learning to Associate Every Segment for Video Panoptic Segmentation
Temporal correspondence - linking pixels or objects across frames - is a fundamental supervisory signal for the video models.
Ranked #6 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
TubeFormer-DeepLab: Video Mask Transformer
We present TubeFormer-DeepLab, the first attempt to tackle multiple core video segmentation tasks in a unified manner.
MaMMUT: A Simple Architecture for Joint Learning for MultiModal Tasks
We propose a novel paradigm of training with a decoder-only model for multimodal tasks, which is surprisingly effective in jointly learning of these disparate vision-language tasks.
Video-kMaX: A Simple Unified Approach for Online and Near-Online Video Panoptic Segmentation
The meta architecture of the proposed Video-kMaX consists of two components: within clip segmenter (for clip-level segmentation) and cross-clip associater (for association beyond clips).
Neural Image-based Avatars: Generalizable Radiance Fields for Human Avatar Modeling
We present a method that enables synthesizing novel views and novel poses of arbitrary human performers from sparse multi-view images.
RECLIP: Resource-efficient CLIP by Training with Small Images
We present RECLIP (Resource-efficient CLIP), a simple method that minimizes computational resource footprint for CLIP (Contrastive Language Image Pretraining).
Contrastive Feature Masking Open-Vocabulary Vision Transformer
We present Contrastive Feature Masking Vision Transformer (CFM-ViT) - an image-text pretraining methodology that achieves simultaneous learning of image- and region-level representation for open-vocabulary object detection (OVD).
Ranked #5 on
Open Vocabulary Object Detection
on LVIS v1.0
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities
We propose a multimodal model, called Mirasol3B, consisting of an autoregressive component for the time-synchronized modalities (audio and video), and an autoregressive component for the context modalities which are not necessarily aligned in time but are still sequential.
Ranked #1 on
Audio Classification
on VGGSound
