Search Results for author:
Found 64 papers, 30 papers with code
URVOS: Unified Referring Video Object Segmentation Network with a Large-Scale Benchmark
We propose a unified referring video object segmentation network (URVOS).
Ranked #7 on
Referring Video Object Segmentation
on MeViS
Local All-Pair Correspondence for Point Tracking
We introduce LocoTrack, a highly accurate and efficient model designed for the task of tracking any point (TAP) across video sequences.
Ranked #1 on
Point Tracking
on TAP-Vid-DAVIS-First
Exploring Scalability of Self-Training for Open-Vocabulary Temporal Action Localization
First, a class-agnostic action localizer is trained on a human-labeled TAL dataset and used to generate pseudo-labels for unlabeled videos.
MaGGIe: Masked Guided Gradual Human Instance Matting
Human matting is a foundation task in image and video processing, where human foreground pixels are extracted from the input.
IMIL: Interactive Medical Image Learning Framework
We propose the Interactive Medical Image Learning (IMIL) framework, a novel approach for improving the training of medical image analysis algorithms that enables clinician-guided intermediate training data augmentations on misprediction outliers, focusing the algorithm on relevant visual information.
Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models
While there has been significant progress in customizing text-to-image generation models, generating images that combine multiple personalized concepts remains challenging.
FlowTrack: Revisiting Optical Flow for Long-Range Dense Tracking
In the domain of video tracking existing methods often grapple with a trade-off between spatial density and temporal range.
Putting the Object Back into Video Object Segmentation
We present Cutie, a video object segmentation (VOS) network with object-level memory reading, which puts the object representation from memory back into the video object segmentation result.
Ranked #1 on
Semi-Supervised Video Object Segmentation
on MOSE
Tracking Anything with Decoupled Video Segmentation
To 'track anything' without training on video data for every individual task, we develop a decoupled video segmentation approach (DEVA), composed of task-specific image-level segmentation and class/task-agnostic bi-directional temporal propagation.
Ranked #1 on
Unsupervised Video Object Segmentation
on DAVIS 2016 val
(using extra training data)
Open-Vocabulary Video Segmentation
Open-World Video Segmentation
+7
Studying the Impact of Augmentations on Medical Confidence Calibration
In this paper, we evaluate the effects of three modern augmentation techniques, CutMix, MixUp, and CutOut on the calibration and performance of CNNs for medical tasks.
Long-range Multimodal Pretraining for Movie Understanding
While great progress has been attained, there is still a need for a pretrained multimodal model that can perform well in the ever-growing set of movie understanding tasks the community has been establishing.
XMem++: Production-level Video Segmentation From Few Annotated Frames
Despite advancements in user-guided video segmentation, extracting complex objects consistently for highly complex scenes is still a labor-intensive task, especially for production.
INVE: Interactive Neural Video Editing
We present Interactive Neural Video Editing (INVE), a real-time video editing solution, which can assist the video editing process by consistently propagating sparse frame edits to the entire video clip.
Mask-Guided Matting in the Wild
Mask-guided matting has shown great practicality compared to traditional trimap-based methods.
Bridging Images and Videos: A Simple Learning Framework for Large Vocabulary Video Object Detection
First, no tracking supervisions are in LVIS, which leads to inconsistent learning of detection (with LVIS and TAO) and tracking (only with TAO).
Tracking by Associating Clips
The tracking-by-detection paradigm today has become the dominant method for multi-object tracking and works by detecting objects in each frame and then performing data association across frames.
Videogenic: Identifying Highlight Moments in Videos with Professional Photographs as a Prior
This paper investigates the challenge of extracting highlight moments from videos.
VideoMap: Supporting Video Editing Exploration, Brainstorming, and Prototyping in the Latent Space
Video editing is a creative and complex endeavor and we believe that there is potential for reimagining a new video editing interface to better support the creative and exploratory nature of video editing.
A Generalized Framework for Video Instance Segmentation
Notably, we greatly outperform the state-of-the-art on the long VIS benchmark (OVIS), improving 5. 6 AP with ResNet-50 backbone.
Ranked #8 on
Video Instance Segmentation
on YouTube-VIS 2021
(using extra training data)
Per-Clip Video Object Segmentation
In this per-clip inference scheme, we update the memory with an interval and simultaneously process a set of consecutive frames (i. e. clip) between the memory updates.
One-Trimap Video Matting
A key of OTVM is the joint modeling of trimap propagation and alpha prediction.
The Anatomy of Video Editing: A Dataset and Benchmark Suite for AI-Assisted Video Editing
Machine learning is transforming the video editing industry.
VITA: Video Instance Segmentation via Object Token Association
Specifically, we use an image object detector as a means of distilling object-specific contexts into object tokens.
Ranked #13 on
Video Instance Segmentation
on YouTube-VIS 2021
(using extra training data)
Information-Theoretic Bias Reduction via Causal View of Spurious Correlation
We propose an information-theoretic bias measurement technique through a causal interpretation of spurious correlation, which is effective to identify the feature-level algorithmic bias by taking advantage of conditional mutual information.
Hierarchical Memory Matching Network for Video Object Segmentation
Based on a recent memory-based method [33], we propose two advanced memory read modules that enable us to perform memory reading in multiple scales while exploiting temporal smoothness.
Studying the Effects of Self-Attention for Medical Image Analysis
The use of computer vision to automate the classification of medical images is widely studied.
Unsupervised Learning of Debiased Representations with Pseudo-Attributes
Dataset bias is a critical challenge in machine learning since it often leads to a negative impact on a model due to the unintended decision rules captured by spurious correlations.
Learning to Associate Every Segment for Video Panoptic Segmentation
Temporal correspondence - linking pixels or objects across frames - is a fundamental supervisory signal for the video models.
Ranked #6 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
APES: Audiovisual Person Search in Untrimmed Video
To showcase the potential of our new dataset, we propose an audiovisual baseline and benchmark for person retrieval.
Polygonal Point Set Tracking
Since the existing datasets are not suitable to validate our method, we build a new polygonal point set tracking dataset and demonstrate the superior performance of our method over the baselines and existing contour-based VOS methods.
Exemplar-Based Open-Set Panoptic Segmentation Network
We extend panoptic segmentation to the open-world and introduce an open-set panoptic segmentation (OPS) task.
Video Panoptic Segmentation
In this paper, we propose and explore a new video extension of this task, called video panoptic segmentation.
Ranked #7 on
Video Panoptic Segmentation
on Cityscapes-VPS
(using extra training data)
Active Speakers in Context
Current methods for active speak er detection focus on modeling short-term audiovisual information from a single speaker.
Active Speaker Detection
Audio-Visual Active Speaker Detection
History for Visual Dialog: Do we really need it?
Visual Dialog involves "understanding" the dialog history (what has been discussed previously) and the current question (what is asked), in addition to grounding information in the image, to generate the correct response.
DMV: Visual Object Tracking via Part-level Dense Memory and Voting-based Retrieval
We propose a novel memory-based tracker via part-level dense memory and voting-based retrieval, called DMV.
Onion-Peel Networks for Deep Video Completion
Given a set of reference images and a target image with holes, our network fills the hole by referring the contents in the reference images.
Align-and-Attend Network for Globally and Locally Coherent Video Inpainting
Our video inpainting network consists of two stages.
Learning Video Representations from Correspondence Proposals
In particular, it can effectively learn representations for videos by mixing appearance and long-range motion with an RGB-only input.
Deep Blind Video Decaptioning by Temporal Aggregation and Recurrence
Blind video decaptioning is a problem of automatically removing text overlays and inpainting the occluded parts in videos without any input masks.
Deep Video Inpainting
Video inpainting aims to fill spatio-temporal holes with plausible content in a video.
Ranked #7 on
Video Inpainting
on DAVIS
Fast User-Guided Video Object Segmentation by Interaction-and-Propagation Networks
We propose a new multi-round training scheme for the interactive video object segmentation so that the networks can learn how to understand the user's intention and update incorrect estimations during the training.
Ranked #6 on
Interactive Video Object Segmentation
on DAVIS 2017
(AUC-J metric)
Video Object Segmentation using Space-Time Memory Networks
In our framework, the past frames with object masks form an external memory, and the current frame as the query is segmented using the mask information in the memory.
Ranked #4 on
Interactive Video Object Segmentation
on DAVIS 2017
(using extra training data)
GAPLE: Generalizable Approaching Policy LEarning for Robotic Object Searching in Indoor Environment
We study the problem of learning a generalizable action policy for an intelligent agent to actively approach an object of interest in an indoor environment solely from its visual inputs.
Learning to Blend Photos
Specifically, given a foreground image and a background image, our proposed method automatically generates a set of blending photos with scores that indicate the aesthetics quality with the proposed quality network and policy network.
What do I Annotate Next? An Empirical Study of Active Learning for Action Localization
In this paper, we introduce a novel active learning framework for temporal localization that aims to mitigate this data dependency issue.
Contemplating Visual Emotions: Understanding and Overcoming Dataset Bias
While machine learning approaches to visual emotion recognition offer great promise, current methods consider training and testing models on small scale datasets covering limited visual emotion concepts.
BAM: Bottleneck Attention Module
In this work, we focus on the effect of attention in general deep neural networks.
CBAM: Convolutional Block Attention Module
We propose Convolutional Block Attention Module (CBAM), a simple yet effective attention module for feed-forward convolutional neural networks.
Ranked #7 on
Object Detection
on PKU-DDD17-Car
Fast Video Object Segmentation by Reference-Guided Mask Propagation
We validate our method on four benchmark sets that cover single and multiple object segmentation.
Distort-and-Recover: Color Enhancement using Deep Reinforcement Learning
In addition, we present a 'distort-and-recover' training scheme which only requires high-quality reference images for training instead of input and retouched image pairs.
The AdobeIndoorNav Dataset: Towards Deep Reinforcement Learning based Real-world Indoor Robot Visual Navigation
Synthetic data suffers from domain gap to the real-world scenes while visual inputs rendered from 3D reconstructed scenes have undesired holes and artifacts.
Robotics
Reflectance Capture Using Univariate Sampling of BRDFs
A collocated setup provides only a 1-D "univariate" sampling of the 4-D BRDF.
Gradient-based Camera Exposure Control for Outdoor Mobile Platforms
We introduce a novel method to automatically adjust camera exposure for image processing and computer vision applications on mobile robot platforms.
Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks
One of the bottlenecks in training for better representations is the amount of available per-pixel ground truth data that is required for core scene understanding tasks such as semantic segmentation, normal prediction, and object edge detection.
Action-Driven Object Detection with Top-Down Visual Attentions
In this paper, we present an "action-driven" detection mechanism using our "top-down" visual attention model.
Stereo Matching With Color and Monochrome Cameras in Low-Light Conditions
Consumer devices with stereo cameras have become popular because of their low-cost depth sensing capability.
Fine-scale Surface Normal Estimation using a Single NIR Image
We present surface normal estimation using a single near infrared (NIR) image.
Complementary Sets of Shutter Sequences for Motion Deblurring
In this paper, we present a novel multi-image motion deblurring method utilizing the coded exposure technique.
Automatic Content-Aware Color and Tone Stylization
We introduce a new technique that automatically generates diverse, visually compelling stylizations for a photograph in an unsupervised manner.
Vision System and Depth Processing for DRC-HUBO+
This paper presents a vision system and a depth processing algorithm for DRC-HUBO+, the winner of the DRC finals 2015.
AttentionNet: Aggregating Weak Directions for Accurate Object Detection
We present a novel detection method using a deep convolutional neural network (CNN), named AttentionNet.
One-Day Outdoor Photometric Stereo via Skylight Estimation
We present an outdoor photometric stereo method using images captured in a single day.
Fisher Kernel for Deep Neural Activations
In this paper, we present a straightforward framework for better image representation by combining the two approaches.
Color Transfer Using Probabilistic Moving Least Squares
This paper introduces a new color transfer method which is a process of transferring color of an image to match the color of another image of the same scene.
