Search Results for author:
Found 92 papers, 46 papers with code
Shuffle and Attend: Video Domain Adaptation
As the first novelty, we propose an attention mechanism which focuses on more discriminative clips and directly optimizes for video-level (cf.
Ranked #3 on
Unsupervised Domain Adaptation
on UCF-HMDB
Single-Image 3D Human Digitization with Shape-Guided Diffusion
We present an approach to generate a 360-degree view of a person with a consistent, high-resolution appearance from a single input image.
OmnimatteRF: Robust Omnimatte with 3D Background Modeling
Video matting has broad applications, from adding interesting effects to casually captured movies to assisting video production professionals.
Dynamic Mesh-Aware Radiance Fields
Embedding polygonal mesh assets within photorealistic Neural Radience Fields (NeRF) volumes, such that they can be rendered and their dynamics simulated in a physically consistent manner with the NeRF, is under-explored from the system perspective of integrating NeRF into the traditional graphics pipeline.
3D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fields
Motion magnification helps us visualize subtle, imperceptible motion.
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video
UrbanIR uses a novel loss to make very good estimates of shadow volumes in the original scene.
Seeing the World through Your Eyes
The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like.
Grounded Text-to-Image Synthesis with Attention Refocusing
Driven by scalable diffusion models trained on large-scale paired text-image datasets, text-to-image synthesis methods have shown compelling results.
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
Despite tremendous progress in generating high-quality images using diffusion models, synthesizing a sequence of animated frames that are both photorealistic and temporally coherent is still in its infancy.
Ranked #3 on
Text-to-Video Generation
on UCF-101
Neural-PBIR Reconstruction of Shape, Material, and Illumination
In the last stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination.
Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation
In contrast, we propose a parameter-free temporal shift module that can leverage the spatial U-Net as is for video generation.
Expressive Text-to-Image Generation with Rich Text
For each region, we enforce its text attributes by creating region-specific detailed prompts and applying region-specific guidance, and maintain its fidelity against plain-text generation through region-based injections.
$\text{DC}^2$: Dual-Camera Defocus Control by Learning to Refocus
Smartphone cameras today are increasingly approaching the versatility and quality of professional cameras through a combination of hardware and software advancements.
Consistent View Synthesis with Pose-Guided Diffusion Models
In this work, we propose a pose-guided diffusion model to generate a consistent long-term video of novel views from a single image.
Progressively Optimized Local Radiance Fields for Robust View Synthesis
For handling unknown poses, we jointly estimate the camera poses with radiance field in a progressive manner.
DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs
Re-rendering the portrait at a proper focal length and camera distance effectively corrects perspective distortions and produces more natural-looking results.
Text-driven Visual Synthesis with Latent Diffusion Prior
There has been tremendous progress in large-scale text-to-image synthesis driven by diffusion models enabling versatile downstream applications such as 3D object synthesis from texts, image editing, and customized generation.
In-N-Out: Face Video Inversion and Editing with Volumetric Decomposition
We evaluate our method's reconstruction accuracy and editability on challenging real videos and showcase favorable results against other baselines.
Shape-aware Text-driven Layered Video Editing
Temporal consistency is essential for video editing applications.
Robust Dynamic Radiance Fields
Dynamic radiance field reconstruction methods aim to model the time-varying structure and appearance of a dynamic scene.
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Volumetric scene representations enable photorealistic view synthesis for static scenes and form the basis of several existing 6-DoF video techniques.
Ranked #1 on
Novel View Synthesis
on DONeRF: Evaluation Dataset
3D Motion Magnification: Visualizing Subtle Motions from Time-Varying Radiance Fields
Motion magnification helps us visualize subtle, imperceptible motion.
DC2: Dual-Camera Defocus Control by Learning To Refocus
Smartphone cameras today are increasingly approaching the versatility and quality of professional cameras through a combination of hardware and software advancements.
ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field
Physical simulations produce excellent predictions of weather effects.
AMICO: Amodal Instance Composition
Image composition aims to blend multiple objects to form a harmonized image.
Temporally Consistent Semantic Video Editing
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e. g., changing object classes, modifying attributes, or transferring styles.
Learning Dynamic View Synthesis With Few RGBD Cameras
Experiments on the HTI dataset show that our method outperforms the baseline per-frame image fidelity and spatial-temporal consistency.
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer
Videos are created to express emotion, exchange information, and share experiences.
Ranked #12 on
Video Generation
on UCF-101
Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature
In this paper, we investigate using rolling shutter with a global reset feature (RSGR) to restore clean global shutter (GS) videos.
Learning Instance-Specific Adaptation for Cross-Domain Segmentation
We propose a test-time adaptation method for cross-domain image segmentation.
Boosting View Synthesis With Residual Transfer
We present a simple but effective technique to boost the rendering quality, which can be easily integrated with most view synthesis methods.
Learning Neural Light Fields With Ray-Space Embedding
Our method supports rendering with a single network evaluation per pixel for small baseline light fields and with only a few evaluations per pixel for light fields with larger baselines.
Learning Neural Light Fields with Ray-Space Embedding Networks
Our method supports rendering with a single network evaluation per pixel for small baseline light field datasets and can also be applied to larger baselines with only a few evaluations per pixel.
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
We present an algorithm for re-rendering a person from a single image under arbitrary poses.
Dynamic View Synthesis from Dynamic Monocular Video
We present an algorithm for generating novel views at arbitrary viewpoints and any input time step given a monocular video of a dynamic scene.
DropLoss for Long-Tail Instance Segmentation
Based on this insight, we develop DropLoss -- a novel adaptive loss to compensate for this imbalance without a trade-off between rare and frequent categories.
Learning Representational Invariances for Data-Efficient Action Recognition
Data augmentation is a ubiquitous technique for improving image classification when labeled data is scarce.
Hybrid Neural Fusion for Full-frame Video Stabilization
Existing video stabilization methods often generate visible distortion or require aggressive cropping of frame boundaries, resulting in smaller field of views.
Portrait Neural Radiance Fields from a Single Image
We present a method for estimating Neural Radiance Fields (NeRF) from a single headshot portrait.
Robust Consistent Video Depth Estimation
We present an algorithm for estimating consistent dense depth maps and camera poses from a monocular video.
Space-time Neural Irradiance Fields for Free-Viewpoint Video
We present a method that learns a spatiotemporal neural irradiance field for dynamic scenes from a single video.
Continuous and Diverse Image-to-Image Translation via Signed Attribute Vectors
Generating a smooth sequence of intermediate results bridges the gap of two different domains, facilitating the morphing effect across domains.
Few-Shot Adaptation of Generative Adversarial Networks
We validate our method in a challenging few-shot setting of 5-100 images in the target domain.
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation
We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
Flow-edge Guided Video Completion
We present a new flow-based video completion algorithm.
Ranked #5 on
Video Inpainting
on DAVIS
DRG: Dual Relation Graph for Human-Object Interaction Detection
We tackle the challenging problem of human-object interaction (HOI) detection.
Ranked #25 on
Human-Object Interaction Detection
on V-COCO
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior for solving various inverse image restoration tasks.
Semantic View Synthesis
We tackle a new problem of semantic view synthesis -- generating free-viewpoint rendering of a synthesized scene using a semantic label map as input.
Learning to See Through Obstructions with Layered Decomposition
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera.
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling
Monocular visual odometry (VO) suffers severely from error accumulation during frame-to-frame pose estimation.
FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning
Recent state-of-the-art semi-supervised learning (SSL) methods use a combination of image-based transformations and consistency regularization as core components.
Instance-aware Image Colorization
Previous methods leverage the deep neural network to map input grayscale images to plausible color outputs directly.
Ranked #2 on
Point-interactive Image Colorization
on Oxford 102 Flowers
(using extra training data)
Consistent Video Depth Estimation
We present an algorithm for reconstructing dense, geometrically consistent depth for all pixels in a monocular video.
3D Photography using Context-aware Layered Depth Inpainting
We propose a method for converting a single RGB-D input image into a 3D photo - a multi-layer representation for novel view synthesis that contains hallucinated color and depth structures in regions occluded in the original view.
Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline
We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization.
Learning to See Through Obstructions
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions or raindrops, from a short sequence of images captured by a moving camera.
Deep Semantic Matching with Foreground Detection and Cycle-Consistency
Establishing dense semantic correspondences between object instances remains a challenging problem due to background clutter, significant scale and pose differences, and large intra-class variations.
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
Few-shot classification aims to recognize novel categories with only few labeled images in each class.
CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency
Unsupervised domain adaptation algorithms aim to transfer the knowledge learned from one domain to another (e. g., synthetic to real images).
Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition
We validate the effectiveness of our method by transferring our pre-trained model to three different tasks, including action classification, temporal localization, and spatio-temporal action detection.
Guided Image-to-Image Translation with Bi-Directional Feature Transformation
We address the problem of guided image-to-image translation where we translate an input image into another while respecting the constraints provided by an external, user-provided guidance image.
Ranked #1 on
Image Reconstruction
on Edge-to-Clothes
Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation
In contrast to existing algorithms that tackle the tasks of semantic matching and object co-segmentation in isolation, our method exploits the complementary nature of the two tasks.
Manifold Graph with Learned Prototypes for Semi-Supervised Image Classification
We then show that when combined with these regularizers, the proposed method facilitates the propagation of information from generated prototypes to image data to further improve results.
DRIT++: Diverse Image-to-Image Translation via Disentangled Representations
In this work, we present an approach based on disentangled representation for generating diverse outputs without paired training images.
A Closer Look at Few-shot Classification
Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples.
Deep Paper Gestalt
Recent years have witnessed a significant increase in the number of paper submissions to computer vision conferences.
DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
We present an unsupervised learning framework for simultaneously training single-view depth prediction and optical flow estimation models using unlabeled video sequences.
VideoMatch: Matching based Video Object Segmentation
Due to the formulation as a prediction task, most of these methods require fine-tuning during test time, such that the deep nets memorize the appearance of the objects of interest in the given video.
Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagation
We even demonstrate competitive results comparable to deep learning based methods in the semi-supervised setting on the DAVIS dataset.
Ranked #3 on
Video Salient Object Detection
on DAVSOD-Difficult20
(using extra training data)
iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection
Our core idea is that the appearance of a person or an object instance contains informative cues on which relevant parts of an image to attend to for facilitating interaction prediction.
Ranked #2 on
Human-Object Interaction Detection
on Ambiguious-HOI
Diverse Image-to-Image Translation via Disentangled Representations
Our model takes the encoded content features extracted from a given input and the attribute vectors sampled from the attribute space to produce diverse outputs at test time.
Domain Adaptation
Multimodal Unsupervised Image-To-Image Translation
+4
Learning Blind Video Temporal Consistency
Our method takes the original unprocessed and per-frame processed videos as inputs to produce a temporally consistent video.
DeepMVS: Learning Multi-view Stereopsis
We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction.
MaskRNN: Instance Level Video Object Segmentation
Instance level video object segmentation is an important technique for video editing and compression.
Semi-Supervised Learning for Optical Flow with Generative Adversarial Networks
Convolutional neural networks (CNNs) have recently been applied to the optical flow estimation problem.
Progressive Representation Adaptation for Weakly Supervised Object Localization
In classification adaptation, we transfer a pre-trained network to a multi-label classification task for recognizing the presence of a certain object in an image.
Joint Image Filtering with Deep Convolutional Networks
In contrast to existing methods that consider only the guidance image, the proposed algorithm can selectively transfer salient structures that are consistent with both guidance and target images.
Tracking Persons-of-Interest via Unsupervised Representation Adaptation
Multi-face tracking in unconstrained videos is a challenging problem as faces of one person often appear drastically different in multiple shots due to significant variations in scale, pose, expression, illumination, and make-up.
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks
However, existing methods often require a large number of network parameters and entail heavy computational loads at runtime for generating high-accuracy super-resolution results.
Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight
Our core idea is that the adversarial examples targeting at a neural network-based policy are not effective for the frame prediction model.
Unsupervised Representation Learning by Sorting Sequences
We present an unsupervised representation learning approach using videos without semantic labels.
Ranked #46 on
Self-Supervised Action Recognition
on HMDB51
Robust Visual Tracking via Hierarchical Convolutional Features
Specifically, we learn adaptive correlation filters on the outputs from each convolutional layer to encode the target appearance.
Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking
Second, we learn a correlation filter over a feature pyramid centered at the estimated target position for predicting scale changes.
Removing Rain From Single Images via a Deep Detail Network
We propose a new deep network architecture for removing rain streaks from individual images based on the deep convolutional neural network (CNN).
Learning Structured Semantic Embeddings for Visual Recognition
First, we exploit the discriminative constraints to capture the intra- and inter-class relationships of image embeddings.
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
Convolutional neural networks have recently demonstrated high-quality reconstruction for single-image super-resolution.
Ranked #39 on
Image Super-Resolution
on BSD100 - 4x upscaling
Clearing the Skies: A deep network architecture for single-image rain removal
We introduce a deep network architecture called DerainNet for removing rain streaks from an image.
Ranked #11 on
Single Image Deraining
on Test100
(SSIM metric)
Weakly Supervised Object Localization With Progressive Domain Adaptation
In this paper, we address this problem by progressive domain adaptation with two main steps: classification adaptation and detection adaptation.
A Comparative Study for Single Image Blind Deblurring
Using these datasets, we conduct a large-scale user study to quantify the performance of several representative state-of-the-art blind deblurring algorithms.
Detecting Migrating Birds at Night
In this paper, we present a vision-based system for detecting migrating birds in flight at night.
Hierarchical Convolutional Features for Visual Tracking
The outputs of the last convolutional layers encode the semantic information of targets and such representations are robust to significant appearance variations.
Single Image Super-Resolution From Transformed Self-Exemplars
However, the internal dictionary obtained from the given image may not always be sufficiently expressive to cover the textural appearance variations in the scene.
