Search Results for author:
Found 115 papers, 49 papers with code
Shuffle and Attend: Video Domain Adaptation
As the first novelty, we propose an attention mechanism which focuses on more discriminative clips and directly optimizes for video-level (cf.
Ranked #3 on
Unsupervised Domain Adaptation
on UCF-HMDB
Why Are Web AI Agents More Vulnerable Than Standalone LLMs? A Security Analysis
Recent advancements in Web AI agents have demonstrated remarkable capabilities in addressing complex web navigation tasks.
Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation
We present an approach that reconstructs objects from images taken under different illuminations by first relighting the images under a single reference illumination with a multiview relighting diffusion model and then reconstructing the object's geometry and appearance with a radiance field architecture that is robust to the small remaining inconsistencies among the relit images.
Illusion3D: 3D Multiview Illusion with 2D Diffusion Priors
Automatically generating multiview illusions is a compelling challenge, where a single piece of visual content offers distinct interpretations from different viewing perspectives.
Textured Gaussians for Enhanced 3D Scene Appearance Modeling
As such, each Gaussian can represent a richer set of texture patterns and geometric structures, instead of just a single color and ellipsoid as in naive Gaussian Splatting.
Generative Omnimatte: Learning to Decompose Video into Layers
Our method does not assume a stationary scene or require camera pose or depth information and produces clean, complete layers, including convincing completions of occluded dynamic regions.
Planar Reflection-Aware Neural Radiance Fields
Neural Radiance Fields (NeRF) have demonstrated exceptional capabilities in reconstructing complex scenes with high fidelity.
Flash-Splat: 3D Reflection Removal with Flash Cues and Gaussian Splats
We introduce a simple yet effective approach for separating transmitted and reflected light.
Modeling Ambient Scene Dynamics for Free-view Synthesis
We introduce a novel method for dynamic free-view synthesis of an ambient scenes from a monocular capture bringing a immersive quality to the viewing experience.
Rethinking Score Distillation as a Bridge Between Image Distributions
We compare our method to existing approaches for score distillation sampling and show that it can produce high-frequency details with realistic colors.
Coherent Zero-Shot Visual Instruction Generation
Despite the advances in text-to-image synthesis, particularly with diffusion models, generating visual instructions that require consistent representation and smooth state transitions of objects across sequential steps remains a formidable challenge.
VividDream: Generating 3D Scene with Ambient Dynamics
An ensemble of animated videos is then generated using video diffusion models with quality refinement techniques and conditioned on renderings of the static 3D scene from the sampled camera trajectories.
VideoGigaGAN: Towards Detail-rich Video Super-Resolution
We introduce VideoGigaGAN, a new generative VSR model that can produce videos with high-frequency details and temporal consistency.
Ranked #16 on
Video Super-Resolution
on Vid4 - 4x upscaling
(PSNR metric)
On the Content Bias in Fréchet Video Distance
We show that FVD with features extracted from the recent large-scale self-supervised video models is less biased toward image quality.
Taming Latent Diffusion Model for Neural Radiance Field Inpainting
These two problems are further reinforced with the use of pixel-distance losses.
Recent Trends in 3D Reconstruction of General Non-Rigid Scenes
Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision.
Magic Fixup: Streamlining Photo Editing by Watching Dynamic Videos
We show that by using simple segmentations and coarse 2D manipulations, we can synthesize a photorealistic edit faithful to the user's input while addressing second-order effects like harmonizing the lighting and physical interactions between edited objects.
CTGAN: Semantic-guided Conditional Texture Generator for 3D Shapes
The entertainment industry relies on 3D visual content to create immersive experiences, but traditional methods for creating textured 3D models can be time-consuming and subjective.
IRIS: Inverse Rendering of Indoor Scenes from Low Dynamic Range Images
Our results show that IRIS effectively recovers HDR lighting, accurate material, and plausible camera response functions, supporting photorealistic relighting and object insertion.
TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion
In contrast, TextureDreamer can transfer highly detailed, intricate textures from real-world environments to arbitrary objects with only a few casually captured images, potentially significantly democratizing texture creation.
LTM: Lightweight Textured Mesh Extraction and Refinement of Large Unbounded Scenes for Efficient Storage and Real-time Rendering
We validate the effectiveness of the proposed method on large unbounded scenes from mip-NeRF 360 Tanks & Temples and Deep Blending datasets achieving at-par rendering quality with 73x reduced triangles and 11x reduction in memory footprint.
On the Content Bias in Frechet Video Distance
Frechet Video Distance (FVD) a prominent metric for evaluating video generation models is known to conflict with human perception occasionally.
FlowVid: Taming Imperfect Optical Flows for Consistent Video-to-Video Synthesis
This enables our model for video synthesis by editing the first frame with any prevalent I2I models and then propagating edits to successive frames.
Fast View Synthesis of Casual Videos with Soup-of-Planes
Specifically, we build a global static scene model using an extended plane-based scene representation to synthesize temporally coherent novel video.
Single-Image 3D Human Digitization with Shape-Guided Diffusion
We present an approach to generate a 360-degree view of a person with a consistent, high-resolution appearance from a single input image.
OmnimatteRF: Robust Omnimatte with 3D Background Modeling
Video matting has broad applications, from adding interesting effects to casually captured movies to assisting video production professionals.
Dynamic Mesh-Aware Radiance Fields
Embedding polygonal mesh assets within photorealistic Neural Radience Fields (NeRF) volumes, such that they can be rendered and their dynamics simulated in a physically consistent manner with the NeRF, is under-explored from the system perspective of integrating NeRF into the traditional graphics pipeline.
3D Motion Magnification: Visualizing Subtle Motions with Time Varying Radiance Fields
Motion magnification helps us visualize subtle, imperceptible motion.
UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Video
We present UrbanIR (Urban Scene Inverse Rendering), a new inverse graphics model that enables realistic, free-viewpoint renderings of scenes under various lighting conditions with a single video.
Seeing the World through Your Eyes
The reflective nature of the human eye is an underappreciated source of information about what the world around us looks like.
Grounded Text-to-Image Synthesis with Attention Refocusing
Driven by the scalable diffusion models trained on large-scale datasets, text-to-image synthesis methods have shown compelling results.
Preserve Your Own Correlation: A Noise Prior for Video Diffusion Models
Despite tremendous progress in generating high-quality images using diffusion models, synthesizing a sequence of animated frames that are both photorealistic and temporally coherent is still in its infancy.
Ranked #7 on
Text-to-Video Generation
on UCF-101
Neural-PBIR Reconstruction of Shape, Material, and Illumination
In the last stage, initialized by the neural predictions, we perform PBIR to refine the initial results and obtain the final high-quality reconstruction of object shape, material, and illumination.
Ranked #1 on
Depth Prediction
on Stanford-ORB
Latent-Shift: Latent Diffusion with Temporal Shift for Efficient Text-to-Video Generation
In contrast, we propose a parameter-free temporal shift module that can leverage the spatial U-Net as is for video generation.
Expressive Text-to-Image Generation with Rich Text
For each region, we enforce its text attributes by creating region-specific detailed prompts and applying region-specific guidance, and maintain its fidelity against plain-text generation through region-based injections.
$\text{DC}^2$: Dual-Camera Defocus Control by Learning to Refocus
Smartphone cameras today are increasingly approaching the versatility and quality of professional cameras through a combination of hardware and software advancements.
Consistent View Synthesis with Pose-Guided Diffusion Models
In this work, we propose a pose-guided diffusion model to generate a consistent long-term video of novel views from a single image.
Progressively Optimized Local Radiance Fields for Robust View Synthesis
For handling unknown poses, we jointly estimate the camera poses with radiance field in a progressive manner.
DisCO: Portrait Distortion Correction with Perspective-Aware 3D GANs
Close-up facial images captured at short distances often suffer from perspective distortion, resulting in exaggerated facial features and unnatural/unattractive appearances.
Text-driven Visual Synthesis with Latent Diffusion Prior
There has been tremendous progress in large-scale text-to-image synthesis driven by diffusion models enabling versatile downstream applications such as 3D object synthesis from texts, image editing, and customized generation.
In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing
3D-aware GANs offer new capabilities for view synthesis while preserving the editing functionalities of their 2D counterparts.
Shape-aware Text-driven Layered Video Editing
Temporal consistency is essential for video editing applications.
HyperReel: High-Fidelity 6-DoF Video with Ray-Conditioned Sampling
Volumetric scene representations enable photorealistic view synthesis for static scenes and form the basis of several existing 6-DoF video techniques.
Ranked #1 on
Novel View Synthesis
on DONeRF: Evaluation Dataset
Robust Dynamic Radiance Fields
Dynamic radiance field reconstruction methods aim to model the time-varying structure and appearance of a dynamic scene.
DC2: Dual-Camera Defocus Control by Learning To Refocus
Smartphone cameras today are increasingly approaching the versatility and quality of professional cameras through a combination of hardware and software advancements.
3D Motion Magnification: Visualizing Subtle Motions from Time-Varying Radiance Fields
Motion magnification helps us visualize subtle, imperceptible motion.
ClimateNeRF: Extreme Weather Synthesis in Neural Radiance Field
Physical simulations produce excellent predictions of weather effects.
AMICO: Amodal Instance Composition
Image composition aims to blend multiple objects to form a harmonized image.
Temporally Consistent Semantic Video Editing
Generative adversarial networks (GANs) have demonstrated impressive image generation quality and semantic editing capability of real images, e. g., changing object classes, modifying attributes, or transferring styles.
Learning Dynamic View Synthesis With Few RGBD Cameras
Experiments on the HTI dataset show that our method outperforms the baseline per-frame image fidelity and spatial-temporal consistency.
Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformer
Videos are created to express emotion, exchange information, and share experiences.
Ranked #22 on
Video Generation
on UCF-101
Neural Global Shutter: Learn to Restore Video from a Rolling Shutter Camera with Global Reset Feature
In this paper, we investigate using rolling shutter with a global reset feature (RSGR) to restore clean global shutter (GS) videos.
Learning Instance-Specific Adaptation for Cross-Domain Segmentation
We propose a test-time adaptation method for cross-domain image segmentation.
Learning Neural Light Fields With Ray-Space Embedding
Our method supports rendering with a single network evaluation per pixel for small baseline light fields and with only a few evaluations per pixel for light fields with larger baselines.
Boosting View Synthesis With Residual Transfer
We present a simple but effective technique to boost the rendering quality, which can be easily integrated with most view synthesis methods.
Learning Neural Light Fields with Ray-Space Embedding Networks
Our method supports rendering with a single network evaluation per pixel for small baseline light field datasets and can also be applied to larger baselines with only a few evaluations per pixel.
Pose with Style: Detail-Preserving Pose-Guided Image Synthesis with Conditional StyleGAN
We present an algorithm for re-rendering a person from a single image under arbitrary poses.
Dynamic View Synthesis from Dynamic Monocular Video
We present an algorithm for generating novel views at arbitrary viewpoints and any input time step given a monocular video of a dynamic scene.
DropLoss for Long-Tail Instance Segmentation
Based on this insight, we develop DropLoss -- a novel adaptive loss to compensate for this imbalance without a trade-off between rare and frequent categories.
Learning Representational Invariances for Data-Efficient Action Recognition
Data augmentation is a ubiquitous technique for improving image classification when labeled data is scarce.
Hybrid Neural Fusion for Full-frame Video Stabilization
Existing video stabilization methods often generate visible distortion or require aggressive cropping of frame boundaries, resulting in smaller field of views.
Portrait Neural Radiance Fields from a Single Image
We present a method for estimating Neural Radiance Fields (NeRF) from a single headshot portrait.
Robust Consistent Video Depth Estimation
We present an algorithm for estimating consistent dense depth maps and camera poses from a monocular video.
Space-time Neural Irradiance Fields for Free-Viewpoint Video
We present a method that learns a spatiotemporal neural irradiance field for dynamic scenes from a single video.
Continuous and Diverse Image-to-Image Translation via Signed Attribute Vectors
Generating a smooth sequence of intermediate results bridges the gap of two different domains, facilitating the morphing effect across domains.
Few-Shot Adaptation of Generative Adversarial Networks
We validate our method in a challenging few-shot setting of 5-100 images in the target domain.
PseudoSeg: Designing Pseudo Labels for Semantic Segmentation
We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
Flow-edge Guided Video Completion
We present a new flow-based video completion algorithm.
Ranked #2 on
Video Inpainting
on HQVI (480p)
NAS-DIP: Learning Deep Image Prior with Neural Architecture Search
Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior for solving various inverse image restoration tasks.
DRG: Dual Relation Graph for Human-Object Interaction Detection
We tackle the challenging problem of human-object interaction (HOI) detection.
Ranked #26 on
Human-Object Interaction Detection
on V-COCO
Semantic View Synthesis
We tackle a new problem of semantic view synthesis -- generating free-viewpoint rendering of a synthesized scene using a semantic label map as input.
Learning to See Through Obstructions with Layered Decomposition
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions, or adherent raindrops, from a short sequence of images captured by a moving camera.
Learning Monocular Visual Odometry via Self-Supervised Long-Term Modeling
Monocular visual odometry (VO) suffers severely from error accumulation during frame-to-frame pose estimation.
FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning
Recent state-of-the-art semi-supervised learning (SSL) methods use a combination of image-based transformations and consistency regularization as core components.
Instance-aware Image Colorization
Previous methods leverage the deep neural network to map input grayscale images to plausible color outputs directly.
Ranked #2 on
Point-interactive Image Colorization
on CUB-200-2011
(using extra training data)
Consistent Video Depth Estimation
We present an algorithm for reconstructing dense, geometrically consistent depth for all pixels in a monocular video.
3D Photography using Context-aware Layered Depth Inpainting
We propose a method for converting a single RGB-D input image into a 3D photo - a multi-layer representation for novel view synthesis that contains hallucinated color and depth structures in regions occluded in the original view.
Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline
We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization.
Learning to See Through Obstructions
We present a learning-based approach for removing unwanted obstructions, such as window reflections, fence occlusions or raindrops, from a short sequence of images captured by a moving camera.
Deep Semantic Matching with Foreground Detection and Cycle-Consistency
Establishing dense semantic correspondences between object instances remains a challenging problem due to background clutter, significant scale and pose differences, and large intra-class variations.
Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformation
Few-shot classification aims to recognize novel categories with only few labeled images in each class.
Ranked #7 on
Cross-Domain Few-Shot
on CUB
CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency
Unsupervised domain adaptation algorithms aim to transfer the knowledge learned from one domain to another (e. g., synthetic to real images).
Why Can't I Dance in the Mall? Learning to Mitigate Scene Bias in Action Recognition
We validate the effectiveness of our method by transferring our pre-trained model to three different tasks, including action classification, temporal localization, and spatio-temporal action detection.
Guided Image-to-Image Translation with Bi-Directional Feature Transformation
We address the problem of guided image-to-image translation where we translate an input image into another while respecting the constraints provided by an external, user-provided guidance image.
Ranked #1 on
Image Reconstruction
on Edge-to-Clothes
Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation
In contrast to existing algorithms that tackle the tasks of semantic matching and object co-segmentation in isolation, our method exploits the complementary nature of the two tasks.
Manifold Graph with Learned Prototypes for Semi-Supervised Image Classification
We then show that when combined with these regularizers, the proposed method facilitates the propagation of information from generated prototypes to image data to further improve results.
A Closer Look at Few-shot Classification
Few-shot classification aims to learn a classifier to recognize unseen classes during training with limited labeled examples.
Deep Paper Gestalt
Recent years have witnessed a significant increase in the number of paper submissions to computer vision conferences.
DF-Net: Unsupervised Joint Learning of Depth and Flow using Cross-Task Consistency
We present an unsupervised learning framework for simultaneously training single-view depth prediction and optical flow estimation models using unlabeled video sequences.
Unsupervised Video Object Segmentation using Motion Saliency-Guided Spatio-Temporal Propagation
We even demonstrate competitive results comparable to deep learning based methods in the semi-supervised setting on the DAVIS dataset.
Ranked #4 on
Video Salient Object Detection
on DAVSOD-Difficult20
(using extra training data)
VideoMatch: Matching based Video Object Segmentation
Due to the formulation as a prediction task, most of these methods require fine-tuning during test time, such that the deep nets memorize the appearance of the objects of interest in the given video.
iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection
Our core idea is that the appearance of a person or an object instance contains informative cues on which relevant parts of an image to attend to for facilitating interaction prediction.
Ranked #2 on
Human-Object Interaction Detection
on Ambiguious-HOI
Diverse Image-to-Image Translation via Disentangled Representations
Our model takes the encoded content features extracted from a given input and the attribute vectors sampled from the attribute space to produce diverse outputs at test time.
Learning Blind Video Temporal Consistency
Our method takes the original unprocessed and per-frame processed videos as inputs to produce a temporally consistent video.
DeepMVS: Learning Multi-view Stereopsis
We present DeepMVS, a deep convolutional neural network (ConvNet) for multi-view stereo reconstruction.
MaskRNN: Instance Level Video Object Segmentation
Instance level video object segmentation is an important technique for video editing and compression.
Semi-Supervised Learning for Optical Flow with Generative Adversarial Networks
Convolutional neural networks (CNNs) have recently been applied to the optical flow estimation problem.
Progressive Representation Adaptation for Weakly Supervised Object Localization
In classification adaptation, we transfer a pre-trained network to a multi-label classification task for recognizing the presence of a certain object in an image.
Joint Image Filtering with Deep Convolutional Networks
In contrast to existing methods that consider only the guidance image, the proposed algorithm can selectively transfer salient structures that are consistent with both guidance and target images.
Tracking Persons-of-Interest via Unsupervised Representation Adaptation
Multi-face tracking in unconstrained videos is a challenging problem as faces of one person often appear drastically different in multiple shots due to significant variations in scale, pose, expression, illumination, and make-up.
Fast and Accurate Image Super-Resolution with Deep Laplacian Pyramid Networks
However, existing methods often require a large number of network parameters and entail heavy computational loads at runtime for generating high-accuracy super-resolution results.
Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight
Our core idea is that the adversarial examples targeting at a neural network-based policy are not effective for the frame prediction model.
Unsupervised Representation Learning by Sorting Sequences
We present an unsupervised representation learning approach using videos without semantic labels.
Ranked #46 on
Self-Supervised Action Recognition
on HMDB51
Robust Visual Tracking via Hierarchical Convolutional Features
Specifically, we learn adaptive correlation filters on the outputs from each convolutional layer to encode the target appearance.
Adaptive Correlation Filters with Long-Term and Short-Term Memory for Object Tracking
Second, we learn a correlation filter over a feature pyramid centered at the estimated target position for predicting scale changes.
Removing Rain From Single Images via a Deep Detail Network
We propose a new deep network architecture for removing rain streaks from individual images based on the deep convolutional neural network (CNN).
Learning Structured Semantic Embeddings for Visual Recognition
First, we exploit the discriminative constraints to capture the intra- and inter-class relationships of image embeddings.
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
Convolutional neural networks have recently demonstrated high-quality reconstruction for single-image super-resolution.
Ranked #46 on
Image Super-Resolution
on BSD100 - 4x upscaling
Clearing the Skies: A deep network architecture for single-image rain removal
We introduce a deep network architecture called DerainNet for removing rain streaks from an image.
Ranked #11 on
Single Image Deraining
on Test100
(SSIM metric)
Detecting Migrating Birds at Night
In this paper, we present a vision-based system for detecting migrating birds in flight at night.
Weakly Supervised Object Localization With Progressive Domain Adaptation
In this paper, we address this problem by progressive domain adaptation with two main steps: classification adaptation and detection adaptation.
A Comparative Study for Single Image Blind Deblurring
Using these datasets, we conduct a large-scale user study to quantify the performance of several representative state-of-the-art blind deblurring algorithms.
Hierarchical Convolutional Features for Visual Tracking
The outputs of the last convolutional layers encode the semantic information of targets and such representations are robust to significant appearance variations.
Single Image Super-Resolution From Transformed Self-Exemplars
However, the internal dictionary obtained from the given image may not always be sufficiently expressive to cover the textural appearance variations in the scene.
