Search Results for author:
Found 31 papers, 8 papers with code
Motion Prompting: Controlling Video Generation with Motion Trajectories
Motion control is crucial for generating expressive and compelling video content; however, most existing video generation models rely mainly on text prompts for control, which struggle to capture the nuances of dynamic actions and temporal compositions.
High-Resolution Frame Interpolation with Patch-based Cascaded Diffusion
To address these issues, we introduce a patch-based cascaded pixel diffusion model for high resolution frame interpolation, HIFI, that excels in these scenarios while achieving competitive performance on standard benchmarks.
Ranked #1 on
Video Frame Interpolation
on X4K1000FPS
A Simple Approach to Unifying Diffusion-based Conditional Generation
Recent progress in image generation has sparked research into controlling these models through condition signals, with various methods addressing specific challenges in conditional generation.
MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion
Estimating geometry from dynamic scenes, where objects move and deform over time, remains a core challenge in computer vision.
WonderWorld: Interactive 3D Scene Generation from a Single Image
Existing scene generation approaches fall short of speed as they often require (1) progressively generating many views and depth maps, and (2) time-consuming optimization of the scene geometry representations.
DreamWalk: Style Space Exploration using Diffusion Guidance
Text-conditioned diffusion models can generate impressive images, but fall short when it comes to fine-grained control.
Lumiere: A Space-Time Diffusion Model for Video Generation
We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis.
Ranked #5 on
Text-to-Video Generation
on UCF-101
Efficient Hybrid Zoom using Camera Fusion on Mobile Phones
DSLR cameras can achieve multiple zoom levels via shifting lens distances or swapping lens types.
Boundary Attention: Learning curves, corners, junctions and grouping
We present a lightweight network that infers grouping and boundaries, including curves, corners and junctions.
Zero-Shot Metric Depth with a Field-of-View Conditioned Diffusion Model
In contrast, we advocate a generic, task-agnostic diffusion model, with several advancements such as log-scale depth parameterization to enable joint modeling of indoor and outdoor scenes, conditioning on the field-of-view (FOV) to handle scale ambiguity and synthetically augmenting FOV during training to generalize beyond the limited camera intrinsics in training datasets.
Ranked #22 on
Monocular Depth Estimation
on NYU-Depth V2
(using extra training data)
WonderJourney: Going from Anywhere to Everywhere
We introduce WonderJourney, a modularized framework for perpetual 3D scene generation.
DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback
Then, it uses two VLMs to select the best generation: a Visual Question Answering model that measures the alignment of generated images to the text, and another that measures the generation's aesthetic quality.
Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence
This paper identifies the importance of being geometry-aware for semantic correspondence and reveals a limitation of the features of current foundation models under simple post-processing.
Ranked #1 on
Semantic correspondence
on SPair-71k
ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Image
Further, we observe that Score Distillation Sampling (SDS) tends to truncate the distribution of complex backgrounds during distillation of 360-degree scenes, and propose "SDS anchoring" to improve the diversity of synthesized novel views.
Substance or Style: What Does Your Image Embedding Know?
We find that image-text models (CLIP and ALIGN) are better at recognizing new examples of style transfer than masking-based models (CAN and MAE).
A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence
Text-to-image diffusion models have made significant advances in generating and editing high-quality images.
Ranked #1 on
Dense Pixel Correspondence Estimation
on TSS
Dense Pixel Correspondence Estimation
Representation Learning
+2
Accidental Light Probes
Recovering lighting in a scene from a single image is a fundamental problem in computer vision.
Self-supervised AutoFlow
Recently, AutoFlow has shown promising results on learning a training set for optical flow, but requires ground truth labels in the target domain to compute its search metric.
Disentangling Architecture and Training for Optical Flow
Our newly trained RAFT achieves an Fl-all score of 4. 31% on KITTI 2015, more accurate than all published optical flow methods at the time of writing.
Pyramid Adversarial Training Improves ViT Performance
In this work, we present pyramid adversarial training (PyramidAT), a simple and effective technique to improve ViT's overall performance.
Ranked #9 on
Domain Generalization
on ImageNet-C
(using extra training data)
Deep survival analysis with longitudinal X-rays for COVID-19
Time-to-event analysis is an important statistical tool for allocating clinical resources such as ICU beds.
OCONet: Image Extrapolation by Object Completion
Existing methods struggle to extrapolate images with salient objects in the foreground or are limited to very specific objects such as humans, but tend to work well on indoor/outdoor scenes.
AutoFlow: Learning a Better Training Set for Optical Flow
Synthetic datasets play a critical role in pre-training CNN models for optical flow, but they are painstaking to generate and hard to adapt to new applications.
Robust image stitching with multiple registrations
Here, we observe that the use of a single registration often leads to errors, especially in scenes with significant depth variation or object motion.
Object-centered image stitching
Image stitching is typically decomposed into three phases: registration, which aligns the source images with a common target image; seam finding, which determines for each target pixel the source image it should come from; and blending, which smooths transitions over the seams.
Learning to Autofocus
Autofocus is an important task for digital cameras, yet current approaches often exhibit poor performance.
Channel selection using Gumbel Softmax
Important applications such as mobile computing require reducing the computational costs of neural network inference.
A discriminative view of MRF pre-processing algorithms
While Markov Random Fields (MRFs) are widely used in computer vision, they present a quite challenging inference problem.
