Search Results for author:
Found 36 papers, 16 papers with code
SinGAN: Learning a Generative Model from a Single Natural Image
We introduce SinGAN, an unconditional generative model that can be learned from a single natural image.
MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation
In this work, we present MultiDiffusion, a unified framework that enables versatile and controllable image generation, using a pre-trained text-to-image diffusion model, without any further training or finetuning.
TokenFlow: Consistent Diffusion Features for Consistent Video Editing
In this work, we present a framework that harnesses the power of a text-to-image diffusion model for the task of text-driven video editing.
Text2LIVE: Text-Driven Layered Image and Video Editing
Given an input image or video and a target text prompt, our goal is to edit the appearance of existing objects (e. g., object's texture) or augment the scene with visual effects (e. g., smoke, fire) in a semantically meaningful manner.
Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
Large-scale text-to-image generative models have been a revolutionary breakthrough in the evolution of generative AI, allowing us to synthesize diverse images that convey highly complex visual concepts.
Ranked #10 on
Text-based Image Editing
on PIE-Bench
Layered Neural Atlases for Consistent Video Editing
We present a method that decomposes, or "unwraps", an input video into a set of layered 2D atlases, each providing a unified representation of the appearance of an object (or background) over the video.
Splicing ViT Features for Semantic Appearance Transfer
Specifically, our goal is to generate an image in which objects in a source structure image are "painted" with the visual appearance of their semantically related objects in a target appearance image.
Deep ViT Features as Dense Visual Descriptors
To distill the power of ViT features from convoluted design choices, we restrict ourselves to lightweight zero-shot methodologies (e. g., binning and clustering) applied directly to the features.
Ranked #6 on
Feature Upsampling
on ImageNet
DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data
Furthermore, our metric outperforms both prior learned metrics and recent large vision models on these tasks.
Self-Distilled StyleGAN: Towards Generation from Internet Photos
To meet these challenges, we proposed a StyleGAN-based self-distillation approach, which consists of two main components: (i) A generative-based self-filtering of the dataset to eliminate outlier images, in order to generate an adequate training set, and (ii) Perceptual clustering of the generated images to detect the inherent data modalities, which are then employed to improve StyleGAN's "truncation trick" in the image synthesis process.
Looking to Listen at the Cocktail Party: A Speaker-Independent Audio-Visual Model for Speech Separation
Solving this task using only audio as input is extremely challenging and does not provide an association of the separated speech signals with speakers in the video.
Speech2Face: Learning the Face Behind a Voice
How much can we infer about a person's looks from the way they speak?
Semantic Pyramid for Image Generation
We demonstrate that our model results in a versatile and flexible framework that can be used in various classic and novel image generation tasks.
SpeedNet: Learning the Speediness in Videos
We demonstrate how those learned features can boost the performance of self-supervised action recognition, and can be used for video retrieval.
Teaching CLIP to Count to Ten
Our counting loss is deployed over automatically-created counterfactual examples, each consisting of an image and a caption containing an incorrect object count.
Layered Neural Rendering for Retiming People in Video
We present a method for retiming people in an ordinary, natural video -- manipulating and editing the time in which different motions of individuals in the video occur.
Smart, Sparse Contours to Represent and Edit Images
We study the problem of reconstructing an image from information stored at contour locations.
Best-Buddies Similarity - Robust Template Matching using Mutual Nearest Neighbors
We propose a novel method for template matching in unconstrained environments.
MoSculp: Interactive Visualization of Shape and Time
We present a system that allows users to visualize complex human motion via 3D motion sculptures---a representation that conveys the 3D structure swept by a human body as it moves through space.
Sparse, Smart Contours to Represent and Edit Images
We study the problem of reconstructing an image from information stored at contour locations.
Modifying Non-Local Variations Across Multiple Views
We present an algorithm for modifying small non-local variations between repeating structures and patterns in multiple images of the same scene.
Best-Buddies Similarity for Robust Template Matching
We propose a novel method for template matching in unconstrained environments.
On the Effectiveness of Visible Watermarks
Since such an attack relies on the consistency of watermarks across image collection, we explore and evaluate how it is affected by various types of inconsistencies in the watermark embedding that could potentially be used to make watermarking more secured.
Learning the Depths of Moving People by Watching Frozen People
We present a method for predicting dense depth in scenarios where both a monocular camera and people in the scene are freely moving.
Omnimatte: Associating Objects and Their Effects in Video
We show results on real-world videos containing interactions between different types of subjects (cars, animals, people) and complex effects, ranging from semi-transparent elements such as smoke and reflections, to fully opaque effects such as objects attached to the subject.
Consistent Depth of Moving Objects in Video
We present a method to estimate depth of a dynamic scene, containing arbitrary moving objects, from an ordinary video captured with a moving camera.
Diverse Generation from a Single Video Made Possible
GANs are able to perform generation and manipulation tasks, trained on a single video.
Diverse Video Generation from a Single Video
GANs are able to perform generation and manipulation tasks, trained on a single video.
Imagic: Text-Based Real Image Editing with Diffusion Models
In this paper we demonstrate, for the very first time, the ability to apply complex (e. g., non-rigid) text-guided semantic edits to a single real image.
Neural Congealing: Aligning Images to a Joint Semantic Atlas
We present Neural Congealing -- a zero-shot self-supervised framework for detecting and jointly aligning semantically-common content across a given set of images.
State of the Art on Diffusion Models for Visual Computing
The field of visual computing is rapidly advancing due to the emergence of generative artificial intelligence (AI), which unlocks unprecedented capabilities for the generation, editing, and reconstruction of images, videos, and 3D scenes.
Disentangling Structure and Appearance in ViT Feature Space
Specifically, our goal is to generate an image in which objects in a source structure image are "painted" with the visual appearance of their semantically related objects in a target appearance image.
Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer
This loss guides the generation process to preserve the overall motion of the input video while complying with the target object in terms of shape and fine-grained motion traits.
Lumiere: A Space-Time Diffusion Model for Video Generation
We introduce Lumiere -- a text-to-video diffusion model designed for synthesizing videos that portray realistic, diverse and coherent motion -- a pivotal challenge in video synthesis.
Ranked #6 on
Text-to-Video Generation
on UCF-101
SceneScape: Text-Driven Consistent Scene Generation
We present a method for text-driven perpetual view generation -- synthesizing long-term videos of various scenes solely, given an input text prompt describing the scene and camera poses.
DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video
Specifically, our framework simultaneously adopts DINO's features to fit to the motion observations of the test video, while training a tracker that directly leverages the refined features.
