Search Results for author:
Found 14 papers, 8 papers with code
VideoPoet: A Large Language Model for Zero-Shot Video Generation
We present VideoPoet, a language model capable of synthesizing high-quality video, with matching audio, from a large variety of conditioning signals.
Ranked #3 on
Text-to-Video Generation
on MSR-VTT
Photorealistic Video Generation with Diffusion Models
We present W. A. L. T, a transformer-based approach for photorealistic video generation via diffusion modeling.
Ranked #1 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Language Model Beats Diffusion -- Tokenizer is Key to Visual Generation
While Large Language Models (LLMs) are the dominant models for generative tasks in language, they do not perform as well as diffusion models on image and video generation.
Ranked #2 on
Video Prediction
on Kinetics-600 12 frames, 64x64
Blind Motion Deblurring with Pixel-Wise Kernel Estimation via Kernel Prediction Networks
Then, a second network trained jointly with the first one, unrolls a non-blind deconvolution method using the motion kernel field estimated by the first network.
Scaling Painting Style Transfer
Neural style transfer is a deep learning technique that produces an unprecedentedly rich style transfer from a style image to a content image and is particularly impressive when it comes to transferring style from a painting to an image.
MAGVIT: Masked Generative Video Transformer
We introduce the MAsked Generative VIdeo Transformer, MAGVIT, to tackle various video synthesis tasks with a single model.
Ranked #1 on
Video Prediction
on Something-Something V2
Visual Prompt Tuning for Generative Transfer Learning
We base our framework on state-of-the-art generative vision transformers that represent an image as a sequence of visual tokens to the autoregressive or non-autoregressive transformers.
Rethinking Motion Deblurring Training: A Segmentation-Based Method for Simulating Non-Uniform Motion Blurred Images
Successful training of end-to-end deep networks for real motion deblurring requires datasets of sharp/blurred image pairs that are realistic and diverse enough to achieve generalization to real blurred images.
Improved Masked Image Generation with Token-Critic
Given a masked-and-reconstructed real image, the Token-Critic model is trained to distinguish which visual tokens belong to the original image and which were sampled by the generative transformer.
Non-uniform Blur Kernel Estimation via Adaptive Basis Decomposition
In recent years, the removal of motion blur in photographs has seen impressive progress in the hands of deep learning-based methods, trained to map directly from blurry to sharp images.
Detecting Out-Of-Distribution Samples Using Low-Order Deep Features Statistics
The ability to detect when an input sample was not drawn from the training distribution is an important desirable property of deep neural networks.
Overcoming the Disentanglement vs Reconstruction Trade-off via Jacobian Supervision
A major challenge in learning image representations is the disentangling of the factors of variation underlying the image formation.
Psychophysics, Gestalts and Games
Using a computational quantitative version of the non-accidentalness principle, we raise the possibility that the psychophysical and the (older) gestaltist setups, both applicable on dot or Gabor patterns, find a useful complement in a Turing test.
OLÉ: Orthogonal Low-rank Embedding, A Plug and Play Geometric Loss for Deep Learning
Deep neural networks trained using a softmax layer at the top and the cross-entropy loss are ubiquitous tools for image classification.
