Zero-Shot Video Captioning with Evolving Pseudo-Tokens

1 code implementation • 22 Jul 2022 • Yoad Tewel, Yoav Shalev, Roy Nadler, Idan Schwartz, Lior Wolf

We introduce a zero-shot video captioning method that employs two frozen networks: the GPT-2 language model and the CLIP image-text matching model.

Image Captioning Image-text matching +6

End to End Lip Synchronization with a Temporal AutoEncoder

1 code implementation • 30 Mar 2022 • Yoav Shalev, Lior Wolf

We study the problem of syncing the lip movement in a video with the audio stream.

ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmetic

1 code implementation • CVPR 2022 • Yoad Tewel, Yoav Shalev, Idan Schwartz, Lior Wolf

While such models can provide a powerful score for matching and subsequent zero-shot tasks, they are not capable of generating caption given an image.

Contrastive Learning Descriptive +6

Image Animation with Refined Masking

no code implementations • 1 Jan 2021 • Yoav Shalev, Lior Wolf

Conditioned on the source image, the transformed mask is then decoded by a multi-scale generator that renders a realistic image, in which the content of the source frame is animated by the pose in the driving video.

Image Animation

Image Animation with Perturbed Masks

2 code implementations • CVPR 2022 • Yoav Shalev, Lior Wolf

We present a novel approach for image-animation of a source image by a driving video, both depicting the same type of object.

Image Animation