VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding

facebookresearch/fairseq EMNLP 2021

We present VideoCLIP, a contrastive approach to pre-train a unified model for zero-shot video and text understanding, without using any labels on downstream tasks.

Action Segmentation Video Retrieval

17,003
8.07 stars / hour

Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training

hpcaitech/colossalai 28 Oct 2021

The Transformer architecture has improved the performance of deep learning models in domains such as Computer Vision and Natural Language Processing.

2D Human Pose Estimation

3,439
3.09 stars / hour

PaddleNLP

PaddlePaddle/PaddleNLP 17 Sep 2019

Easy-to-use and powerful NLP library with Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including Neural Search, Question Answering, Information Extraction and Sentiment Analysis end-to-end system.

Language Modelling Reading Comprehension

4,079
2.19 stars / hour

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

hongfz16/avatarclip 17 May 2022

Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.

Language Modelling motion synthesis +1

266
1.75 stars / hour

BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving

zhangyp15/beverse 19 May 2022

Specifically, BEVerse first performs shared feature extraction and lifting to generate 4D BEV representations from multi-timestamp and multi-view images.

3D Object Detection Autonomous Driving +3

40
1.40 stars / hour

Towards Unified Keyframe Propagation Models

runwayml/guided-inpainting 19 May 2022

We evaluate our two-stream approach for inpainting tasks, where experiments show that it improves both the propagation of features within a single frame as required for image inpainting, as well as their propagation from keyframes to target frames.

Image Inpainting Video Inpainting

31
1.15 stars / hour

Towards An End-to-End Framework for Flow-Guided Video Inpainting

MCG-NKU/E2FGVI 6 Apr 2022

Optical flow, which captures motion information across frames, is exploited in recent video inpainting methods through propagating pixels along its trajectories.

Optical Flow Estimation Video Inpainting

245
0.88 stars / hour

Zero-Shot Text-to-Image Generation

borisdayma/dalle-mini 24 Feb 2021

Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset.

Text to image generation Zero-Shot Text-to-Image Generation

1,483
0.82 stars / hour

Automated Crossword Solving

albertkx/berkeley-crossword-solver ACL 2022

We present the Berkeley Crossword Solver, a state-of-the-art approach for automatically solving crossword puzzles.

Question Answering

20
0.79 stars / hour

Extracting Triangular 3D Models, Materials, and Lighting From Images

NVlabs/nvdiffrec 24 Nov 2021

We present an efficient method for joint optimization of topology, materials and lighting from multi-view image observations.

631
0.79 stars / hour