VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding

facebookresearch/fairseq EMNLP 2021

We present VideoCLIP, a contrastive approach to pre-train a unified model for zero-shot video and text understanding, without using any labels on downstream tasks.

Action Segmentation Video Retrieval

8.19 stars / hour


PaddlePaddle/PaddleNLP ACL 2021

Easy-to-use and powerful NLP library with Awesome model zoo, supporting wide-range of NLP tasks from research to industrial applications, including Neural Search, Question Answering, Information Extraction and Sentiment Analysis end-to-end system.

Document Image Classification Document Layout Analysis +4

3.58 stars / hour

AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars

hongfz16/avatarclip 17 May 2022

Our key insight is to take advantage of the powerful vision-language model CLIP for supervising neural human generation, in terms of 3D geometry, texture and animation.

Language Modelling motion synthesis +1

2.89 stars / hour

Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training

hpcaitech/colossalai 28 Oct 2021

The Transformer architecture has improved the performance of deep learning models in domains such as Computer Vision and Natural Language Processing.

2D Human Pose Estimation

2.37 stars / hour

Vision Transformer Adapter for Dense Predictions

czczup/vit-adapter 17 May 2022

When fine-tuning on downstream tasks, a modality-specific adapter is used to introduce the data and tasks' prior information into the model, making it suitable for these tasks.

Instance Segmentation Object Detection +1

0.86 stars / hour

Towards An End-to-End Framework for Flow-Guided Video Inpainting

MCG-NKU/E2FGVI 6 Apr 2022

Optical flow, which captures motion information across frames, is exploited in recent video inpainting methods through propagating pixels along its trajectories.

Optical Flow Estimation Video Inpainting

0.76 stars / hour

Zero-Shot Text-to-Image Generation

borisdayma/dalle-mini 24 Feb 2021

Text-to-image generation has traditionally focused on finding better modeling assumptions for training on a fixed dataset.

Text to image generation Zero-Shot Text-to-Image Generation

0.72 stars / hour

Neo: Generalizing Confusion Matrix Visualization to Hierarchical and Multi-Output Labels

apple/ml-hierarchical-confusion-matrix 24 Oct 2021

The confusion matrix, a ubiquitous visualization for helping people evaluate machine learning models, is a tabular layout that compares predicted class labels against actual class labels over all data instances.

0.72 stars / hour

A Lightweight Instrument-Agnostic Model for Polyphonic Note Transcription and Multipitch Estimation

spotify/basic-pitch 18 Mar 2022

Despite its simplicity, benchmark results show our system's note estimation to be substantially better than a comparable baseline, and its frame-level accuracy to be only marginally below those of specialized state-of-the-art AMT systems.

Music Transcription

0.62 stars / hour

PennyLane: Automatic differentiation of hybrid quantum-classical computations

PennyLaneAI/pennylane 12 Nov 2018

PennyLane is a Python 3 software framework for optimization and machine learning of quantum and hybrid quantum-classical computations.

0.62 stars / hour