Trending Research

Reformer: The Efficient Transformer

13 Jan 2020google/trax

Large Transformer models routinely achieve state-of-the-art results on a number of tasks but training these models can be prohibitively costly, especially on long sequences.

LANGUAGE MODELLING

3,107
2.34 stars / hour

Depth-Aware Video Frame Interpolation

CVPR 2019 baowenbo/DAIN

The proposed model then warps the input frames, depth maps, and contextual features based on the optical flow and local interpolation kernels for synthesizing the output frame.

OPTICAL FLOW ESTIMATION VIDEO FRAME INTERPOLATION

1,944
1.40 stars / hour

Self-training with Noisy Student improves ImageNet classification

11 Nov 2019google-research/noisystudent

To achieve this result, we first train an EfficientNet model on labeled ImageNet images and use it as a teacher to generate pseudo labels on 300M unlabeled images.

 SOTA for Image Classification on ImageNet (using extra training data)

DATA AUGMENTATION IMAGE CLASSIFICATION

100
0.96 stars / hour

ZeRO: Memory Optimization Towards Training A Trillion Parameter Models

4 Oct 2019microsoft/DeepSpeed

Moving forward, we will work on unlocking stage-2 optimizations, with up to 8x memory savings per device, and ultimately stage-3 optimizations, reducing memory linearly with respect to the number of devices and potentially scaling to models of arbitrary size.

1,840
0.86 stars / hour

Large Batch Optimization for Deep Learning: Training BERT in 76 minutes

ICLR 2020 microsoft/DeepSpeed

In this paper, we first study a principled layerwise adaptation strategy to accelerate training of deep neural networks using large mini-batches.

#9 best model for Question Answering on SQuAD1.1 dev (F1 metric)

QUESTION ANSWERING STOCHASTIC OPTIMIZATION

1,840
0.86 stars / hour

WeatherBench: A benchmark dataset for data-driven weather forecasting

2 Feb 2020pangeo-data/WeatherBench

Data-driven approaches, most prominently deep learning, have become powerful tools for prediction in many domains.

WEATHER FORECASTING

101
0.67 stars / hour

From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Network

8 Jul 2019sshaoshuai/PartA2-Net

3D object detection from LiDAR point cloud is a challenging problem in 3D scene understanding and has many practical applications.

3D OBJECT DETECTION SCENE UNDERSTANDING

41
0.59 stars / hour

First Order Motion Model for Image Animation

NeurIPS 2019 AliaksandrSiarohin/first-order-model

To achieve this, we decouple appearance and motion information using a self-supervised formulation.

IMAGE ANIMATION VIDEO RECONSTRUCTION

164
0.52 stars / hour

ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

26 Sep 2019google-research/ALBERT

Increasing model size when pretraining natural language representations often results in improved performance on downstream tasks.

LINGUISTIC ACCEPTABILITY NATURAL LANGUAGE INFERENCE QUESTION ANSWERING SEMANTIC TEXTUAL SIMILARITY

1,626
0.44 stars / hour