Search Results for author:
Found 4 papers, 2 papers with code
Tempo: Accelerating Transformer-Based Model Training through Memory Footprint Reduction
We implement Tempo and evaluate the throughput, memory usage, and accuracy/loss on the BERT Large pre-training task.
Hidet: Task-Mapping Programming Paradigm for Deep Learning Tensor Programs
With the proposed paradigm, we implement a deep learning compiler Hidet.
Echo: Compiler-based GPU Memory Footprint Reduction for LSTM RNN Training
For each feature map recomputation to be effective and efficient, its effect on (1) the total memory footprint, and (2) the total execution time has to be carefully estimated.
TBD: Benchmarking and Analyzing Deep Neural Network Training
Our primary goal in this work is to break this myopic view by (i) proposing a new benchmark for DNN training, called TBD (TBD is short for Training Benchmark for DNNs), that uses a representative set of DNN models that cover a wide range of machine learning applications: image classification, machine translation, speech recognition, object detection, adversarial networks, reinforcement learning, and (ii) by performing an extensive performance analysis of training these different applications on three major deep learning frameworks (TensorFlow, MXNet, CNTK) across different hardware configurations (single-GPU, multi-GPU, and multi-machine).
