Search Results for author:
Found 7 papers, 1 papers with code
ST-MoE: Designing Stable and Transferable Sparse Expert Models
But advancing the state-of-the-art across a broad set of natural language tasks has been hindered by training instabilities and uncertain quality during fine-tuning.
Ranked #1 on
Common Sense Reasoning
on ARC (Easy)
Exploring the limits of Concurrency in ML Training on Google TPUs
Recent results in language understanding using neural networks have required training hardware of unprecedentedscale, with thousands of chips cooperating on a single training run.
Highly Available Data Parallel ML training on Mesh Networks
Packets must be routed around the failed chips for full connectivity.
Training EfficientNets at Supercomputer Scale: 83% ImageNet Top-1 Accuracy in One Hour
EfficientNets are a family of state-of-the-art image classification models based on efficiently scaled convolutional neural networks.
Scale MLPerf-0.6 models on Google TPU-v3 Pods
The recent submission of Google TPU-v3 Pods to the industry wide MLPerf v0. 6 training benchmark demonstrates the scalability of a suite of industry relevant ML models.
Image Classification at Supercomputer Scale
Deep learning is extremely computationally intensive, and hardware vendors have responded by building faster accelerators in large clusters.
PowerAI DDL
We train Resnet-101 on Imagenet 22K with 64 IBM Power8 S822LC servers (256 GPUs) in about 7 hours to an accuracy of 33. 8 % validation accuracy.
