Search Results for author:
Found 8 papers, 1 papers with code
MTrainS: Improving DLRM training efficiency using heterogeneous memories
In Deep Learning Recommendation Models (DLRM), sparse features capturing categorical inputs through embedding tables are the major contributors to model size and require high memory bandwidth.
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers.
Mixed-Precision Embedding Using a Cache
We propose a novel change to embedding tables using a cache memory architecture, where the majority of rows in an embedding is trained in low precision, and the most frequently or recently accessed rows cached and trained in full precision.
Hybrid Composition with IdleBlock: More Efficient Networks for Image Recognition
We propose a new building block, IdleBlock, which naturally prunes connections within the block.
Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implications
The application of deep learning techniques resulted in remarkable improvement of machine learning models.
On Periodic Functions as Regularizers for Quantization of Neural Networks
We apply these functions component-wise and add the sum over the model parameters as a regularizer to the model loss during training.
High performance ultra-low-precision convolutions on mobile devices
Many applications of mobile deep learning, especially real-time computer vision workloads, are constrained by computation power.
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
To achieve this result, we adopt a hyper-parameter-free linear scaling rule for adjusting learning rates as a function of minibatch size and develop a new warmup scheme that overcomes optimization challenges early in training.
