Search Results for author:
Found 2 papers, 1 papers with code
Efficient GPT Model Pre-training using Tensor Train Matrix Representation
On the downstream tasks, including language understanding and text summarization, the model performs similarly to the original GPT-2 model.
Few-Bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction
Every modern neural network model has quite a few pointwise nonlinearities in its architecture, and such operation induces additional memory costs which -- as we show -- can be significantly reduced by quantization of the gradients.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
