Search Results for author:
Found 4 papers, 1 papers with code
Quantization of Large Language Models with an Overdetermined Basis
In this paper, we introduce an algorithm for data quantization based on the principles of Kashin representation.
LoTR: Low Tensor Rank Weight Adaptation
In this paper we generalize and extend an idea of low-rank adaptation (LoRA) of large language models (LLMs) based on Transformer architecture.
Run LoRA Run: Faster and Lighter LoRA Implementations
LoRA is a technique that reduces the number of trainable parameters in a neural network by introducing low-rank adapters to linear layers.
Memory-Efficient Backpropagation through Large Linear Layers
Also, we investigate the variance of the gradient estimate induced by the randomized matrix multiplication.
Cannot find the paper you are looking for? You can
Submit a new open access paper.
