Search Results for author:
Found 11 papers, 8 papers with code
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
Key-value (KV) caching has become the de-facto to accelerate generation speed for large language models (LLMs) inference.
Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs
In human-written articles, we often leverage the subtleties of text style, such as bold and italics, to guide the attention of readers.
Efficient Long-Range Transformers: You Need to Attend More, but Not Necessarily at Every Layer
These models leverage the attention mechanism to capture long- and short-range dependencies in the sequence.
LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximation
Pruning enhances the diversity of low-rank approximations, and low-rank approximation prevents pruning from losing too many expressive neurons.
AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning
Therefore, many fine-tuning methods are proposed to learn incremental updates of pre-trained weights in a parameter efficient way, e. g., low-rank increments.
Less is More: Task-aware Layer-wise Distillation for Language Model Compression
As such, TED reduces the knowledge gap between the two models and helps the student to fit better on the target task.
PLATON: Pruning Large Transformer Models with Upper Confidence Bound of Weight Importance
Large Transformer-based models have exhibited superior performance in various natural language processing and computer vision tasks.
MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation
We propose MoEBERT, which uses a Mixture-of-Experts structure to increase model capacity and inference speed.
A Biased Graph Neural Network Sampler with Near-Optimal Regret
Graph neural networks (GNN) have recently emerged as a vehicle for applying deep network architectures to graph and relational data.
A Non-asymptotic comparison of SVRG and SGD: tradeoffs between compute and speed
In this paper, we present a non-asymptotic analysis of SVRG under a noisy least squares regression problem.
AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods
Adam is shown not being able to converge to the optimal solution in certain cases.
