Search Results for author:
Found 8 papers, 8 papers with code
Minimal Ranks, Maximum Confidence: Parameter-efficient Uncertainty Quantification for LoRA
Low-Rank Adaptation (LoRA) enables parameter-efficient fine-tuning of large language models by decomposing weight updates into low-rank matrices, significantly reducing storage and computational overhead.
LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters
The rapid expansion of large language models (LLMs) has underscored the need for parameter-efficient fine-tuning methods, with LoRA (Low-Rank Adaptation) emerging as a popular solution.
r-softmax: Generalized Softmax with Controllable Sparsity Rate
Nowadays artificial neural network models achieve remarkable results in many disciplines.
Step by Step Loss Goes Very Far: Multi-Step Quantization for Adversarial Text Attacks
We propose a novel gradient-based attack against transformer-based language models that searches for an adversarial example in a continuous space of token probabilities.
Revisiting Offline Compression: Going Beyond Factorization-based Methods for Transformer Language Models
Recent transformer language models achieve outstanding results in many natural language processing (NLP) tasks.
Direction is what you need: Improving Word Embedding Compression in Large Language Models
The adoption of Transformer-based models in natural language processing (NLP) has led to great success using a massive number of parameters.
Zero Time Waste: Recycling Predictions in Early Exit Neural Networks
The problem of reducing processing time of large deep learning models is a fundamental challenge in many real-world applications.
Finding the Optimal Network Depth in Classification Tasks
We develop a fast end-to-end method for training lightweight neural networks using multiple classifier heads.
