Search Results for author:
Found 8 papers, 1 papers with code
KroneckerBERT: Significant Compression of Pre-trained Language Models Through Kronecker Decomposition and Knowledge Distillation
We push the limits of state-of-the-art Transformer-based pre-trained language model compression using Kronecker decomposition.
QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning
Finetuning large language models requires huge GPU memory, restricting the choice to acquire Larger models.
Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference
We extend SortedNet to generative NLP tasks, making large language models dynamic without any Pre-Training and by only replacing Standard Fine-Tuning (SFT) with Sorted Fine-Tuning (SoFT).
SortedNet, a Place for Every Network and Every Network in its Place: Towards a Generalized Solution for Training Many-in-One Neural Networks
Deep neural networks (DNNs) must cater to a variety of users with different performance needs and budgets, leading to the costly practice of training, storing, and maintaining numerous specific models.
On the Transferability of Whisper-based Representations for "In-the-Wild" Cross-Task Downstream Speech Applications
Large self-supervised pre-trained speech models have achieved remarkable success across various speech-processing tasks.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
KronA: Parameter Efficient Tuning with Kronecker Adapter
We apply the proposed methods for fine-tuning T5 on the GLUE benchmark to show that incorporating the Kronecker-based modules can outperform state-of-the-art PET methods.
Kronecker Decomposition for GPT Compression
GPT is an auto-regressive Transformer-based pre-trained language model which has attracted a lot of attention in the natural language processing (NLP) domain due to its state-of-the-art performance in several downstream tasks.
FoCL: Feature-Oriented Continual Learning for Generative Models
In this paper, we propose a general framework in continual learning for generative models: Feature-oriented Continual Learning (FoCL).
