Search Results for author:
Found 21 papers, 9 papers with code
TreeKV: Smooth Key-Value Cache Compression with Tree Structures
Efficient key-value (KV) cache compression is critical for scaling transformer-based Large Language Models (LLMs) in long sequences and resource-limited settings.
FlatQuant: Flatness Matters for LLM Quantization
In this paper, we propose FlatQuant (Fast and Learnable Affine Transformation), a new post-training quantization approach to enhance flatness of weights and activations.
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions
GPT-4o, an omni-modal model that enables vocal conversations with diverse emotions and tones, marks a milestone for omni-modal foundation models.
S2D: Sorted Speculative Decoding For More Efficient Deployment of Nested Large Language Models
In SD, usually, one draft model is used to serve a specific target model; however, in practice, LLMs are diverse, and we might need to deal with many target models or more than one target model simultaneously.
Visually Guided Generative Text-Layout Pre-training for Document Intelligence
Prior study shows that pre-training techniques can boost the performance of visual document understanding (VDU), which typically requires models to gain abilities to perceive and reason both document texts and layouts (e. g., locations of texts and table-cells).
MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric
We find that directly using smaller pre-trained models and applying magnitude-based pruning on CLIP models leads to inflexibility and inferior performance.
IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact
Such outliers are found to allocate most of the attention scores on initial tokens of input, termed as pivot tokens, which are crucial to the performance of quantized LLMs.
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding
While various vision-language pre-training objectives are studied in existing solutions, the document textline, as an intrinsic granularity in VDU, has seldom been explored so far.
Dynamically pruning segformer for efficient semantic segmentation
In this paper, we seek to design a lightweight SegFormer for efficient semantic segmentation.
Towards Efficient Post-training Quantization of Pre-trained Language Models
Experiments on GLUE and SQuAD benchmarks show that our proposed PTQ solution not only performs close to QAT, but also enjoys significant reductions in training time, memory overhead, and data consumption.
Discrete Auto-regressive Variational Attention Models for Text Modeling
We further design discrete latent space for the variational attention and mathematically show that our model is free from posterior collapse.
BinaryBERT: Pushing the Limit of BERT Quantization
In this paper, we propose BinaryBERT, which pushes BERT quantization to the limit by weight binarization.
Revisiting Parameter Sharing for Automatic Neural Channel Number Search
Nevertheless, it is unclear how parameter sharing affects the searching process.
Discrete Variational Attention Models for Language Generation
Variational autoencoders have been widely applied for natural language generation, however, there are two long-standing problems: information under-representation and posterior collapse.
Efficient Bitwidth Search for Practical Mixed Precision Neural Network
Network quantization has rapidly become one of the most widely used methods to compress and accelerate deep neural networks.
RTN: Reparameterized Ternary Network
We first bring up three omitted issues in extremely low-bit networks: the squashing range of quantized values; the gradient vanishing during backpropagation and the unexploited hardware acceleration of ternary networks.
Few Shot Network Compression via Cross Distillation
The core challenge of few shot network compression lies in high estimation errors from the original network during inference, since the compressed network can easily over-fits on the few training instances.
Structured Pruning of Recurrent Neural Networks through Neuron Selection
Recurrent neural networks (RNNs) have recently achieved remarkable successes in a number of applications.
DART: Domain-Adversarial Residual-Transfer Networks for Unsupervised Cross-Domain Image Classification
In this paper, we propose a new unsupervised domain adaptation method named Domain-Adversarial Residual-Transfer (DART) learning of Deep Neural Networks to tackle cross-domain image classification tasks.
PocketFlow: An Automated Framework for Compressing and Accelerating Deep Neural Networks
Deep neural networks are widely used in various domains, but the prohibitive computational complexity prevents their deployment on mobile devices.
Stochastic Sequential Neural Networks with Structured Inference
Inheriting these advantages of stochastic neural sequential models, we propose a structured and stochastic sequential neural network, which models both the long-term dependencies via recurrent neural networks and the uncertainty in the segmentation and labels via discrete random variables.
