Search Results for author:
Found 14 papers, 6 papers with code
CHAI: Clustered Head Attention for Efficient LLM Inference
We observe that there is a high amount of redundancy across heads on which tokens they pay attention to.
Decoding Speculative Decoding
However, our experiments indicate the contrary with throughput diminishing as the probability of generated tokens to be accepted by the target model increases.
MultiFusionNet: Multilayer Multimodal Fusion of Deep Neural Networks for Chest X-Ray Image Classification
The proposed multilayer multimodal fusion model, along with the FDSFM module, holds promise for accurate disease classification and can also be extended to other disease classifications in chest X-ray images.
Cuttlefish: Low-Rank Model Training without All the Tuning
Cuttlefish leverages the observation that after a few epochs of full-rank training, the stable rank (i. e., an approximation of the true rank) of each layer stabilizes at a constant value.
BagPipe: Accelerating Deep Recommendation Model Training
Based on these insights, we develop Bagpipe, a system for training deep recommendation models that uses caching and prefetching to overlap remote embedding accesses with the computation.
Pufferfish: Communication-efficient Models At No Extra Cost
In this work, we present Pufferfish, a communication and computation efficient distributed training framework that incorporates the gradient compression into the model training process via training low-rank, pre-factorized deep networks.
On the Utility of Gradient Compression in Distributed Training Systems
A rich body of prior work has highlighted the existence of communication bottlenecks in synchronous data-parallel training.
AutoFreeze: Automatically Freezing Model Blocks to Accelerate Fine-tuning
With the rapid adoption of machine learning (ML), a number of domains now use the approach of fine tuning models which were pre-trained on a large corpus of data.
Regularized Graph Convolutional Networks for Short Text Classification
The lack of structure in short text sequences limits the success of popular NLP methods based on deep learning.
Accordion: Adaptive Gradient Communication via Critical Learning Regime Identification
The techniques usually require choosing a static compression ratio, often requiring users to balance the trade-off between model accuracy and per-iteration speedup.
Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
Due to its decentralized nature, Federated Learning (FL) lends itself to adversarial attacks in the form of backdoors during training.
Scalable K-Medoids via True Error Bound and Familywise Bandits
K-Medoids(KM) is a standard clustering method, used extensively on semi-metric data. Error analyses of KM have traditionally used an in-sample notion of error, which can be far from the true error and suffer from generalization gap.
Graph based Question Answering System
In today's digital age in the dawning era of big data analytics it is not the information but the linking of information through entities and actions which defines the discourse.
A Novel Approach to Develop a New Hybrid Technique for Trademark Image Retrieval
Trademark Image Retrieval is playing a vital role as a part of CBIR System.
