Search Results for author:
Found 6 papers, 2 papers with code
Rapid Switching and Multi-Adapter Fusion via Sparse High Rank Adapters
In this paper, we propose Sparse High Rank Adapters (SHiRA) that directly finetune 1-2% of the base model weights while leaving others unchanged, thus, resulting in a highly sparse adapter.
Sparse High Rank Adapters
However, from a mobile deployment standpoint, we can either avoid inference overhead in the fused mode but lose the ability to switch adapters rapidly, or suffer significant (up to 30% higher) inference latency while enabling rapid switching in the unfused mode.
Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Models
Large generative models such as large language models (LLMs) and diffusion models have revolutionized the fields of NLP and computer vision respectively.
Moccasin: Efficient Tensor Rematerialization for Neural Networks
The deployment and training of neural networks on edge computing devices pose many challenges.
Neural Topological Ordering for Computation Graphs
Recent works on machine learning for combinatorial optimization have shown that learning based approaches can outperform heuristic methods in terms of speed and performance.
SwiftNet: Using Graph Propagation as Meta-knowledge to Search Highly Representative Neural Architectures
Designing neural architectures for edge devices is subject to constraints of accuracy, inference latency, and computational cost.
