Search Results for author:
Found 14 papers, 1 papers with code
LLaMA-NAS: Efficient Neural Architecture Search for Large Language Models
In addition to finding smaller, higher-performing network architectures, our method does so more effectively and efficiently than certain pruning or sparsification techniques.
SimQ-NAS: Simultaneous Quantization Policy and Neural Architecture Search
In this paper, we demonstrate that by using multi-objective search algorithms paired with lightly trained predictors, we can efficiently search for both the sub-network architecture and the corresponding quantization policy and outperform their respective baselines across different performance objectives such as accuracy, model size, and latency.
InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning
However, training super-networks from scratch can be extremely time consuming and compute intensive especially for large models that rely on a two-stage training process of pre-training and fine-tuning.
Sensi-BERT: Towards Sensitivity Driven Fine-Tuning for Parameter-Efficient BERT
In this paper, we present Sensi-BERT, a sensitivity driven efficient fine-tuning of BERT models that can take an off-the-shelf pre-trained BERT model and yield highly parameter-efficient models for downstream tasks.
Sparse Mixture Once-for-all Adversarial Training for Efficient In-Situ Trade-Off Between Accuracy and Robustness of DNNs
Existing deep neural networks (DNNs) that achieve state-of-the-art (SOTA) performance on both clean and adversarially-perturbed images rely on either activation or weight conditioned convolution operations.
A Hardware-Aware Framework for Accelerating Neural Architecture Search Across Modalities
Recent advances in Neural Architecture Search (NAS) such as one-shot NAS offer the ability to extract specialized hardware-aware sub-network configurations from a task-specific super-network.
Accelerating Neural Architecture Exploration Across Modalities Using Genetic Algorithms
Neural architecture search (NAS), the study of automating the discovery of optimal deep neural network architectures for tasks in domains such as computer vision and natural language processing, has seen rapid growth in the machine learning research community.
A Hardware-Aware System for Accelerating Deep Neural Network Optimization
These methods decouple the super-network training from the sub-network search and thus decrease the computational burden of specializing to different hardware platforms.
TrimBERT: Tailoring BERT for Trade-offs
Models based on BERT have been extremely successful in solving a variety of natural language processing (NLP) tasks.
Undivided Attention: Are Intermediate Layers Necessary for BERT?
In recent times, BERT-based models have been extremely successful in solving a variety of natural language processing (NLP) tasks such as reading comprehension, natural language inference, sentiment analysis, etc.
Attention-based Image Upsampling
Convolutional layers are an integral part of many deep neural network solutions in computer vision.
Compact Scene Graphs for Layout Composition and Patch Retrieval
Structured representations such as scene graphs serve as an efficient and compact representation that can be used for downstream rendering or retrieval tasks.
Fast Weight Long Short-Term Memory
Associative memory using fast weights is a short-term memory mechanism that substantially improves the memory capacity and time scale of recurrent neural networks (RNNs).
Driving in the Matrix: Can Virtual Worlds Replace Human-Generated Annotations for Real World Tasks?
Deep learning has rapidly transformed the state of the art algorithms used to address a variety of problems in computer vision and robotics.
