Search Results for author:
Found 25 papers, 9 papers with code
Llama Guard 3-1B-INT4: Compact and Efficient Safeguard for Human-AI Conversations
This paper presents Llama Guard 3-1B-INT4, a compact and efficient Llama Guard model, which has been open-sourced to the community during Meta Connect 2024.
DεpS: Delayed ε-Shrinking for Faster Once-For-All Training
Hence, we propose Delayed $\epsilon$-Shrinking (D$\epsilon$pS) that starts the process of shrinking the full model when it is partially trained (~50%) which leads to training cost improvement and better in-place knowledge distillation to smaller models.
SpinQuant: LLM quantization with learned rotations
With 4-bit quantization of weight, activation, and KV-cache, SpinQuant narrows the accuracy gap on zero-shot reasoning tasks with full precision to merely 2. 9 points on the LLaMA-2 7B model, surpassing LLM-QAT by 19. 1 points and SmoothQuant by 25. 0 points.
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
The resultant models, denoted as MobileLLM-LS, demonstrate a further accuracy enhancement of 0. 7%/0. 8% than MobileLLM 125M/350M.
SiGeo: Sub-One-Shot NAS via Information Theory and Geometry of Loss Landscape
Neural Architecture Search (NAS) has become a widely used tool for automating neural network design.
Rankitect: Ranking Architecture Search Battling World-class Engineers at Meta Scale
In industry system, such as ranking system in Meta, it is unclear whether NAS algorithms from the literature can outperform production baselines because of: (1) scale - Meta ranking systems serve billions of users, (2) strong baselines - the baselines are production models optimized by hundreds to thousands of world-class engineers for years since the rise of deep learning, (3) dynamic baselines - engineers may have established new and stronger baselines during NAS search, and (4) efficiency - the search pipeline must yield results quickly in alignment with the productionization life cycle.
DistDNAS: Search Efficient Feature Interactions within 2 Hours
To optimize search efficiency, DistDNAS distributes the search and aggregates the choice of optimal interaction modules on varying data dates, achieving over 25x speed-up and reducing search cost from 2 days to 2 hours.
Efficient Edge Inference by Selective Query
Training a hybrid learner is difficult since we lack annotations of hard edge-examples.
PerfSAGE: Generalized Inference Performance Predictor for Arbitrary Deep Learning Models on Edge Devices
The ability to accurately predict deep neural network (DNN) inference performance metrics, such as latency, power, and memory footprint, for an arbitrary DNN on a target hardware platform is essential to the design of DNN based models.
Restructurable Activation Networks
To address this question, we propose a new paradigm called Restructurable Activation Networks (RANs) that manipulate the amount of non-linearity in models to improve their hardware-awareness and efficiency.
Magnitude-aware Probabilistic Speaker Embeddings
We propose a new probabilistic speaker embedding extractor using the information encoded in the embedding magnitude and leverage it in the speaker verification pipeline.
UDC: Unified DNAS for Compressible TinyML Models
Deploying TinyML models on low-cost IoT hardware is very challenging, due to limited device memory capacity.
Hybrid Cloud-Edge Networks for Efficient Inference
The first network is a low-capacity network that can be deployed on an edge device, whereas the second is a high-capacity network deployed in the cloud.
MicroNets: Neural Network Architectures for Deploying TinyML Applications on Commodity Microcontrollers
To address this challenge, neural architecture search (NAS) promises to help design accurate ML models that meet the tight MCU memory, latency and energy constraints.
Ranked #1 on
Keyword Spotting
on Google Speech Commands V2 12
MANGO: A Python Library for Parallel Hyperparameter Tuning
Tuning hyperparameters for machine learning algorithms is a tedious task, one that is typically done manually.
TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids
Modern speech enhancement algorithms achieve remarkable noise suppression by means of large recurrent neural networks (RNNs).
Pushing the limits of RNN Compression
This paper introduces a method to compress RNNs for resource constrained environments using Kronecker product (KP).
Compressing RNNs for IoT devices by 15-38x using Kronecker Products
Recurrent Neural Networks (RNN) can be difficult to deploy on resource constrained devices due to their size. As a result, there is a need for compression techniques that can significantly compress RNNs without negatively impacting task accuracy.
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers
The vast majority of processors in the world are actually microcontroller units (MCUs), which find widespread use performing simple control tasks in applications ranging from automobiles to medical devices and office equipment.
Multimodal Sparse Bayesian Dictionary Learning
This paper addresses the problem of learning dictionaries for multimodal datasets, i. e. datasets collected from multiple data sources.
Re-Weighted Learning for Sparsifying Deep Neural Networks
This paper addresses the topic of sparsifying deep neural networks (DNN's).
Relevance Subject Machine: A Novel Person Re-identification Framework
We propose a novel method called the Relevance Subject Machine (RSM) to solve the person re-identification (re-id) problem.
Robust Bayesian Method for Simultaneous Block Sparse Signal Recovery with Applications to Face Recognition
In this paper, we present a novel Bayesian approach to recover simultaneously block sparse signals in the presence of outliers.
A Unified Framework for Sparse Non-Negative Least Squares using Multiplicative Updates and the Non-Negative Matrix Factorization Problem
We show that the proposed framework encompasses a large class of S-NNLS algorithms and provide a computationally efficient inference procedure based on multiplicative update rules.
Rectified Gaussian Scale Mixtures and the Sparse Non-Negative Least Squares Problem
We refer to the proposed method as rectified sparse Bayesian learning (R-SBL).
