Search Results for author:
Found 17 papers, 6 papers with code
BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillation
The upscaling of Large Language Models (LLMs) has yielded impressive advances in natural language processing, yet it also poses significant deployment challenges.
Exploring the Impact of In-Browser Deep Learning Inference on Quality of User Experience and Performance
Additionally, we noticed that in-browser inference increases the time it takes for graphical user interface (GUI) components to load in web browsers by a significant 67. 2\%, which severely impacts the overall QoE for users of web applications that depend on this technology.
AFPQ: Asymmetric Floating Point Quantization for LLMs
Large language models (LLMs) show great performance in various tasks, but face deployment challenges from limited memory capacity and bandwidth.
Accelerating In-Browser Deep Learning Inference on Diverse Edge Clients through Just-in-Time Kernel Optimizations
Web applications are increasingly becoming the primary platform for AI service delivery, making in-browser deep learning (DL) inference more prominent.
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
To tackle the high compute requirements of LLMs, the Mixture-of-Experts (MoE) architecture was introduced which is able to scale its model size without proportionally scaling up its computational requirements.
Constraint-aware and Ranking-distilled Token Pruning for Efficient Transformer Inference
Deploying pre-trained transformer models like BERT on downstream tasks in resource-constrained scenarios is challenging due to their high inference cost, which grows rapidly with input sequence length.
Accurate and Structured Pruning for Efficient Automatic Speech Recognition
In this paper, we propose a novel compression strategy that leverages structured pruning and knowledge distillation to reduce the model size and inference cost of the Conformer model while preserving high recognition performance.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Adam Accumulation to Reduce Memory Footprints of both Activations and Gradients for Large-scale DNN Training
We find that previous gradient accumulation reduces activation memory but fails to be compatible with gradient memory reduction due to a contradiction between preserving gradients and releasing gradients.
Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Models
In this study, we conduct a comparative analysis of INT and FP quantization with the same bit-width, revealing that the optimal quantization format varies across different layers due to the complexity and diversity of tensor distribution.
ElasticViT: Conflict-aware Supernet Training for Deploying Fast Vision Transformer on Diverse Mobile Devices
However, prior supernet training methods that rely on uniform sampling suffer from the gradient conflict issue: the sampled subnets can have vastly different model sizes (e. g., 50M vs. 2G FLOPs), leading to different optimization directions and inferior performance.
SpaceEvo: Hardware-Friendly Search Space Design for Efficient INT8 Inference
The combination of Neural Architecture Search (NAS) and quantization has proven successful in automatically designing low-FLOPs INT8 quantized neural networks (QNN).
LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup
On-device Deep Neural Network (DNN) inference consumes significant computing resources and development efforts.
SwiftPruner: Reinforced Evolutionary Pruning for Efficient Ad Relevance
Remarkably, under our latency requirement of 1900us on CPU, SwiftPruner achieves a 0. 86% higher AUC than the state-of-the-art uniform sparse baseline for BERT-Mini on a large scale real-world dataset.
Turbo: Opportunistic Enhancement for Edge Video Analytics
Edge computing is being widely used for video analytics.
In-Bed Human Pose Estimation from Unseen and Privacy-Preserving Image Domains
Medical applications have benefited greatly from the rapid advancement in computer vision.
Electric Field Tunable Topological Phases in Graphene Nanoribbons
Graphene nanoribbons (GNRs) possess distinct symmetry-protected topological phases.
Mesoscale and Nanoscale Physics Materials Science Computational Physics
