Network Pruning
214 papers with code • 5 benchmarks • 5 datasets
Network Pruning is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.
Source: Ensemble Knowledge Distillation for Learning Improved and Efficient Networks
Benchmarks
These leaderboards are used to track progress in Network Pruning
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Network Pruning models and implementations
CIFAR-10
ImageNet
CIFAR-100
MNIST
Netzschleuder
Latest papers
Aggressive or Imperceptible, or Both: Network Pruning Assisted Hybrid Byzantines in Federated Learning
CRYPTO-KU/FL-Byzantine-Library
•
•
Hence, inspired by the sparse neural networks, we introduce a hybrid sparse Byzantine attack that is composed of two parts: one exhibiting a sparse nature and attacking only certain NN locations with higher sensitivity, and the other being more silent but accumulating over time, where each ideally targets a different type of defence mechanism, and together they form a strong but imperceptible attack.
Auto-Train-Once: Controller Network Guided Automatic Network Pruning from Scratch
Current techniques for deep neural network (DNN) pruning often involve intricate multi-step processes that require domain-specific expertise, making their widespread adoption challenging.
Adversarial Fine-tuning of Compressed Neural Networks for Joint Improvement of Robustness and Efficiency
We present experiments on two benchmark datasets showing that adversarial fine-tuning of compressed models can achieve robustness performance comparable to adversarially trained models, while also improving computational efficiency.
FALCON: FLOP-Aware Combinatorial Optimization for Neural Network Pruning
In this paper, we propose FALCON, a novel combinatorial-optimization-based framework for network pruning that jointly takes into account model accuracy (fidelity), FLOPs, and sparsity constraints.
What to Do When Your Discrete Optimization Is the Size of a Neural Network?
Oftentimes, machine learning applications using neural networks involve solving discrete optimization problems, such as in pruning, parameter-isolation-based continual learning and training of binary networks.
Less is KEN: a Universal and Simple Non-Parametric Pruning Algorithm for Large Language Models
itsmattei/ken
•
•
This approach maintains model performance while allowing storage of only the optimized subnetwork, leading to significant memory savings.
Fluctuation-based Adaptive Structured Pruning for Large Language Models
Retraining-free is important for LLMs' pruning methods.
Towards Higher Ranks via Adversarial Weight Pruning
huawei-noah/Efficient-Computing
•
•
NeurIPS 2023
To this end, we propose a Rank-based PruninG (RPG) method to maintain the ranks of sparse weights in an adversarial manner.
LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS
Recent advancements in real-time neural rendering using point-based techniques have paved the way for the widespread adoption of 3D representations.
Filter-Pruning of Lightweight Face Detectors Using a Geometric Median Criterion
idt-iti/lightweight-face-detector-pruning
•
•
Face detectors are becoming a crucial component of many applications, including surveillance, that often have to run on edge devices with limited processing power and memory.
