Network Pruning

210 papers with code • 5 benchmarks • 5 datasets

Network Pruning is a popular approach to reduce a heavy network to obtain a light-weight form by removing redundancy in the heavy network. In this approach, a complex over-parameterized network is first trained, then pruned based on come criterions, and finally fine-tuned to achieve comparable performance with reduced parameters.

Source: Ensemble Knowledge Distillation for Learning Improved and Efficient Networks

Libraries

Use these libraries to find Network Pruning models and implementations

Latest papers with no code

Accelerating Convolutional Neural Network Pruning via Spatial Aura Entropy

no code yet • 8 Dec 2023

We propose a novel method to improve MI computation for CNN pruning, using the spatial aura entropy.

An End-to-End Network Pruning Pipeline with Sparsity Enforcement

no code yet • 4 Dec 2023

Neural networks have emerged as a powerful tool for solving complex tasks across various domains, but their increasing size and computational requirements have posed significant challenges in deploying them on resource-constrained devices.

Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning

no code yet • 23 Nov 2023

The improvement in the performance of efficient and lightweight models (i. e., the student model) is achieved through knowledge distillation (KD), which involves transferring knowledge from more complex models (i. e., the teacher model).

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

no code yet • 21 Nov 2023

Fine-tuning large pre-trained models has become the de facto strategy for developing both task-specific and general-purpose machine learning systems, including developing models that are safe to deploy.

Data Augmentations in Deep Weight Spaces

no code yet • 15 Nov 2023

Learning in weight spaces, where neural networks process the weights of other deep neural networks, has emerged as a promising research direction with applications in various fields, from analyzing and editing neural fields and implicit neural representations, to network pruning and quantization.

Criticality-Guided Efficient Pruning in Spiking Neural Networks Inspired by Critical Brain Hypothesis

no code yet • 5 Nov 2023

Firstly, we propose a low-cost metric for the criticality in SNNs.

Importance Estimation with Random Gradient for Neural Network Pruning

no code yet • 31 Oct 2023

Global Neuron Importance Estimation is used to prune neural networks for efficiency reasons.

SparseByteNN: A Novel Mobile Inference Acceleration Framework Based on Fine-Grained Group Sparsity

no code yet • 30 Oct 2023

Our framework consists of two parts: (a) A fine-grained kernel sparsity schema with a sparsity granularity between structured pruning and unstructured pruning.

Linear Mode Connectivity in Sparse Neural Networks

no code yet • 28 Oct 2023

We find that distilled data, a synthetic summarization of the real data, paired with Iterative Magnitude Pruning (IMP) unveils a new class of sparse networks that are more stable to SGD noise on the real data, than either the dense model, or subnetworks found with real data in IMP.

GraFT: Gradual Fusion Transformer for Multimodal Re-Identification

no code yet • 25 Oct 2023

Object Re-Identification (ReID) is pivotal in computer vision, witnessing an escalating demand for adept multimodal representation learning.