Search Results for author:
Found 20 papers, 17 papers with code
LiteFocus: Accelerated Diffusion Inference for Long Audio Synthesis
Latent diffusion models have shown promising results in audio generation, making notable advancements over traditional methods.
Isomorphic Pruning for Vision Models
For instance, we improve the accuracy of DeiT-Tiny from 74. 52% to 77. 50% by pruning an off-the-shelf DeiT-Base model.
PruningBench: A Comprehensive Benchmark of Structural Pruning
PruningBench showcases the following three characteristics: 1) PruningBench employs a unified and consistent framework for evaluating the effectiveness of diverse structural pruning techniques; 2) PruningBench systematically evaluates 16 existing pruning methods, encompassing a wide array of models (e. g., CNNs and ViTs) and tasks (e. g., classification and detection); 3) PruningBench provides easily implementable interfaces to facilitate the implementation of future pruning methods, and enables the subsequent researchers to incorporate their work into our leaderboards.
AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoising
To address this, we introduce AsyncDiff, a universal and plug-and-play acceleration scheme that enables model parallelism across multiple devices.
Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching
To address the challenge of the exponential search space in deep models for identifying layers to cache and remove, we propose a novel differentiable optimization objective.
SlimSAM: 0.1% Data Makes Segment Anything Slim
To address this challenging trade-off, we introduce SlimSAM, a novel data-efficient SAM compression method that achieves superior performance with extremely less training data.
DeepCache: Accelerating Diffusion Models for Free
Diffusion models have recently gained unprecedented attention in the field of image synthesis due to their remarkable generative capabilities.
LLM-Pruner: On the Structural Pruning of Large Language Models
With LLM being a general-purpose task solver, we explore its compression in a task-agnostic manner, which aims to preserve the multi-task solving and language generation ability of the original LLM.
Structural Pruning for Diffusion Models
Generative modeling has recently undergone remarkable advancements, primarily propelled by the transformative implications of Diffusion Probabilistic Models (DPMs).
DepGraph: Towards Any Structural Pruning
Structural pruning enables model acceleration by removing structurally-grouped parameters from neural networks.
Federated Selective Aggregation for Knowledge Amalgamation
The goal of FedSA is to train a student model for a new task with the help of several decentralized teachers, whose pre-training tasks and data are different and agnostic.
Prompting to Distill: Boosting Data-Free Knowledge Distillation via Reinforced Prompt
Data-free knowledge distillation (DFKD) conducts knowledge distillation via eliminating the dependence of original training data, and has recently achieved impressive results in accelerating pre-trained language models.
Knowledge Amalgamation for Object Detection with Transformers
Moreover, the transformer-based students excel in learning amalgamated knowledge, as they have mastered heterogeneous detection tasks rapidly and achieved superior or at least comparable performance to those of the teachers in their specializations.
Up to 100$\times$ Faster Data-free Knowledge Distillation
At the heart of our approach is a novel strategy to reuse the shared common features in training data so as to synthesize different data instances.
Mosaicking to Distill: Knowledge Distillation from Out-of-Domain Data
Knowledge distillation~(KD) aims to craft a compact student model that imitates the behavior of a pre-trained teacher in a target domain.
Contrastive Model Inversion for Data-Free Knowledge Distillation
In this paper, we propose Contrastive Model Inversion~(CMI), where the data diversity is explicitly modeled as an optimizable objective, to alleviate the mode collapse issue.
Adversarial Self-Supervised Data-Free Distillation for Text Classification
To the best of our knowledge, our framework is the first data-free distillation framework designed for NLP tasks.
Impression Space from Deep Template Network
This network is referred to as the {\emph{Template Network}} because its filters will be used as templates to reconstruct images from the impression.
Data-Free Adversarial Distillation
Knowledge Distillation (KD) has made remarkable progress in the last few years and become a popular paradigm for model compression and knowledge transfer.
Knowledge Amalgamation from Heterogeneous Networks by Common Feature Learning
An increasing number of well-trained deep networks have been released online by researchers and developers, enabling the community to reuse them in a plug-and-play way without accessing the training annotations.
