Search Results for author:
Found 9 papers, 7 papers with code
GhostNetV3: Exploring the Training Strategies for Compact Models
In this paper, by systematically investigating the impact of different training ingredients, we introduce a strong training strategy for compact models.
SAM-DiffSR: Structure-Modulated Diffusion Model for Image Super-Resolution
In this paper, we propose the SAM-DiffSR model, which can utilize the fine-grained structure information from SAM in the process of sampling noise to improve the image quality without additional computational cost during inference.
Data-efficient Large Vision Models through Sequential Autoregression
Training general-purpose vision models on purely sequential visual data, eschewing linguistic inputs, has heralded a new frontier in visual understanding.
One-for-All: Bridge the Gap Between Heterogeneous Architectures in Knowledge Distillation
To tackle the challenge in distilling heterogeneous models, we propose a simple yet effective one-for-all KD framework called OFA-KD, which significantly improves the distillation performance between heterogeneous architectures.
DeViT: Decomposing Vision Transformers for Collaborative Inference in Edge Devices
Our objective is to achieve fast and energy-efficient collaborative inference while maintaining comparable accuracy compared with large ViTs.
VanillaKD: Revisit the Power of Vanilla Knowledge Distillation from Small Scale to Large Scale
The tremendous success of large models trained on extensive datasets demonstrates that scale is a key ingredient in achieving superior results.
Multi-Agent Collaborative Inference via DNN Decoupling: Intermediate Feature Compression and Edge Learning
In this paper, we study the multi-agent collaborative inference scenario, where a single edge server coordinates the inference of multiple UEs.
CDFKD-MFS: Collaborative Data-free Knowledge Distillation via Multi-level Feature Sharing
To tackle this challenge, we propose a framework termed collaborative data-free knowledge distillation via multi-level feature sharing (CDFKD-MFS), which consists of a multi-header student module, an asymmetric adversarial data-free KD module, and an attention-based aggregation module.
Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation
Specifically, we train a tiny student model to match a pre-trained teacher model in the patch-level manifold space.
