Search Results for author:
Found 35 papers, 20 papers with code
SepPrune: Structured Pruning for Efficient Deep Speech Separation
Although deep learning has substantially advanced speech separation in recent years, most existing studies continue to prioritize separation quality while overlooking computational efficiency, an essential factor for low-latency speech processing in real-time applications.
PrePrompt: Predictive prompting for class incremental learning
Class Incremental Learning (CIL) based on pre-trained models offers a promising direction for open-world continual learning.
Efficient Continual Learning through Frequency Decomposition and Integration
Inspired by this, we propose the Frequency Decomposition and Integration Network (FDINet), a novel framework that decomposes and integrates information across frequencies.
Enhancing Image Generation Fidelity via Progressive Prompts
We find that deeper layers are always responsible for high - level content control, while shallow layers handle low - level content control.
FedKD-hybrid: Federated Hybrid Knowledge Distillation for Lithography Hotspot Detection
Federated Learning (FL) provides novel solutions for machine learning (ML)-based lithography hotspot detection (LHD) under distributed privacy-preserving settings.
Multi-party Collaborative Attention Control for Image Customization
The rapid development of diffusion models has fueled a growing demand for customized image generation.
ECG-guided individual identification via PPG
To this end, electrocardiogram (ECG) signals have been introduced as a novel modality to enhance the density of input information.
MPQ-DM: Mixed Precision Quantization for Extremely Low Bit Diffusion Models
However, the existing quantization methods for diffusion models still cause severe degradation in performance, especially under extremely low bit-widths (2-4 bit).
SGLP: A Similarity Guided Fast Layer Partition Pruning for Compressing Large Deep Models
Based on similarity matrix derived from CKA, we employ Fisher Optimal Segmentation to partition the network into multiple segments, which provides a basis for removing the layers in a segment-wise manner.
Relational Diffusion Distillation for Efficient Image Generation
Therefore, many training-free sampling methods have been proposed to reduce the number of sampling steps required for diffusion models.
Prototype-Driven Multi-Feature Generation for Visible-Infrared Person Re-identification
The primary challenges in visible-infrared person re-identification arise from the differences between visible (vis) and infrared (ir) images, including inter-modal and intra-modal variations.
Online Policy Distillation with Decision-Attention
However, the teacher-student framework requires a well-trained teacher model which is computationally expensive. In the light of online knowledge distillation, we study the knowledge transfer between different policies that can learn diverse knowledge from the same environment. In this work, we propose Online Policy Distillation (OPD) with Decision-Attention (DA), an online learning framework in which different policies operate in the same environment to learn different perspectives of the environment and transfer knowledge to each other to obtain better performance together.
Exemplar-Free Class Incremental Learning via Incremental Representation
Exemplar-Free Class Incremental Learning (efCIL) aims to continuously incorporate the knowledge from new classes while retaining previously learned information, without storing any old-class exemplars (i. e., samples).
CLIP-KD: An Empirical Study of CLIP Model Distillation
The unified method is applied to distill several student models trained on CC3M+12M.
Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation
Knowledge Distillation (KD) aims to optimize a lightweight network from the perspective of over-parameterized training.
Team AcieLee: Technical Report for EPIC-SOUNDS Audio-Based Interaction Recognition Challenge 2023
In this report, we describe the technical details of our submission to the EPIC-SOUNDS Audio-Based Interaction Recognition Challenge 2023, by Team "AcieLee" (username: Yuqi\_Li).
eTag: Class-Incremental Learning with Embedding Distillation and Task-Oriented Generation
Most successful CIL methods incrementally train a feature extractor with the aid of stored exemplars, or estimate the feature distribution with the stored prototypes.
VL-Match: Enhancing Vision-Language Pretraining with Token-Level and Instance-Level Matching
Vision-Language Pretraining (VLP) has significantly improved the performance of various vision-language tasks with the matching of images and texts.
MixSKD: Self-Knowledge Distillation from Mixup for Image Recognition
MixSKD mutually distills feature maps and probability distributions between the random pair of original images and their mixup images in a meaningful way.
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition
This enables each network to learn extra contrastive knowledge from others, leading to better feature representations, thus improving the performance of visual recognition tasks.
Localizing Semantic Patches for Accelerating Image Classification
This ensures the exact mapping from a high-level spatial location to the specific input image patch.
Cross-Image Relational Knowledge Distillation for Semantic Segmentation
Current Knowledge Distillation (KD) methods for semantic segmentation often guide the student to mimic the teacher's structured information generated from individual data samples.
Prior Gradient Mask Guided Pruning-Aware Fine-Tuning
In detail, the proposed PGMPF selectively suppresses the gradient of those ”unimportant” parameters via a prior gradient mask generated by the pruning criterion during fine-tuning.
Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution
Each auxiliary branch is guided to learn self-supervision augmented task and distill this distribution from teacher to student.
Hierarchical Self-supervised Augmented Knowledge Distillation
We therefore adopt an alternative self-supervised augmented task to guide the network to learn the joint distribution of the original recognition task and self-supervised auxiliary task.
Ranked #31 on
Knowledge Distillation
on ImageNet
Mutual Contrastive Learning for Visual Representation Learning
We present a collaborative learning method called Mutual Contrastive Learning (MCL) for general visual representation learning.
Learning Heatmap-Style Jigsaw Puzzles Provides Good Pretraining for 2D Human Pose Estimation
We note that 2D pose estimation task is highly dependent on the contextual relationship between image patches, thus we introduce a self-supervised method for pretraining 2D pose estimation networks.
Softer Pruning, Incremental Regularization
Network pruning is widely used to compress Deep Neural Networks (DNNs).
Multi-view Contrastive Learning for Online Knowledge Distillation
Previous Online Knowledge Distillation (OKD) often carries out mutually exchanging probability distributions, but neglects the useful representational knowledge.
Localizing Interpretable Multi-scale informative Patches Derived from Media Classification Task
Deep convolutional neural networks (CNN) always depend on wider receptive field (RF) and more complex non-linearity to achieve state-of-the-art performance, while suffering the increased difficult to interpret how relevant patches contribute the final prediction.
DRNet: Dissect and Reconstruct the Convolutional Neural Network via Interpretable Manners
For VGG16 pre-trained on ImageNet, our method averagely gains 14. 29\% accuracy promotion for two-classes sub-tasks.
Rethinking the Number of Channels for the Convolutional Neural Network
In this paper, we propose a method for efficient automatic architecture search which is special to the widths of networks instead of the connections of neural architecture.
Gated Convolutional Networks with Hybrid Connectivity for Image Classification
We propose a simple yet effective method to reduce the redundancy of DenseNet by substantially decreasing the number of stacked modules by replacing the original bottleneck by our SMG module, which is augmented by local residual.
Ranked #66 on
Image Classification
on CIFAR-10
Multi-Objective Pruning for CNNs Using Genetic Algorithm
In this work, we propose a heuristic genetic algorithm (GA) for pruning convolutional neural networks (CNNs) according to the multi-objective trade-off among error, computation and sparsity.
EENA: Efficient Evolution of Neural Architecture
Latest algorithms for automatic neural architecture search perform remarkable but are basically directionless in search space and computational expensive in training of every intermediate architecture.
