Search Results for author:
Found 27 papers, 14 papers with code
Exemplar-Free Class Incremental Learning via Incremental Representation
Exemplar-Free Class Incremental Learning (efCIL) aims to continuously incorporate the knowledge from new classes while retaining previously learned information, without storing any old-class exemplars (i. e., samples).
E2Net: Resource-Efficient Continual Learning with Elastic Expansion Network
In E2Net, we propose Representative Network Distillation to identify the representative core subnet by assessing parameter quantity and output similarity with the working network, distilling analogous subnets within the working network to mitigate reliance on rehearsal buffers and facilitating knowledge transfer across previous tasks.
CLIP-KD: An Empirical Study of Distilling CLIP Models
CLIP has become a promising language-supervised visual pre-training framework and achieves excellent performance over a wide range of tasks.
Categories of Response-Based, Feature-Based, and Relation-Based Knowledge Distillation
Knowledge Distillation (KD) aims to optimize a lightweight network from the perspective of over-parameterized training.
Team AcieLee: Technical Report for EPIC-SOUNDS Audio-Based Interaction Recognition Challenge 2023
In this report, we describe the technical details of our submission to the EPIC-SOUNDS Audio-Based Interaction Recognition Challenge 2023, by Team "AcieLee" (username: Yuqi\_Li).
Modeling Dual Period-Varying Preferences for Takeaway Recommendation
Different from traditional recommendation, takeaway recommendation faces two main challenges: (1) Dual Interaction-Aware Preference Modeling.
eTag: Class-Incremental Learning with Embedding Distillation and Task-Oriented Generation
Most successful CIL methods incrementally train a feature extractor with the aid of stored exemplars, or estimate the feature distribution with the stored prototypes.
Lung Nodule Segmentation and Uncertain Region Prediction with an Uncertainty-Aware Attention Mechanism
Radiologists possess diverse training and clinical experiences, leading to variations in the segmentation annotations of lung nodules and resulting in segmentation uncertainty. Conventional methods typically select a single annotation as the learning target or attempt to learn a latent space comprising multiple annotations.
MixSKD: Self-Knowledge Distillation from Mixup for Image Recognition
MixSKD mutually distills feature maps and probability distributions between the random pair of original images and their mixup images in a meaningful way.
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition
This enables each network to learn extra contrastive knowledge from others, leading to better feature representations, thus improving the performance of visual recognition tasks.
Localizing Semantic Patches for Accelerating Image Classification
This ensures the exact mapping from a high-level spatial location to the specific input image patch.
Cross-Image Relational Knowledge Distillation for Semantic Segmentation
Current Knowledge Distillation (KD) methods for semantic segmentation often guide the student to mimic the teacher's structured information generated from individual data samples.
Prior Gradient Mask Guided Pruning-Aware Fine-Tuning
In detail, the proposed PGMPF selectively suppresses the gradient of those ”unimportant” parameters via a prior gradient mask generated by the pruning criterion during fine-tuning.
Lifelong Generative Learning via Knowledge Reconstruction
Generative models often incur the catastrophic forgetting problem when they are used to sequentially learning multiple tasks, i. e., lifelong generative learning.
Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution
Each auxiliary branch is guided to learn self-supervision augmented task and distill this distribution from teacher to student.
Hierarchical Self-supervised Augmented Knowledge Distillation
We therefore adopt an alternative self-supervised augmented task to guide the network to learn the joint distribution of the original recognition task and self-supervised auxiliary task.
Ranked #20 on
Knowledge Distillation
on ImageNet
Mutual Contrastive Learning for Visual Representation Learning
We present a collaborative learning method called Mutual Contrastive Learning (MCL) for general visual representation learning.
GHFP: Gradually Hard Filter Pruning
Filter pruning is widely used to reduce the computation of deep learning, enabling the deployment of Deep Neural Networks (DNNs) in resource-limited devices.
Softer Pruning, Incremental Regularization
Network pruning is widely used to compress Deep Neural Networks (DNNs).
Multi-view Contrastive Learning for Online Knowledge Distillation
Previous Online Knowledge Distillation (OKD) often carries out mutually exchanging probability distributions, but neglects the useful representational knowledge.
Localizing Interpretable Multi-scale informative Patches Derived from Media Classification Task
Deep convolutional neural networks (CNN) always depend on wider receptive field (RF) and more complex non-linearity to achieve state-of-the-art performance, while suffering the increased difficult to interpret how relevant patches contribute the final prediction.
Towards More Efficient and Effective Inference: The Joint Decision of Multi-Participants
Existing approaches to improve the performances of convolutional neural networks by optimizing the local architectures or deepening the networks tend to increase the size of models significantly.
DRNet: Dissect and Reconstruct the Convolutional Neural Network via Interpretable Manners
For VGG16 pre-trained on ImageNet, our method averagely gains 14. 29\% accuracy promotion for two-classes sub-tasks.
Rethinking the Number of Channels for the Convolutional Neural Network
In this paper, we propose a method for efficient automatic architecture search which is special to the widths of networks instead of the connections of neural architecture.
Gated Convolutional Networks with Hybrid Connectivity for Image Classification
We propose a simple yet effective method to reduce the redundancy of DenseNet by substantially decreasing the number of stacked modules by replacing the original bottleneck by our SMG module, which is augmented by local residual.
Ranked #60 on
Image Classification
on CIFAR-10
Multi-Objective Pruning for CNNs Using Genetic Algorithm
In this work, we propose a heuristic genetic algorithm (GA) for pruning convolutional neural networks (CNNs) according to the multi-objective trade-off among error, computation and sparsity.
EENA: Efficient Evolution of Neural Architecture
Latest algorithms for automatic neural architecture search perform remarkable but are basically directionless in search space and computational expensive in training of every intermediate architecture.
