Search Results for author:
Found 19 papers, 6 papers with code
Paraphrasing Complex Network: Network Compression via Factor Transfer
Among the model compression methods, a method called knowledge transfer is to train a student network with a stronger teacher network.
Vehicle Image Generation Going Well with The Surroundings
Using a subnetwork based on a precedent work of image completion, our model makes the shape of an object.
StackNet: Stacking Parameters for Continual learning
The StackNet guarantees no degradation in the performance of the previously learned tasks and the index module shows high confidence in finding the origin of an input sample.
HC-Net: Memory-based Incremental Dual-Network System for Continual learning
The C-Net guarantees no degradation in the performance of the previously learned tasks and the H-Net shows high confidence in finding the origin of an input sample.
Feature Fusion for Online Mutual Knowledge Distillation
We propose a learning framework named Feature Fusion Learning (FFL) that efficiently trains a powerful classifier through a fusion module which combines the feature maps generated from parallel neural networks.
QKD: Quantization-aware Knowledge Distillation
First, Self-studying (SS) phase fine-tunes a quantized low-precision student network without KD to obtain a good initialization.
Feature-map-level Online Adversarial Knowledge Distillation
By training a network to fool the corresponding discriminator, it can learn the other network's feature map distribution.
Position-based Scaled Gradient for Model Quantization and Pruning
Second, we empirically show that PSG acting as a regularizer to a weight vector is favorable for model compression domains such as quantization and pruning.
Prototype-based Personalized Pruning
We verify the usefulness of PPP on a couple of tasks in computer vision and Keyword spotting.
PQK: Model Compression via Pruning, Quantization, and Knowledge Distillation
Unlike traditional pruning and KD, PQK makes use of unimportant weights pruned in the pruning process to make a teacher network for training a better student network without pre-training the teacher model.
Dynamic Collective Intelligence Learning: Finding Efficient Sparse Model via Refined Gradients for Pruned Weights
To alleviate this problem, dynamic pruning methods have emerged, which try to find diverse sparsity patterns during training by utilizing Straight-Through-Estimator (STE) to approximate gradients of pruned weights.
Towards Robust Domain Generalization in 2D Neural Audio Processing
While using two-dimensional convolutional neural networks (2D-CNNs) in image processing, it is possible to manipulate domain information using channel statistics, and instance normalization has been a promising way to get domain-invariant features.
Domain Generalization on Efficient Acoustic Scene Classification using Residual Normalization
Moreover, we introduce an efficient architecture, BC-ResNet-ASC, a modified version of the baseline architecture with a limited receptive field.
Self-Distilled Self-Supervised Representation Learning
Through self-distillation, the intermediate layers are better suited for instance discrimination, making the performance of an early-exited sub-network not much degraded from that of the full network.
Detection of Word Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation
Word-level adversarial attacks have shown success in NLP models, drastically decreasing the performance of transformer-based models in recent years.
Domain Generalization with Relaxed Instance Frequency-wise Normalization for Multi-device Acoustic Scene Classification
While using two-dimensional convolutional neural networks (2D-CNNs) in image processing, it is possible to manipulate domain information using channel statistics, and instance normalization has been a promising way to get domain-invariant features.
QTI Submission to DCASE 2021: residual normalization for device-imbalanced acoustic scene classification with efficient design
The goal of the task is to design an audio scene classification system for device-imbalanced datasets under the constraints of model complexity.
Magnitude Attention-based Dynamic Pruning
Existing pruning methods utilize the importance of each weight based on specified criteria only when searching for a sparse structure but do not utilize it during training.
Detection of Adversarial Examples in Text Classification: Benchmark and Baseline via Robust Density Estimation
Word-level adversarial attacks have shown success in NLP models, drastically decreasing the performance of transformer-based models in recent years.
