Search Results for author:
Found 19 papers, 7 papers with code
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training
In this paper, we introduce Saliency-Based Adaptive Masking (SBAM), a novel and cost-effective approach that significantly enhances the pre-training performance of Masked Image Modeling (MIM) approaches by prioritizing token salience.
Hyperparameters in Continual Learning: a Reality Check
In the Hyperparameter Tuning phase, each algorithm is iteratively trained with different hyperparameter values to find the optimal hyperparameter values.
Sy-CON: Symmetric Contrastive Loss for Continual Self-Supervised Representation Learning
We first argue that the conventional loss form of continual learning which consists of single task-specific loss (for plasticity) and a regularizer (for stability) may not be ideal for contrastive loss based CSSL that focus on representation learning.
Learning to Unlearn: Instance-wise Unlearning for Pre-trained Classifiers
Since the recent advent of regulations for data protection (e. g., the General Data Protection Regulation), there has been increasing demand in deleting information learned from sensitive data in pre-trained models without retraining from scratch.
Knowledge Unlearning for Mitigating Privacy Risks in Language Models
Pretrained Language Models (LMs) memorize a vast amount of knowledge during initial pretraining, including information that may violate the privacy of personal lives and identities.
Ranked #3 on
Language Modelling
on The Pile
(Test perplexity metric)
Towards More Objective Evaluation of Class Incremental Learning: Representation Learning Perspective
While the common method for evaluating CIL algorithms is based on average test accuracy for all learned classes, we argue that maximizing accuracy alone does not necessarily lead to effective CIL algorithms.
Rebalancing Batch Normalization for Exemplar-based Class-Incremental Learning
The main reason for the ineffectiveness of their method lies in not fully addressing the data imbalance issue, especially in computing the gradients for learning the affine transformation parameters of BN.
Supervised Neural Discrete Universal Denoiser for Adaptive Denoising
Namely, we make the supervised pre-training of Neural DUDE compatible with the adaptive fine-tuning of the parameters based on the given noisy data subject to denoising.
Observations on K-image Expansion of Image-Mixing Augmentation for Classification
Image-mixing augmentations (e. g., Mixup and CutMix), which typically involve mixing two images, have become the de-facto training techniques for image classification.
NCIS: Neural Contextual Iterative Smoothing for Purifying Adversarial Perturbations
We propose a novel and effective purification based adversarial defense method against pre-processor blind white- and black-box attacks.
SSUL: Semantic Segmentation with Unknown Label for Exemplar-based Class-Incremental Learning
While the recent CISS algorithms utilize variants of the knowledge distillation (KD) technique to tackle the problem, they failed to fully address the critical challenges in CISS causing the catastrophic forgetting; the semantic drift of the background class and the multi-label prediction issue.
Ranked #1 on
Disjoint 15-5
on PASCAL VOC 2012
FBI-Denoiser: Fast Blind Image Denoiser for Poisson-Gaussian Noise
To that end, we propose Fast Blind Image Denoiser (FBI-Denoiser) for Poisson-Gaussian noise, which consists of two neural network models; 1) PGE-Net that estimates Poisson-Gaussian noise parameters 2000 times faster than the conventional methods and 2) FBI-Net that realizes a much more efficient BSN for pixelwise affine denoiser in terms of the number of parameters and inference speed.
CPR: Classifier-Projection Regularization for Continual Learning
Inspired by both recent results on neural networks with wide local minima and information theory, CPR adds an additional regularization term that maximizes the entropy of a classifier's output probability.
Continual Learning with Node-Importance based Adaptive Group Sparse Regularization
We propose a novel regularization-based continual learning method, dubbed as Adaptive Group Sparsity based Continual Learning (AGS-CL), using two group sparsity-based penalties.
Uncertainty-based Continual Learning with Adaptive Regularization
We introduce a new neural network-based continual learning algorithm, dubbed as Uncertainty-regularized Continual Learning (UCL), which builds on traditional Bayesian online learning framework with variational inference.
Ranked #11 on
Continual Learning
on ASC (19 tasks)
GAN2GAN: Generative Noise Learning for Blind Denoising with Single Noisy Images
We tackle a challenging blind image denoising problem, in which only single distinct noisy images are available for training a denoiser, and no information about noise is known, except for it being zero-mean, additive, and independent of the clean image.
DoPAMINE: Double-sided Masked CNN for Pixel Adaptive Multiplicative Noise Despeckling
We propose DoPAMINE, a new neural network based multiplicative noise despeckling algorithm.
Fully Convolutional Pixel Adaptive Image Denoiser
We propose a new image denoising algorithm, dubbed as Fully Convolutional Adaptive Image DEnoiser (FC-AIDE), that can learn from an offline supervised training set with a fully convolutional neural network as well as adaptively fine-tune the supervised model for each given noisy image.
Neural Affine Grayscale Image Denoising
We propose a new grayscale image denoiser, dubbed as Neural Affine Image Denoiser (Neural AIDE), which utilizes neural network in a novel way.
