Search Results for author:
Found 21 papers, 12 papers with code
Continual Learners are Incremental Model Generalizers
We believe this paper breaks the barriers between pre-training and fine-tuning steps and leads to a sustainable learning framework in which the continual learner incrementally improves model generalization, yielding better transfer to unseen tasks.
Progressive Neural Representation for Sequential Video Compilation
Neural Implicit Representations (NIR) have gained significant attention recently due to their ability to represent complex and high-dimensional data.
Text-Conditioned Sampling Framework for Text-to-Image Generation with Masked Generative Models
Token-based masked generative models are gaining popularity for their fast inference time with parallel decoding.
Forget-free Continual Learning with Soft-Winning SubNetworks
Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which states that competitive smooth (non-binary) subnetworks exist within a dense network in continual learning tasks, we investigate two proposed architecture-based continual learning methods which sequentially learn and select adaptive binary- (WSN) and non-binary Soft-Subnetworks (SoftNet) for each task.
class-incremental learning
Few-Shot Class-Incremental Learning
+1
Efficient Video Representation Learning via Motion-Aware Token Selection
Recently emerged Masked Video Modeling techniques demonstrated their potential by significantly outperforming previous methods in self-supervised learning for video.
Ranked #1 on
Object State Change Classification
on Ego4D
Object State Change Classification
Object State Change Classification on Ego4D
+3
On the Soft-Subnetwork for Few-shot Class Incremental Learning
Inspired by Regularized Lottery Ticket Hypothesis (RLTH), which hypothesizes that there exist smooth (non-binary) subnetworks within a dense network that achieve the competitive performance of the dense network, we propose a few-shot class incremental learning (FSCIL) method referred to as \emph{Soft-SubNetworks (SoftNet)}.
class-incremental learning
Few-Shot Class-Incremental Learning
+1
BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation
Neural network quantization aims to transform high-precision weights and activations of a given neural network into low-precision weights/activations for reduced memory usage and computation, while preserving the performance of the original model.
Forget-free Continual Learning with Winning Subnetworks
Inspired by Lottery Ticket Hypothesis that competitive subnetworks exist within a dense network, we propose a continual learning method referred to as Winning SubNetworks (WSN), which sequentially learns and selects an optimal subnetwork for each task.
Personalized Subgraph Federated Learning
To this end, we introduce a new subgraph FL problem, personalized subgraph FL, which focuses on the joint improvement of the interrelated local GNNs rather than learning a single global model, and propose a novel framework, FEDerated Personalized sUBgraph learning (FED-PUB), to tackle it.
Bitwidth Heterogeneous Federated Learning with Progressive Weight Dequantization
We introduce a pragmatic FL scenario with bitwidth heterogeneity across the participating devices, dubbed as Bitwidth Heterogeneous Federated Learning (BHFL).
Representational Continuity for Unsupervised Continual Learning
Continual learning (CL) aims to learn a sequence of tasks without forgetting the previously acquired knowledge.
Online Coreset Selection for Rehearsal-based Continual Learning
We validate the effectiveness of our coreset selection mechanism over various standard, imbalanced, and noisy datasets against strong continual learning baselines, demonstrating that it improves task adaptation and prevents catastrophic forgetting in a sample-efficient manner.
Rapid Neural Pruning for Novel Datasets with Set-based Task-Adaptive Meta-Pruning
As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.
Federated Semi-Supervised Learning with Inter-Client Consistency & Disjoint Learning
Through extensive experimental validation of our method in the two different scenarios, we show that our method outperforms both local semi-supervised learning and baselines which naively combine federated learning with semi-supervised learning.
Rapid Structural Pruning of Neural Networks with Set-based Task-Adaptive Meta-Pruning
As deep neural networks are growing in size and being increasingly deployed to more resource-limited devices, there has been a recent surge of interest in network pruning methods, which aim to remove less important weights or activations of a given network.
Federated Continual Learning with Weighted Inter-client Transfer
There has been a surge of interest in continual learning and federated learning, both of which are important in deep neural networks in real-world scenarios.
Scalable and Order-robust Continual Learning with Additive Parameter Decomposition
First, a continual learning model should effectively handle catastrophic forgetting and be efficient to train even with a large number of tasks.
ADAPTIVE NETWORK SPARSIFICATION VIA DEPENDENT VARIATIONAL BETA-BERNOULLI DROPOUT
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Adaptive Network Sparsification with Dependent Variational Beta-Bernoulli Dropout
With such input-independent dropout, each neuron is evolved to be generic across inputs, which makes it difficult to sparsify networks without accuracy loss.
Lifelong Learning with Dynamically Expandable Networks
We propose a novel deep network architecture for lifelong learning which we refer to as Dynamically Expandable Network (DEN), that can dynamically decide its network capacity as it trains on a sequence of tasks, to learn a compact overlapping knowledge sharing structure among tasks.
Combined Group and Exclusive Sparsity for Deep Neural Networks
The number of parameters in a deep neural network is usually very large, which helps with its learning capacity but also hinders its scalability and practicality due to memory/time inefficiency and overfitting.
