Search Results for author:
Found 17 papers, 12 papers with code
Class-incremental Learning for Time Series: Benchmark and Evaluation
Real-world environments are inherently non-stationary, frequently introducing new classes over time.
CompeteSMoE - Effective Training of Sparse Mixture of Experts via Competition
Sparse mixture of experts (SMoE) offers an appealing solution to scale up the model complexity beyond the mean of increasing the network's depth or width.
HyperRouter: Towards Efficient Training and Inference of Sparse Mixture of Experts
By routing input tokens to only a few split experts, Sparse Mixture-of-Experts has enabled efficient training of large language models.
On the Out of Distribution Robustness of Foundation Models in Medical Image Segmentation
Constructing a robust model that can effectively generalize to test samples under distribution shifts remains a significant challenge in the field of medical imaging.
Adaptive-saturated RNN: Remember more with less instability
Orthogonal parameterization is a compelling solution to the vanishing gradient problem (VGP) in recurrent neural networks (RNNs).
Ranked #15 on
Sequential Image Classification
on Sequential MNIST
Continual Learning, Fast and Slow
Motivated by this theory, we propose \emph{DualNets} (for Dual Networks), a general continual learning framework comprising a fast learning system for supervised learning of pattern-separated representation from specific tasks and a slow learning system for representation learning of task-agnostic general representation via Self-Supervised Learning (SSL).
Continual Normalization: Rethinking Batch Normalization for Online Continual Learning
Existing continual learning methods use Batch Normalization (BN) to facilitate training and improve generalization across tasks.
Learning Fast and Slow for Online Time Series Forecasting
The fast adaptation capability of deep neural networks in non-stationary environments is critical for online time series forecasting.
DualNet: Continual Learning, Fast and Slow
According to Complementary Learning Systems (CLS) theory~\citep{mcclelland1995there} in neuroscience, humans do effective \emph{continual learning} through two complementary systems: a fast learning system centered on the hippocampus for rapid learning of the specifics and individual experiences, and a slow learning system located in the neocortex for the gradual acquisition of structured knowledge about the environment.
An Efficient Transformer-Based Model for Vietnamese Punctuation Prediction
In both formal and informal texts, missing punctuation marks make the texts confusing and challenging to read.
TATL: Task Agnostic Transfer Learning for Skin Attributes Detection
Existing skin attributes detection methods usually initialize with a pre-trained Imagenet network and then fine-tune on a medical target task.
Contextual Transformation Networks for Online Continual Learning
Continual learning methods with fixed architectures rely on a single network to learn models that can perform well on all tasks.
Online Continual Learning Under Domain Shift
CIER employs an adversarial training to correct the shift in $P(X, Y)$ by matching $P(X|Y)$, which results in an invariant representation that can generalize to unseen domains during inference.
Bilevel Continual Learning
Continual learning aims to learn continuously from a stream of tasks and data in an online-learning fashion, being capable of exploiting what was learned previously to improve current and future tasks while still being able to perform well on the previous tasks.
Extracting Entities and Topics from News and Connecting Criminal Records
The goal of this paper is to summarize methodologies used in extracting entities and topics from a database of criminal records and from a database of newspapers.
URLNet: Learning a URL Representation with Deep Learning for Malicious URL Detection
This approach allows the model to capture several types of semantic information, which was not possible by the existing models.
Online Deep Learning: Learning Deep Neural Networks on the Fly
Deep Neural Networks (DNNs) are typically trained by backpropagation in a batch learning setting, which requires the entire training data to be made available prior to the learning task.
