Search Results for author:
Found 7 papers, 2 papers with code
Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization
Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets, with a specific focus on selective retention of samples that incur moderately high losses.
Doubly Robust Instance-Reweighted Adversarial Training
Experiments on standard classification datasets demonstrate that our proposed approach outperforms related state-of-the-art baseline methods in terms of average robust performance, and at the same time improves the robustness against attacks on the weakest data points.
Algorithm Design for Online Meta-Learning with Task Boundary Detection
More specifically, we first propose two simple but effective detection mechanisms of task switches and distribution shift based on empirical observations, which serve as a key building block for more elegant online model updates in our algorithm: the task switch detection mechanism allows reusing of the best model available for the current task at hand, and the distribution shift detection mechanism differentiates the meta model update in order to preserve the knowledge for in-distribution tasks and quickly learn the new knowledge for out-of-distribution tasks.
A Primal-Dual Approach to Bilevel Optimization with Multiple Inner Minima
Existing algorithms designed for such a problem were applicable to restricted situations and do not come with a full guarantee of convergence.
On the Convergence Theory for Hessian-Free Bilevel Algorithms
Bilevel optimization has arisen as a powerful tool in modern machine learning.
ES-Based Jacobian Enables Faster Bilevel Optimization
Bilevel optimization (BO) has arisen as a powerful tool for solving many modern machine learning problems.
A sequential guiding network with attention for image captioning
The recent advances of deep learning in both computer vision (CV) and natural language processing (NLP) provide us a new way of understanding semantics, by which we can deal with more challenging tasks such as automatic description generation from natural images.
