Search Results for author:
Found 24 papers, 11 papers with code
Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization
Our study starts from an empirical strategy for the light continual training of LLMs using their original pre-training data sets, with a specific focus on selective retention of samples that incur moderately high losses.
End-to-end Supervised Prediction of Arbitrary-size Graphs with Partially-Masked Fused Gromov-Wasserstein Matching
We present a novel end-to-end deep learning-based approach for Supervised Graph Prediction (SGP).
A Large-Scale Empirical Study on Improving the Fairness of Image Classification Models
Specifically, we target the widely-used application scenario of image classification, and utilized three different datasets and five commonly-used performance metrics to assess in total 13 methods from diverse categories.
Rethinking PGD Attack: Is Sign Function Necessary?
Based on that, we further propose a new raw gradient descent (RGD) algorithm that eliminates the use of sign.
Meta ControlNet: Enhancing Task Adaptation via Meta Learning
However, vanilla ControlNet generally requires extensive training of around 5000 steps to achieve a desirable control for a single task.
EyeLS: Shadow-Guided Instrument Landing System for Intraocular Target Approaching in Robotic Eye Surgery
Robotic ophthalmic surgery is an emerging technology to facilitate high-precision interventions such as retina penetration in subretinal injection and removal of floating tissues in retinal detachment depending on the input imaging modalities such as microscopy and intraoperative OCT (iOCT).
Exploiting Edge Features in Graphs with Fused Network Gromov-Wasserstein Distance
Pairwise comparison of graphs is key to many applications in Machine learning ranging from clustering, kernel-based classification/regression and more recently supervised graph prediction.
M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation
This data-driven procedure yields L2O that can efficiently solve problems similar to those seen in training, that is, drawn from the same ``task distribution".
Learning to Generalize Provably in Learning to Optimize
While the optimizer generalization has been recently studied, the optimizee generalization (or learning to generalize) has not been rigorously studied in the L2O context, which is the aim of this paper.
Embedded Silicon-Organic Integrated Neuromorphic System
The development of artificial intelligence (AI) and robotics are both based on the tenet of "science and technology are people-oriented", and both need to achieve efficient communication with the human brain.
APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Tracking
Based on APT-36K, we benchmark several representative models on the following three tracks: (1) supervised animal pose estimation on a single frame under intra- and inter-domain transfer learning settings, (2) inter-species domain generalization test for unseen animals, and (3) animal pose estimation with animal tracking.
ColibriDoc: An Eye-in-Hand Autonomous Trocar Docking System
For this purpose, we present a platform for autonomous trocar docking that combines computer vision and a robotic setup.
Generalizable Learning to Optimize into Wide Valleys
Learning to optimize (L2O) has gained increasing popularity in various optimization tasks, since classical optimizers usually require laborious, problem-specific design and hyperparameter tuning.
Provably Faster Algorithms for Bilevel Optimization
Bilevel optimization has been widely applied in many important machine learning applications such as hyperparameter optimization and meta-learning.
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers.
Neural Network Training Techniques Regularize Optimization Trajectory: An Empirical Study
Specifically, we find that the optimization trajectories of successful DNN trainings consistently obey a certain regularity principle that regularizes the model update direction to be aligned with the trajectory direction.
Bilevel Optimization: Convergence Analysis and Enhanced Design
For the AID-based method, we orderwisely improve the previous convergence rate analysis due to a more practical parameter selection as well as a warm start strategy, and for the ITD-based method we establish the first theoretical convergence rate.
Provably Faster Algorithms for Bilevel Optimization and Applications to Meta-Learning
For the AID-based method, we orderwisely improve the previous finite-time convergence analysis due to a more practical parameter selection as well as a warm start strategy, and for the ITD-based method we establish the first theoretical convergence rate.
Multi-span Style Extraction for Generative Reading Comprehension
Generative machine reading comprehension (MRC) requires a model to generate well-formed answers.
Theoretical Convergence of Multi-Step Model-Agnostic Meta-Learning
As a popular meta-learning approach, the model-agnostic meta-learning (MAML) algorithm has been widely used due to its simplicity and effectiveness.
Retrospective Reader for Machine Reading Comprehension
Inspired by how humans solve reading comprehension questions, we proposed a retrospective reader (Retro-Reader) that integrates two stages of reading and verification strategies: 1) sketchy reading that briefly investigates the overall interactions of passage and question, and yield an initial judgment; 2) intensive reading that verifies the answer and gives the final prediction.
Ranked #7 on
Question Answering
on SQuAD2.0
Deepening Hidden Representations from Pre-trained Language Models
Transformer-based pre-trained language models have proven to be effective for learning contextualized language representation.
An Optimization Principle Of Deep Learning?
In particular, we observe that the trainings that apply the training techniques achieve accelerated convergence and obey the principle with a large $\gamma$, which is consistent with the $\mathcal{O}(1/\gamma K)$ convergence rate result under the optimization principle.
SGD Converges to Global Minimum in Deep Learning via Star-convex Path
Stochastic gradient descent (SGD) has been found to be surprisingly effective in training a variety of deep neural networks.
