Search Results for author:
Found 10 papers, 5 papers with code
Adaptive Batch Size Schedules for Distributed Training of Language Models with Data and Model Parallelism
Despite this dilemma, the common practice of choosing batch sizes in language model training often prioritizes training efficiency -- employing either constant large sizes with data parallelism or implementing batch size warmup schedules.
AlignAb: Pareto-Optimal Energy Alignment for Designing Nature-Like Antibodies
We present a three-stage framework for training deep learning models specializing in antibody sequence-structure co-design.
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
Despite an understanding of their convergence and the importance of batch sizes for training efficiency and generalization, optimal batch sizes for local gradient methods are difficult to determine.
BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model
We introduce the \textbf{B}i-Directional \textbf{S}parse \textbf{Hop}field Network (\textbf{BiSHop}), a novel end-to-end framework for deep tabular learning.
SMUTF: Schema Matching Using Generative Tags and Hybrid Features
Recognizing the lack of extensive, publicly available datasets for SM, we have created and open-sourced the HDXSM dataset from the public humanitarian data.
Beyond PID Controllers: PPO with Neuralized PID Policy for Proton Beam Intensity Control in Mu2e
We introduce a novel Proximal Policy Optimization (PPO) algorithm aimed at addressing the challenge of maintaining a uniform proton beam intensity delivery in the Muon to Electron Conversion Experiment (Mu2e) at Fermi National Accelerator Laboratory (Fermilab).
On Sparse Modern Hopfield Model
Building upon this, we derive the sparse memory retrieval dynamics from the sparse energy function and show its one-step approximation is equivalent to the sparse-structured attention.
Feature Programming for Multivariate Time Series Prediction
We introduce the concept of programmable feature engineering for time series modeling and propose a feature programming framework.
Person-job fit estimation from candidate profile and related recruitment history with co-attention neural networks
Existing studies on person-job fit, however, mainly focus on calculating the similarity between the candidate resumes and the job postings on the basis of their contents, without taking the recruiters' experience (i. e., historical successful recruitment records) into consideration.
Open-Ended Multi-Modal Relational Reasoning for Video Question Answering
In this paper, we introduce a robotic agent specifically designed to analyze external environments and address participants' questions.
