Search Results for author:
Found 9 papers, 6 papers with code
Algorithmic Stability of Stochastic Gradient Descent with Momentum under Heavy-Tailed Noise
In this work, we establish generalization bounds for SGD with momentum (SGDm) under heavy-tailed gradient noise.
Grokking at the Edge of Numerical Stability
To validate our hypotheses, we introduce two key contributions that address the challenges in grokking tasks: StableMax, a new activation function that prevents SC and enables grokking without regularization, and $\perp$Grad, a training algorithm that promotes quick generalization in grokking tasks by preventing NLM altogether.
Evaluating AI systems under uncertain ground truth: a case study in dermatology
Our approach boils down to generating multiple samples of medical condition probabilities, then evaluating and averaging performance metrics based on these sampled probabilities.
Implicit Compressibility of Overparametrized Neural Networks Trained with Heavy-Tailed SGD
Neural network compression has been an increasingly important subject, not only due to its practical relevance, but also due to its theoretical implications, as there is an explicit connection between compressibility and generalization error.
Algorithmic Stability of Heavy-Tailed Stochastic Gradient Descent on Least Squares
Recent studies have shown that heavy tails can emerge in stochastic optimization and that the heaviness of the tails have links to the generalization error.
DebiasedDTA: A Framework for Improving the Generalizability of Drug-Target Affinity Prediction Models
Here, we present DebiasedDTA, a novel drug-target affinity (DTA) prediction model training framework that addresses dataset biases to improve the generalizability of affinity prediction models.
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks
Neural network compression techniques have become increasingly popular as they can drastically reduce the storage and computation requirements for very large networks.
Bayesian Model Selection for Identifying Markov Equivalent Causal Graphs
We believe that the Bayesian approach to causal discovery both allows the rich methodology of Bayesian inference to be used in various difficult aspects of this problem and provides a unifying framework to causal discovery research.
Bayesian Allocation Model: Inference by Sequential Monte Carlo for Nonnegative Tensor Factorizations and Topic Models using Polya Urns
We introduce a dynamic generative model, Bayesian allocation model (BAM), which establishes explicit connections between nonnegative tensor factorization (NTF), graphical models of discrete probability distributions and their Bayesian extensions, and the topic models such as the latent Dirichlet allocation.
