Search Results for author:
Found 10 papers, 3 papers with code
Variance-reduced Zeroth-Order Methods for Fine-Tuning Language Models
Evaluated across a range of both masked and autoregressive LMs on benchmark GLUE tasks, MeZO-SVRG outperforms MeZO with up to 20% increase in test accuracies in both full- and partial-parameter fine-tuning settings.
Prominent Roles of Conditionally Invariant Components in Domain Adaptation: Theory and Algorithms
Domain adaptation (DA) is a statistical learning problem that arises when the distribution of the source data used to train a model differs from that of the target data used to evaluate the model.
The Effect of SGD Batch Size on Autoencoder Learning: Sparsity, Sharpness, and Feature Learning
On the other hand, for any batch size strictly smaller than the number of samples, SGD finds a global minimum which is sparse and nearly orthogonal to its initialization, showing that the randomness of stochastic gradients induces a qualitatively different type of "feature selection" in this setting.
Interpreting and improving deep-learning models with reality checks
Recent deep-learning models have achieved impressive predictive performance by learning complex functions of many variables, often at the cost of interpretability.
Adaptive wavelet distillation from neural networks through interpretations
Moreover, interpretable models are concise and often yield computational efficiency.
Transformation Importance with Applications to Cosmology
Machine learning lies at the heart of new possibilities for scientific discovery, knowledge generation, and artificial intelligence.
Statistical guarantees for local graph clustering
In this paper, we adopt a statistical perspective on local graph clustering, and we analyze the performance of the l1-regularized PageRank method~(Fountoulakis et.
An equivalence between critical points for rank constraints versus low-rank factorizations
Two common approaches in low-rank optimization problems are either working directly with a rank constraint on the matrix variable, or optimizing over a low-rank factorization so that the rank constraint is implicitly ensured.
Optimization and Control
Alternating minimization and alternating descent over nonconvex sets
We analyze the performance of alternating minimization for loss functions optimized over two variables, where each variable may be restricted to lie in some potentially nonconvex constraint set.
Robust PCA with compressed data
The robust principal component analysis (RPCA) problem seeks to separate low-rank trends from sparse outlierswithin a data matrix, that is, to approximate a $n\times d$ matrix $D$ as the sum of a low-rank matrix $L$ and a sparse matrix $S$. We examine the robust principal component analysis (RPCA) problem under data compression, wherethe data $Y$ is approximately given by $(L + S)\cdot C$, that is, a low-rank $+$ sparse data matrix that has been compressed to size $n\times m$ (with $m$ substantially smaller than the original dimension $d$) via multiplication witha compression matrix $C$.
