Search Results for author:
Found 14 papers, 4 papers with code
Complexity of Finding Stationary Points of Nonconvex Nonsmooth Functions
Therefore, we introduce the notion of (delta, epsilon)-stationarity, a generalization that allows for a point to be within distance delta of an epsilon-stationary point and reduces to epsilon-stationarity for smooth functions.
Unmemorization in Large Language Models via Self-Distillation and Deliberate Imagination
Our results demonstrate the usefulness of this approach across different models and sizes, and also with parameter-efficient fine-tuning, offering a novel pathway to addressing the challenges with private and sensitive data in LLM applications.
Deep hybrid model with satellite imagery: how to combine demand modeling and computer vision for behavior analysis?
The latent space in deep hybrid models can be interpreted, because it reveals meaningful spatial and social patterns.
Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning
Federated Learning is an emerging direction in distributed machine learning that en-ables jointly training a model without sharing the data.
Stochastic Optimization with Non-stationary Noise: The Power of Moment Estimation
In particular, standard results on optimal convergence rates for stochastic optimization assume either there exists a uniform bound on the moments of the gradient noise, or that the noise decays as the algorithm progresses.
IDEAL: Inexact DEcentralized Accelerated Augmented Lagrangian Method
We introduce a framework for designing primal methods under the decentralized optimization setting where local functions are smooth and strongly convex.
Beyond Worst-Case Analysis in Stochastic Approximation: Moment Estimation Improves Instance Complexity
We study oracle complexity of gradient based methods for stochastic approximation problems.
Complexity of Finding Stationary Points of Nonsmooth Nonconvex Functions
In particular, we study the class of Hadamard semi-differentiable functions, perhaps the largest class of nonsmooth functions for which the chain rule of calculus holds.
On the Complexity of Minimizing Convex Finite Sums Without Using the Indices of the Individual Functions
Recent advances in randomized incremental methods for minimizing $L$-smooth $\mu$-strongly convex finite sums have culminated in tight complexity of $\tilde{O}((n+\sqrt{n L/\mu})\log(1/\epsilon))$ and $O(n+\sqrt{nL/\epsilon})$, where $\mu>0$ and $\mu=0$, respectively, and $n$ denotes the number of individual functions.
Perceptual Regularization: Visualizing and Learning Generalizable Representations
We propose a technique termed perceptual regularization that enables both visualization of the latent representation and control over the generality of the learned representation.
ResNet with one-neuron hidden layers is a Universal Approximator
We demonstrate that a very deep ResNet with stacked modules with one neuron per hidden layer and ReLU activation functions can uniformly approximate any Lebesgue integrable function in $d$ dimensions, i. e. $\ell_1(\mathbb{R}^d)$.
Catalyst Acceleration for First-order Convex Optimization: from Theory to Practice
One of the keys to achieve acceleration in theory and in practice is to solve these sub-problems with appropriate accuracy by using the right stopping criterion and the right warm-start strategy.
Catalyst Acceleration for Gradient-Based Non-Convex Optimization
We introduce a generic scheme to solve nonconvex optimization problems using gradient-based algorithms originally designed for minimizing convex functions.
An Inexact Variable Metric Proximal Point Algorithm for Generic Quasi-Newton Acceleration
We propose an inexact variable-metric proximal point algorithm to accelerate gradient-based optimization algorithms.
