Search Results for author:
Found 10 papers, 4 papers with code
Convergence and Stability of the Stochastic Proximal Point Algorithm with Momentum
Stochastic gradient descent with momentum (SGDM) is the dominant algorithm in many optimization scenarios, including convex optimization instances and non-convex neural network training.
Robust Inference for High-Dimensional Linear Models via Residual Randomization
We propose a residual randomization procedure designed for robust Lasso-based inference in the high-dimensional setting.
Invariant Inference via Residual Randomization
We find that invariant inference via residual randomization has three appealing properties: (1) It is valid under weak and interpretable conditions, allowing for problems with heavy-tailed data, finite clustering, and even some high-dimensional settings.
Dynamical systems theory for causal inference with application to synthetic control methods
In this setting, we propose to screen out control units that have a weak dynamical relationship to the single treated unit before the model is fit.
Methodology
Convergence diagnostics for stochastic gradient descent with constant step size
During the transient phase the procedure converges towards a region of interest, and during the stationary phase the procedure oscillates in that region, commonly around a single point.
The Proximal Robbins-Monro Method
Exact implementations of the proximal Robbins-Monro procedure are challenging, but we show that approximate implementations lead to procedures that are easy to implement, and still dominate classical procedures by achieving numerical stability, practically without tradeoffs.
Stochastic gradient descent methods for estimation with large data sets
When the update is based on a noisy gradient, the stochastic approximation is known as standard stochastic gradient descent, which has been fundamental in modern applications with large data sets.
Towards stability and optimality in stochastic gradient descent
For statistical efficiency, AI-SGD employs averaging of the iterates, which achieves the optimal Cram\'{e}r-Rao bound under strong convexity, i. e., it is an optimal unbiased estimator of the true parameter value.
Implicit Temporal Differences
In reinforcement learning, the TD($\lambda$) algorithm is a fundamental policy evaluation method with an efficient online implementation that is suitable for large-scale problems.
Asymptotic and finite-sample properties of estimators based on stochastic gradients
Here, we introduce implicit stochastic gradient descent procedures, which involve parameter updates that are implicitly defined.
