Search Results for author:
Found 22 papers, 6 papers with code
DINGO: Distributed Newton-Type Method for Gradient-Norm Optimization
For optimization of a sum of functions in a distributed computing environment, we present a novel communication efficient Newton-type algorithm that enjoys a variety of advantages over similar existing methods.
Newton-ADMM: A Distributed GPU-Accelerated Optimizer for Multiclass Classification Problems
First-order optimization methods, such as stochastic gradient descent (SGD) and its variants, are widely used in machine learning applications due to their simplicity and low per-iteration costs.
Stability Analysis of Newton-MR Under Hessian Perturbations
Recently, stability of Newton-CG under Hessian perturbations, i. e., inexact curvature information, have been extensively studied.
Optimization and Control
Richer priors for infinitely wide multi-layer perceptrons
The model resulting from partially exchangeable priors is a GP, with an additional level of inference in the sense that the prior and posterior predictive distributions require marginalisation over hyperparameters.
DINO: Distributed Newton-Type Optimization Method
Under minimal assumptions, we guarantee global sub-linear convergence of DINO to a first-order stationary point for general non-convex functions and arbitrary data distribution over the network.
Optimization and Control
Avoiding Kernel Fixed Points: Computing with ELU and GELU Infinite Networks
Secondly, and more generally, we analyse the fixed-point dynamics of iterated kernels corresponding to a broad range of activation functions.
Newton-Type Methods for Non-Convex Optimization Under Inexact Hessian Information
In this light, we consider the canonical problem of finite-sum minimization, provide appropriate uniform and non-uniform sub-sampling strategies to construct such Hessian approximations, and obtain optimal iteration complexity for the corresponding sub-sampled trust-region and cubic regularization methods.
Newton-MR: Inexact Newton Method With Minimum Residual Sub-problem Solver
We consider a variant of inexact Newton Method, called Newton-MR, in which the least-squares sub-problems are solved approximately using Minimum Residual method.
Exchangeability and Kernel Invariance in Trained MLPs
In the analysis of machine learning models, it is often convenient to assume that the parameters are IID.
Implicit Langevin Algorithms for Sampling From Log-concave Densities
Theoretical and algorithmic properties of the resulting sampling methods for $ \theta \in [0, 1] $ and a range of step sizes are established.
Limit theorems for out-of-sample extensions of the adjacency and Laplacian spectral embeddings
In both cases, we prove that when the underlying graph is generated according to a latent space model called the random dot product graph, which includes the popular stochastic block model as a special case, an out-of-sample extension based on a least-squares objective obeys a central limit theorem about the true latent position of the out-of-sample vertex.
LSAR: Efficient Leverage Score Sampling Algorithm for the Analysis of Big Time Series Data
We first develop a new fast algorithm to estimate the leverage scores of an autoregressive (AR) model in big data regimes.
The reproducing Stein kernel approach for post-hoc corrected sampling
Stein importance sampling is a widely applicable technique based on kernelized Stein discrepancy, which corrects the output of approximate sampling algorithms by reweighting the empirical distribution of the samples.
Stochastic Normalizing Flows
We introduce stochastic normalizing flows, an extension of continuous normalizing flows for maximum likelihood estimation and variational inference (VI) using stochastic differential equations (SDEs).
Average-reward model-free reinforcement learning: a systematic review and literature mapping
Reinforcement learning is important part of artificial intelligence.
Non-PSD Matrix Sketching with Applications to Regression and Optimization
In this paper, we present novel dimensionality reduction methods for non-PSD matrices, as well as their ``square-roots", which involve matrices with complex entries.
Monotonicity and Double Descent in Uncertainty Estimation with Gaussian Processes
One prominent issue is the curse of dimensionality: it is commonly believed that the marginal likelihood should be reminiscent of cross-validation metrics and that both should deteriorate with larger input dimensions.
The Interpolating Information Criterion for Overparameterized Models
The problem of model selection is considered for the setting of interpolating estimators, where the number of model parameters exceeds the size of the dataset.
A PAC-Bayesian Perspective on the Interpolating Information Criterion
Deep learning is renowned for its theory-practice gap, whereby principled theory typically fails to provide much beneficial guidance for implementation in practice.
Non-Uniform Smoothness for Gradient Descent
The analysis of gradient descent-type methods typically relies on the Lipschitz continuity of the objective gradient.
SALSA: Sequential Approximate Leverage-Score Algorithm with Application in Analyzing Big Time Series Data
We demonstrate that, with high probability, the accuracy of SALSA's approximations is within $(1 + O({\varepsilon}))$ of the true leverage scores.
Bi-level Guided Diffusion Models for Zero-Shot Medical Imaging Inverse Problems
A central challenge in this approach, however, is how to guide an unconditional prediction to conform to the measurement information.
