Search Results for author:
Found 23 papers, 5 papers with code
Group Distributionally Robust Dataset Distillation with Risk Minimization
However, targeting the training dataset must be thought of as auxiliary in the same sense that the training set is an approximate substitute for the population distribution, and the latter is the data of interest.
Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents
This process allows local devices to train smaller surrogate models while enabling the training of a larger global model on the server, effectively minimizing resource utilization.
Efficient Dataset Distillation via Minimax Diffusion
Observing that key factors for constructing an effective surrogate dataset are representativeness and diversity, we design additional minimax criteria in the generative training to enhance these facets for the generated images of diffusion models.
Quantum Solutions to the Privacy vs. Utility Tradeoff
In this work, we propose a novel architecture (and several variants thereof) based on quantum cryptographic primitives with provable privacy and security guarantees regarding membership inference attacks on generative models.
A Stochastic-Gradient-based Interior-Point Algorithm for Solving Smooth Bound-Constrained Optimization Problems
A stochastic-gradient-based interior-point algorithm for minimizing a continuously differentiable objective function (that may be nonconvex) subject to bound constraints is presented, analyzed, and demonstrated through experimental results.
Riemannian Stochastic Approximation for Minimizing Tame Nonsmooth Objective Functions
In many learning applications, the parameters in a model are structurally constrained in a way that can be modeled as them lying on a Riemannian manifold.
When Do Curricula Work in Federated Learning?
Specifically, we aim to investigate how ordered learning principles can contribute to alleviating the heterogeneity effects in FL.
Jump-Diffusion Langevin Dynamics for Multimodal Posterior Sampling
Bayesian methods of sampling from a posterior distribution are becoming increasingly popular due to their ability to precisely display the uncertainty of a model fit.
Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence
In this paper, we focus on neural networks with two and three layers and provide a rigorous understanding of the properties of the solutions found by SHB: \emph{(i)} stability after dropping out part of the neurons, \emph{(ii)} connectivity along a low-loss path, and \emph{(iii)} convergence to the global optimum.
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces
This small set of principal vectors is provided to the server so that the server can directly identify distribution similarities among the clients to form clusters.
Stochastic Langevin Differential Inclusions with Applications to Machine Learning
In this paper, we show some foundational results regarding the flow and asymptotic properties of Langevin-type Stochastic Differential Inclusions under assumptions appropriate to the machine-learning settings.
Scaling the Wild: Decentralizing Hogwild!-style Shared-memory SGD
Our scheme is based on the following algorithmic tools and features: (a) asynchronous local gradient updates on the shared-memory of workers, (b) partial backpropagation, and (c) non-blocking in-place averaging of the local models.
Randomized Algorithms for Monotone Submodular Function Maximization on the Integer Lattice
Optimization problems with set submodular objective functions have many real-world applications.
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks
Understanding the properties of neural networks trained via stochastic gradient descent (SGD) is at the heart of the theory of deep learning.
Decentralized Bayesian Learning with Metropolis-Adjusted Hamiltonian Monte Carlo
Federated learning performed by a decentralized networks of agents is becoming increasingly important with the prevalence of embedded software on autonomous devices.
Trilevel and Multilevel Optimization using Monotone Operator Theory
We consider rather a general class of multi-level optimization problems, where a convex objective function is to be minimized subject to constraints of optimality of nested convex optimization problems.
Local SGD Meets Asynchrony
On the theoretical side, we show that this method guarantees ergodic convergence for non-convex objectives, and achieves the classic sublinear rate under standard assumptions.
Stochastic Gradient Langevin with Delayed Gradients
Stochastic Gradient Langevin Dynamics (SGLD) ensures strong guarantees with regards to convergence in measure for sampling log-concave posterior distributions by adding noise to stochastic gradient iterates.
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent
Our framework, called elastic consistency enables us to derive convergence bounds for a variety of distributed SGD methods used in practice to train large-scale machine learning models.
Asynchronous Stochastic Subgradient Methods for General Nonsmooth Nonconvex Optimization
This is all the more surprising since these objectives are the ones appearing in the training of deep neural networks.
Lifted Weight Learning of Markov Logic Networks Revisited
We study lifted weight learning of Markov logic networks.
Algorithms for solving optimization problems arising from deep neural net models: smooth problems
Machine Learning models incorporating multiple layered learning networks have been seen to provide effective models for various classification problems.
Algorithms for solving optimization problems arising from deep neural net models: nonsmooth problems
Machine Learning models incorporating multiple layered learning networks have been seen to provide effective models for various classification problems.
