Search Results for author:
Found 30 papers, 8 papers with code
Federated Sinkhorn
In this work we investigate the potential of solving the discrete Optimal Transport (OT) problem with entropy regularization in a federated learning setting.
"Cause" is Mechanistic Narrative within Scientific Domains: An Ordinary Language Philosophical Critique of "Causal Machine Learning"
Causal Learning has emerged as a major theme of research in statistics and machine learning in recent years, promising specific computational techniques to apply to datasets that reveal the true nature of cause and effect in a number of important domains.
Towards Diverse Device Heterogeneous Federated Learning via Task Arithmetic Knowledge Integration
To address these challenges, we introduce TAKFL, a novel KD-based framework that treats the knowledge transfer from each device prototype's ensemble as a separate task, independently distilling each to preserve its unique contributions and avoid dilution.
Dataset Distillation from First Principles: Integrating Core Information Extraction and Purposeful Learning
Dataset distillation (DD) is an increasingly important technique that focuses on constructing a synthetic dataset capable of capturing the core information in training data to achieve comparable performance in models trained on the latter.
Probabilistic Iterative Hard Thresholding for Sparse Learning
For statistical modeling wherein the data regime is unfavorable in terms of dimensionality relative to the sample size, finding hidden sparsity in the ground truth can be critical in formulating an accurate statistical model.
Empirical Bayes for Dynamic Bayesian Networks Using Generalized Variational Inference
This approach uses a recent development of Generalized Variational Inference, and indicates the potential of sampling the uncertainty of a mixture of DAG structures as well as a parameter posterior.
Learning Dynamic Bayesian Networks from Data: Foundations, First Principles and Numerical Comparisons
We present the analytical form of the models, with a comprehensive discussion on the interdependence between structure and weights in a DBN model and their implications for learning.
Group Distributionally Robust Dataset Distillation with Risk Minimization
The most popular methods for constructing the synthetic data rely on matching the convergence properties of training the model with the synthetic dataset and the training dataset.
Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents
This process allows local devices to train smaller surrogate models while enabling the training of a larger global model on the server, effectively minimizing resource utilization.
Efficient Dataset Distillation via Minimax Diffusion
Observing that key factors for constructing an effective surrogate dataset are representativeness and diversity, we design additional minimax criteria in the generative training to enhance these facets for the generated images of diffusion models.
Quantum Solutions to the Privacy vs. Utility Tradeoff
In this work, we propose a novel architecture (and several variants thereof) based on quantum cryptographic primitives with provable privacy and security guarantees regarding membership inference attacks on generative models.
A Stochastic-Gradient-based Interior-Point Algorithm for Solving Smooth Bound-Constrained Optimization Problems
A stochastic-gradient-based interior-point algorithm for minimizing a continuously differentiable objective function (that may be nonconvex) subject to bound constraints is presented, analyzed, and demonstrated through experimental results.
Riemannian Stochastic Approximation for Minimizing Tame Nonsmooth Objective Functions
In many learning applications, the parameters in a model are structurally constrained in a way that can be modeled as them lying on a Riemannian manifold.
When Do Curricula Work in Federated Learning?
Specifically, we aim to investigate how ordered learning principles can contribute to alleviating the heterogeneity effects in FL.
Jump-Diffusion Langevin Dynamics for Multimodal Posterior Sampling
Bayesian methods of sampling from a posterior distribution are becoming increasingly popular due to their ability to precisely display the uncertainty of a model fit.
Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence
In this paper, we focus on neural networks with two and three layers and provide a rigorous understanding of the properties of the solutions found by SHB: \emph{(i)} stability after dropping out part of the neurons, \emph{(ii)} connectivity along a low-loss path, and \emph{(iii)} convergence to the global optimum.
Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspaces
This small set of principal vectors is provided to the server so that the server can directly identify distribution similarities among the clients to form clusters.
Stochastic Langevin Differential Inclusions with Applications to Machine Learning
In this paper, we show some foundational results regarding the flow and asymptotic properties of Langevin-type Stochastic Differential Inclusions under assumptions appropriate to the machine-learning settings.
Scaling the Wild: Decentralizing Hogwild!-style Shared-memory SGD
Our scheme is based on the following algorithmic tools and features: (a) asynchronous local gradient updates on the shared-memory of workers, (b) partial backpropagation, and (c) non-blocking in-place averaging of the local models.
Randomized Algorithms for Monotone Submodular Function Maximization on the Integer Lattice
Optimization problems with set submodular objective functions have many real-world applications.
Mean-field Analysis of Piecewise Linear Solutions for Wide ReLU Networks
Understanding the properties of neural networks trained via stochastic gradient descent (SGD) is at the heart of the theory of deep learning.
Decentralized Bayesian Learning with Metropolis-Adjusted Hamiltonian Monte Carlo
Federated learning performed by a decentralized networks of agents is becoming increasingly important with the prevalence of embedded software on autonomous devices.
Trilevel and Multilevel Optimization using Monotone Operator Theory
We consider rather a general class of multi-level optimization problems, where a convex objective function is to be minimized subject to constraints of optimality of nested convex optimization problems.
Local SGD Meets Asynchrony
On the theoretical side, we show that this method guarantees ergodic convergence for non-convex objectives, and achieves the classic sublinear rate under standard assumptions.
Stochastic Gradient Langevin with Delayed Gradients
Stochastic Gradient Langevin Dynamics (SGLD) ensures strong guarantees with regards to convergence in measure for sampling log-concave posterior distributions by adding noise to stochastic gradient iterates.
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent
Our framework, called elastic consistency enables us to derive convergence bounds for a variety of distributed SGD methods used in practice to train large-scale machine learning models.
Asynchronous Stochastic Subgradient Methods for General Nonsmooth Nonconvex Optimization
This is all the more surprising since these objectives are the ones appearing in the training of deep neural networks.
Lifted Weight Learning of Markov Logic Networks Revisited
We study lifted weight learning of Markov logic networks.
Algorithms for solving optimization problems arising from deep neural net models: nonsmooth problems
Machine Learning models incorporating multiple layered learning networks have been seen to provide effective models for various classification problems.
Algorithms for solving optimization problems arising from deep neural net models: smooth problems
Machine Learning models incorporating multiple layered learning networks have been seen to provide effective models for various classification problems.
