Search Results for author:
Found 29 papers, 14 papers with code
Comparing EM with GD in Mixture Models of Two Components
In the case of mixtures of Bernoullis, we find that there exist one-cluster regions that are stable for GD and therefore trap GD, but those regions are unstable for EM, allowing EM to escape.
Convergence of Gradient Methods on Bilinear Zero-Sum Games
Min-max formulations have attracted great attention in the ML community due to the rise of deep generative models and adversarial methods, while understanding the dynamics of gradient algorithms for solving such formulations has remained a grand challenge.
Pressure-driven switching of magnetism in layered CrCl3
Layered transition-metal compounds with controllable magnetic behaviors provide many fascinating opportunities for the fabrication of high-performance magneto-electric and spintronic devices.
Materials Science
Optimality and Stability in Non-Convex Smooth Games
Convergence to a saddle point for convex-concave functions has been studied for decades, while recent years has seen a surge of interest in non-convex (zero-sum) smooth games, motivated by their recent wide applications.
Federated Learning Meets Multi-objective Optimization
Federated learning has emerged as a promising, massively distributed way to train a joint deep model over large amounts of edge devices while keeping private user data strictly on device.
Newton-type Methods for Minimax Optimization
We prove their local convergence at strict local minimax points, which are surrogates of global solutions.
f-Domain-Adversarial Learning: Theory and Algorithms for Unsupervised Domain Adaptation with Neural Networks
We provide empirical results for several f-divergences and show that some, not considered previously in domain-adversarial learning, achieve state-of-the-art results in practice.
Quantifying and Improving Transferability in Domain Generalization
We then prove that our transferability can be estimated with enough samples and give a new upper bound for the target error based on our transferability.
f-Domain-Adversarial Learning: Theory and Algorithms
Unsupervised domain adaptation is used in many machine learning applications where, during training, a model has access to unlabeled data in the target domain, and a related labeled dataset.
$f$-Mutual Information Contrastive Learning
Self-supervised contrastive learning is an emerging field due to its power in providing good data representations.
Proportional Fairness in Federated Learning
With the increasingly broad deployment of federated learning (FL) systems in the real world, it is critical but challenging to ensure fairness in FL, i. e. reasonably satisfactory performances for each of the numerous diverse clients.
Domain Adversarial Training: A Game Perspective
Defining optimal solutions in domain-adversarial training as a local Nash equilibrium, we show that gradient descent in domain-adversarial training can violate the asymptotic convergence guarantees of the optimizer, oftentimes hindering the transfer performance.
Federated Bayesian Neural Regression: A Scalable Global Federated Gaussian Process
We present Federated Bayesian Neural Regression (FedBNR), an algorithm that learns a scalable stand-alone global federated GP that respects clients' privacy.
Mitigating Data Heterogeneity in Federated Learning with Data Augmentation
Federated Learning (FL) is a prominent framework that enables training a centralized model while securing user privacy by fusing local, decentralized models.
Robust One Round Federated Learning with Predictive Space Bayesian Inference
In contrast, our method performs the aggregation on the predictive posteriors, which are typically easier to approximate owing to the low-dimensionality of the output space.
DP$^2$-VAE: Differentially Private Pre-trained Variational Autoencoders
Similar to other differentially private (DP) learners, the major challenge for DPGM is also how to achieve a subtle balance between utility and privacy.
Label Alignment Regularization for Distribution Shift
Recent work has highlighted the label alignment property (LAP) in supervised learning, where the vector of all labels in the dataset is mostly in the span of the top few singular vectors of the data matrix.
Private GANs, Revisited
We show that the canonical approach for training differentially private GANs -- updating the discriminator with differentially private stochastic gradient descent (DPSGD) -- can yield significantly improved results after modifications to training.
Mathematical Challenges in Deep Learning
Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012.
Understanding the Role of Layer Normalization in Label-Skewed Federated Learning
In this work, we reveal the profound connection between layer normalization and the label shift problem in federated learning.
Understanding Hessian Alignment for Domain Generalization
We validate the OOD generalization ability of proposed methods in different scenarios, including transferability, severe correlation shift, label shift and diversity shift.
Preventing Arbitrarily High Confidence on Far-Away Data in Point-Estimated Discriminative Neural Networks
For instance, ReLU networks - a popular class of neural network architectures - have been shown to almost always yield high confidence predictions when the test data are far away from the training set, even when they are trained with OOD data.
Cross Domain Generative Augmentation: Domain Generalization with Latent Diffusion Models
The VRM reduces the estimation error in ERM by replacing the point-wise kernel estimates with a more precise estimation of true data distribution that reduces the gap between data points \textbf{within each domain}.
Calibrated One Round Federated Learning with Bayesian Inference in the Predictive Space
To improve scalability for larger models, one common Bayesian approach is to approximate the global predictive posterior by multiplying local predictive posteriors.
DFML: Decentralized Federated Mutual Learning
In the realm of real-world devices, centralized servers in Federated Learning (FL) present challenges including communication bottlenecks and susceptibility to a single point of failure.
Parametric Feature Transfer: One-shot Federated Learning with Foundation Models
This paper introduces FedPFT (Federated Learning with Parametric Feature Transfer), a methodology that harnesses the transferability of foundation models to enhance both accuracy and communication efficiency in one-shot FL.
Robust Multi-Task Learning with Excess Risks
To overcome this limitation, we propose Multi-Task Learning with Excess Risks (ExcessMTL), an excess risk-based task balancing method that updates the task weights by their distances to convergence instead.
$f$-MICL: Understanding and Generalizing InfoNCE-based Contrastive Learning
In self-supervised contrastive learning, a widely-adopted objective function is InfoNCE, which uses the heuristic cosine similarity for the representation comparison, and is closely related to maximizing the Kullback-Leibler (KL)-based mutual information.
Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy?
Parameter-efficient fine-tuning stands as the standard for efficiently fine-tuning large language and vision models on downstream tasks.
