Search Results for author:
Found 33 papers, 11 papers with code
On Sampling Strategies for Spectral Model Sharding
The problem of heterogeneous clients in federated learning has recently drawn a lot of attention.
Multi-Draft Speculative Sampling: Canonical Architectures and Theoretical Limits
In this work we show that the optimal scheme can be decomposed into a two-step solution: in the first step an importance sampling (IS) type scheme is used to select one intermediate token; in the second step (single-draft) speculative sampling is applied to generate the output token.
Variational Learning ISTA
Then, we propose to learn a distribution over dictionaries via a variational approach, dubbed Variational Learning ISTA (VLISTA).
Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data
The proliferation of edge devices has brought Federated Learning (FL) to the forefront as a promising paradigm for decentralized and collaborative model training while preserving the privacy of clients' data.
A Mutual Information Perspective on Federated Contrastive Learning
Along with the proposed SimCLR extensions, we also study how different sources of non-i. i. d.-ness can impact the performance of federated unsupervised learning through global mutual information maximization; we find that a global objective is beneficial for some sources of non-i. i. d.-ness but can be detrimental for others.
An Information Theoretic Perspective on Conformal Prediction
Conformal Prediction (CP) is a distribution-free uncertainty estimation framework that constructs prediction sets guaranteed to contain the true answer with a user-specified probability.
DNA: Differentially private Neural Augmentation for contact tracing
The COVID19 pandemic had enormous economic and societal consequences.
InterroGate: Learning to Share, Specialize, and Prune Representations for Multi-task Learning
In this work, we propose \textit{InterroGate}, a novel multi-task learning (MTL) architecture designed to mitigate task interference while optimizing inference computational efficiency.
Protect Your Score: Contact Tracing With Differential Privacy Guarantees
The pandemic in 2020 and 2021 had enormous economic and societal consequences, and studies show that contact tracing algorithms can be key in the early containment of the virus.
Hyperparameter Optimization through Neural Network Partitioning
Well-tuned hyperparameters are crucial for obtaining good generalization behavior in neural networks.
Quantization Robust Federated Learning for Efficient Inference on Heterogeneous Devices
We perform extensive experiments on standard FL benchmarks to evaluate our proposed FedAvg variants for quantization robustness and provide a convergence analysis for our Quantization-Aware variants in FL.
An Expectation-Maximization Perspective on Federated Learning
Federated learning describes the distributed training of models across multiple clients while keeping the data private on-device.
DP-REC: Private & Communication-Efficient Federated Learning
Privacy and communication efficiency are important challenges in federated training of neural networks, and combining them is still an open problem.
Federated Mixture of Experts
Federated learning (FL) has emerged as the predominant approach for collaborative training of neural network models across multiple users, without the need to gather the data at a central location.
Federated Learning of User Verification Models Without Sharing Embeddings
We consider the problem of training User Verification (UV) models in federated setting, where each user has access to the data of only one class and user embeddings cannot be shared with the server or other users.
Secure Federated Learning of User Verification Models
We consider the problem of training User Verification (UV) models in federated setup, where the conventional loss functions are not applicable due to the constraints that each user has access to the data of only one class and user embeddings cannot be shared with the server or other users.
Federated Averaging as Expectation Maximization
Federated averaging (FedAvg), despite its simplicity, has been the main approach in training neural networks in the federated learning setting.
Improving Fair Predictions Using Variational Inference In Causal Models
Recent work on fairness metrics shows the need for causal reasoning in fairness constraints.
Federated Learning of User Authentication Models
In this paper, we propose Federated User Authentication (FedUA), a framework for privacy-preserving training of UA models.
Bayesian Bits: Unifying Quantization and Pruning
We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization.
Up or Down? Adaptive Rounding for Post-Training Quantization
In this paper, we propose AdaRound, a better weight-rounding mechanism for post-training quantization that adapts to the data and the task loss.
Gradient $\ell_1$ Regularization for Quantization Robustness
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization.
The Functional Neural Process
We present a new family of exchangeable stochastic processes, the Functional Neural Processes (FNPs).
DIVA: Domain Invariant Variational Autoencoders
We consider the problem of domain generalization, namely, how to learn representations given data from a set of domains that generalize to data from a previously unseen domain.
DIVA: Domain Invariant Variational Autoencoder
We consider the problem of domain generalization, namely, how to learn representations given data from a set of domains that generalize to data from a previously unseen domain.
Relaxed Quantization for Discretized Neural Networks
Neural network quantization has become an important research area due to its great impact on deployment of large models on resource constrained devices.
Learning Sparse Neural Networks through L_0 Regularization
We further propose the \emph{hard concrete} distribution for the gates, which is obtained by ``stretching'' a binary concrete distribution and then transforming its samples with a hard-sigmoid.
Learning Sparse Neural Networks through $L_0$ Regularization
We further propose the \emph{hard concrete} distribution for the gates, which is obtained by "stretching" a binary concrete distribution and then transforming its samples with a hard-sigmoid.
Causal Effect Inference with Deep Latent-Variable Models
Learning individual-level causal effects from observational data, such as inferring the most effective medication for a specific patient, is a problem of growing importance for policy makers.
Ranked #9 on
Causal Inference
on IHDP
Bayesian Compression for Deep Learning
Compression and computational efficiency in deep learning have become a problem of great significance.
Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
We reinterpret multiplicative noise in neural networks as auxiliary random variables that augment the approximate posterior in a variational setting for Bayesian neural networks.
Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors
We introduce a variational Bayesian neural network where the parameters are governed via a probability distribution on random matrices.
The Variational Fair Autoencoder
We investigate the problem of learning representations that are invariant to certain nuisance or sensitive factors of variation in the data while retaining as much of the remaining information as possible.
Ranked #4 on
Sentiment Analysis
on Multi-Domain Sentiment Dataset
