Search Results for author:
Found 20 papers, 6 papers with code
Minimum Entropy Coupling with Bottleneck
This paper investigates a novel lossy compression framework operating under logarithmic loss, designed to handle situations where the reconstruction distribution diverges from the source distribution.
Multi-Draft Speculative Sampling: Canonical Architectures and Theoretical Limits
In this work we show that the optimal scheme can be decomposed into a two-step solution: in the first step an importance sampling (IS) type scheme is used to select one intermediate token; in the second step (single-draft) speculative sampling is applied to generate the output token.
Robust Federated Finetuning of Foundation Models via Alternating Minimization of LoRA
Parameter-Efficient Fine-Tuning (PEFT) has risen as an innovative training strategy that updates only a select few model parameters, significantly lowering both computational and memory demands.
Rate-Distortion-Perception Tradeoff Based on the Conditional-Distribution Perception Measure
We derive the RDP function for vector Gaussian sources and propose a waterfilling type solution.
Random Edge Coding: One-Shot Bits-Back Coding of Large Labeled Graphs
We present a one-shot method for compressing large labeled graphs called Random Edge Coding.
Quadratic Functional Encryption for Secure Training in Vertical Federated Learning
Vertical federated learning (VFL) enables the collaborative training of machine learning (ML) models in settings where the data is distributed amongst multiple parties who wish to protect the privacy of their individual data.
Sequential Gradient Coding For Straggler Mitigation
In our second scheme, which constitutes our main contribution, we apply GC to a subset of the tasks and repetition for the remainder of the tasks.
Variational Model Inversion Attacks
In this work, we provide a probabilistic interpretation of model inversion attacks, and formulate a variational objective that accounts for both diversity and accuracy.
Compressing Multisets with Large Alphabets using Bits-Back Coding
Current methods which compress multisets at an optimal rate have computational complexity that scales linearly with alphabet size, making them too slow to be practical in many real-world settings.
Regularized Classification-Aware Quantization
Our method performs well on unseen data, and is faster than previous methods proportional to a quadratic term of the dataset size.
Universal Rate-Distortion-Perception Representations for Lossy Compression
In the context of lossy compression, Blau & Michaeli (2019) adopt a mathematical notion of perceptual quality and define the information rate-distortion-perception function, generalizing the classical rate-distortion tradeoff.
On the Generalization of Stochastic Gradient Descent with Momentum
While momentum-based methods, in conjunction with stochastic gradient descent (SGD), are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods.
Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding
Naively applied, our schemes would require more initial bits than the standard bits-back coder, but we show how to drastically reduce this additional cost with couplings in the latent space.
Coded Sequential Matrix Multiplication For Straggler Mitigation
Our first scheme is a modification of the polynomial coding scheme introduced by Yu et al. and places no assumptions on the straggler model.
Time-Resolved fMRI Shared Response Model using Gaussian Process Factor Analysis
Some recent work has implemented probabilistic models to extract a shared representation in task fMRI.
Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms
Finally, we apply these bounds to the study of Langevin dynamics algorithm, showing that conditioning on the super sample allows us to exploit information in the optimization trajectory to obtain tighter bounds based on hypothesis tests.
Sequential Classification with Empirically Observed Statistics
Motivated by real-world machine learning applications, we consider a statistical classification task in a sequential setting where test samples arrive sequentially.
Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates
In this work, we improve upon the stepwise analysis of noisy iterative learning algorithms initiated by Pensia, Jog, and Loh (2018) and recently extended by Bu, Zou, and Veeravalli (2019).
Stability of Stochastic Gradient Method with Momentum for Strongly Convex Loss Functions
While momentum-based methods, in conjunction with the stochastic gradient descent, are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods.
On the Generalization of Stochastic Gradient Descent with Momentum
While momentum-based accelerated variants of stochastic gradient descent (SGD) are widely used when training machine learning models, there is little theoretical understanding on the generalization error of such methods.
