Search Results for author:
Found 17 papers, 1 papers with code
The Fundamental Limits of Least-Privilege Learning
The promise of least-privilege learning -- to find feature representations that are useful for a learning task but prevent inference of any sensitive information unrelated to this task -- is highly appealing.
Batch Universal Prediction
Large language models (LLMs) have recently gained much popularity due to their surprising ability at generating human-like English sentences.
Attention with Markov: A Framework for Principled Analysis of Transformers via Markov Chains
Inspired by the Markovianity of natural languages, we model the data as a Markovian source and utilize this framework to systematically study the interplay between the data-distributional properties, the transformer architecture, the learnt distribution, and the final model performance.
Fantastic Generalization Measures are Nowhere to be Found
We study the notion of a generalization bound being uniformly tight, meaning that the difference between the bound and the population loss is small for all learning algorithms and all population distributions.
Lower Bounds on the Bayesian Risk via Information Measures
We are thus able to provide estimator-independent impossibility results thanks to the Data-Processing Inequalities that divergences satisfy.
Generalization Error Bounds for Noisy, Iterative Algorithms via Maximal Leakage
We adopt an information-theoretic framework to analyze the generalization behavior of the class of iterative, noisy learning algorithms.
Finite Littlestone Dimension Implies Finite Information Complexity
We prove that every online learnable class of functions of Littlestone dimension $d$ admits a learning algorithm with finite information complexity.
From Generalisation Error to Transportation-cost Inequalities and Back
In this work, we connect the problem of bounding the expected generalisation error with transportation-cost inequalities.
A Johnson--Lindenstrauss Framework for Randomly Initialized CNNs
The celebrated Johnson--Lindenstrauss lemma answers this question for linear fully-connected neural networks (FNNs), stating that the geometry is essentially preserved.
A Johnson-Lindenstrauss Framework for Randomly Initialized CNNs
The celebrated Johnson-Lindenstrauss lemma answers this question for linear fully-connected neural networks (FNNs), stating that the geometry is essentially preserved.
Private Retrieval, Computing and Learning: Recent Progress and Future Challenges
Most of our lives are conducted in the cyberspace.
Learning, compression, and leakage: Minimising classification error via meta-universal compression principles
Learning and compression are driven by the common aim of identifying and exploiting statistical regularities in data, which opens the door for fertile collaboration between these areas.
Common Information Components Analysis
We give an information-theoretic interpretation of Canonical Correlation Analysis (CCA) via (relaxed) Wyner's common information.
Robust Generalization via $α$-Mutual Information
The aim of this work is to provide bounds connecting two probability measures of the same event using R\'enyi $\alpha$-Divergences and Sibson's $\alpha$-Mutual Information, a generalization of respectively the Kullback-Leibler Divergence and Shannon's Mutual Information.
Generalization Error Bounds Via Rényi-, $f$-Divergences and Maximal Leakage
In this work, the probability of an event under some joint distribution is bounded by measuring it with the product of the marginals instead (which is typically easier to analyze) together with a measure of the dependence between the two random variables.
A New Approach to Adaptive Data Analysis and Learning via Maximal Leakage
Our contribution consists in the introduction of a new approach, based on the concept of Maximal Leakage, an information-theoretic measure of leakage of information.
Locally Differentially-Private Randomized Response for Discrete Distribution Learning
We derive the respective normalized first-order terms of convergence (as $n\to\infty$), which for a given target privacy $\epsilon$ represent a rule-of-thumb factor by which the sample size must be augmented so as to achieve the same estimation accuracy as that of a non-randomizing channel.
