Search Results for author: Yury Polyanskiy

Found 20 papers, 7 papers with code

A mathematical perspective on Transformers

1 code implementation17 Dec 2023 Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, Philippe Rigollet

Transformers play a central role in the inner workings of large language models.

Kernel-Based Tests for Likelihood-Free Hypothesis Testing

1 code implementation NeurIPS 2023 Patrik Róbert Gerber, Tianze Jiang, Yury Polyanskiy, Rui Sun

Given $n$ observations from two balanced classes, consider the task of labeling an additional $m$ inputs that are known to all belong to \emph{one} of the two classes.

Binary Classification Two-sample testing

The emergence of clusters in self-attention dynamics

1 code implementation NeurIPS 2023 Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, Philippe Rigollet

Cluster locations are determined by the initial tokens, confirming context-awareness of representations learned by Transformers.

On Neural Architectures for Deep Learning-based Source Separation of Co-Channel OFDM Signals

1 code implementation11 Mar 2023 Gary C. F. Lee, Amir Weiss, Alejandro Lancho, Yury Polyanskiy, Gregory W. Wornell

We study the single-channel source separation problem involving orthogonal frequency-division multiplexing (OFDM) signals, which are ubiquitous in many modern-day digital communication systems.

Time Series Time Series Analysis

The Sample Complexity of Approximate Rejection Sampling with Applications to Smoothed Online Learning

no code implementations9 Feb 2023 Adam Block, Yury Polyanskiy

Suppose we are given access to $n$ independent samples from distribution $\mu$ and we wish to output one of them with the goal of making the output distributed as close as possible to a target distribution $\nu$.

Data-Driven Blind Synchronization and Interference Rejection for Digital Communication Signals

1 code implementation11 Sep 2022 Alejandro Lancho, Amir Weiss, Gary C. F. Lee, Jennifer Tang, Yuheng Bu, Yury Polyanskiy, Gregory W. Wornell

We study the potential of data-driven deep learning methods for separation of two communication signals from an observation of their mixture.

Exploiting Temporal Structures of Cyclostationary Signals for Data-Driven Single-Channel Source Separation

1 code implementation22 Aug 2022 Gary C. F. Lee, Amir Weiss, Alejandro Lancho, Jennifer Tang, Yuheng Bu, Yury Polyanskiy, Gregory W. Wornell

We study the problem of single-channel source separation (SCSS), and focus on cyclostationary signals, which are particularly suitable in a variety of application domains.

Sharp regret bounds for empirical Bayes and compound decision problems

no code implementations8 Sep 2021 Yury Polyanskiy, Yihong Wu

We show that for the Poisson model with compactly supported and subexponential priors, the optimal regret scales as $\Theta((\frac{\log n}{\log\log n})^2)$ and $\Theta(\log^3 n)$, respectively, both attained by the original estimator of Robbins.

Density Estimation

Intrinsic Dimension Estimation Using Wasserstein Distances

no code implementations8 Jun 2021 Adam Block, Zeyu Jia, Yury Polyanskiy, Alexander Rakhlin

It has long been thought that high-dimensional data encountered in many practical machine learning tasks have low-dimensional structure, i. e., the manifold hypothesis holds.

BIG-bench Machine Learning

Sequential prediction under log-loss and misspecification

no code implementations29 Jan 2021 Meir Feder, Yury Polyanskiy

The well-specified case corresponds to an additional assumption that the data-generating distribution belongs to the hypothesis class as well.

Density Estimation Model Selection

Stochastic block model entropy and broadcasting on trees with survey

no code implementations29 Jan 2021 Emmanuel Abbe, Elisabetta Cornacchia, Yuzhou Gu, Yury Polyanskiy

The limit of the entropy in the stochastic block model (SBM) has been characterized in the sparse regime for the special case of disassortative communities [COKPZ17] and for the classical case of assortative communities but in the dense regime [DAM16].

Probability Information Theory Information Theory

Self-regularizing Property of Nonparametric Maximum Likelihood Estimator in Mixture Models

no code implementations19 Aug 2020 Yury Polyanskiy, Yihong Wu

Notably, any such Gaussian mixture is statistically indistinguishable from a finite one with $O(\log n)$ components (and this is tight for certain mixtures).

Model Selection

Extrapolating the profile of a finite population

no code implementations21 May 2020 Soham Jana, Yury Polyanskiy, Yihong Wu

Nevertheless, we show that in the sublinear regime of $m =\omega(k/\log k)$, it is possible to consistently estimate in total variation the \emph{profile} of the population, defined as the empirical distribution of the sizes of each type, which determines many symmetric properties of the population.

The Information Bottleneck Problem and Its Applications in Machine Learning

no code implementations30 Apr 2020 Ziv Goldfeld, Yury Polyanskiy

The information bottleneck (IB) theory recently emerged as a bold information-theoretic paradigm for analyzing DL systems.

BIG-bench Machine Learning Dimensionality Reduction

Communication Complexity of Estimating Correlations

no code implementations25 Jan 2019 Uri Hadar, Jingbo Liu, Yury Polyanskiy, Ofer Shayevitz

Our results also imply an $\Omega(n)$ lower bound on the information complexity of the Gap-Hamming problem, for which we show a direct information-theoretic proof.

Estimating Information Flow in Deep Neural Networks

no code implementations12 Oct 2018 Ziv Goldfeld, Ewout van den Berg, Kristjan Greenewald, Igor Melnyk, Nam Nguyen, Brian Kingsbury, Yury Polyanskiy

We then develop a rigorous estimator for $I(X;T)$ in noisy DNNs and observe compression in various models.

Clustering

Information Storage in the Stochastic Ising Model

no code implementations8 May 2018 Ziv Goldfeld, Guy Bresler, Yury Polyanskiy

We first show that at zero temperature, order of $\sqrt{n}$ bits can be stored in the system indefinitely by coding over stable, striped configurations.

Information Theory Statistical Mechanics Information Theory

Sample complexity of population recovery

no code implementations18 Feb 2017 Yury Polyanskiy, Ananda Theertha Suresh, Yihong Wu

For noisy population recovery, the sharp sample complexity turns out to be more sensitive to dimension and scales as $\exp(\Theta(d^{1/3} \log^{2/3}(1/\delta)))$ except for the trivial cases of $\epsilon=0, 1/2$ or $1$.

Cannot find the paper you are looking for? You can Submit a new open access paper.