Search Results for author:
Found 29 papers, 1 papers with code
Optimal Sequential Maximization: One Interview is Enough!
To derive these results we consider a probabilistic setting where several candidates for a position are asked multiple questions with the goal of finding who has the highest probability of answering interview questions correctly.
Linear Regression using Heterogeneous Data Batches
A common approach assumes that the sources fall in one of several unknown subgroups, each with an unknown input distribution and input-output relationship.
TURF: A Two-factor, Universal, Robust, Fast Distribution Learning Algorithm
We derive a near-linear-time and essentially sample-optimal estimator that establishes $c_{t, d}=2$ for all $(t, d)\ne(1, 0)$.
Robust estimation algorithms don't need to know the corruption level
However, their vast majority approach optimal accuracy only when given a tight upper bound on the fraction of corrupt data.
Profile Entropy: A Fundamental Measure for the Learnability and Compressibility of Distributions
The profile of a sample is the multiset of its symbol frequencies.
Linear-Sample Learning of Low-Rank Distributions
Many latent-variable applications, including community detection, collaborative filtering, genomic analysis, and NLP, model data as generated by low-rank matrices.
Profile Entropy: A Fundamental Measure for the Learnability and Compressibility of Discrete Distributions
The profile of a sample is the multiset of its symbol frequencies.
A General Method for Robust Learning from Batches
In many applications, data is collected in batches, some of which are corrupt or even adversarial.
SURF: A Simple, Universal, Robust, Fast Distribution Learning Algorithm
Sample- and computationally-efficient distribution estimation is a fundamental tenet in statistics and machine learning.
Optimal Robust Learning of Discrete Distributions from Batches
Previous estimators for this setting ran in exponential time, and for some regimes required a suboptimal number of batches.
Unified Sample-Optimal Property Estimation in Near-Linear Time
We consider the fundamental learning problem of estimating properties of distributions over large domains.
The Broad Optimality of Profile Maximum Likelihood
In particular, for every alphabet size $k$ and desired accuracy $\varepsilon$: $\textbf{Distribution estimation}$ Under $\ell_1$ distance, PML yields optimal $\Theta(k/(\varepsilon^2\log k))$ sample complexity for sorted-distribution estimation, and a PML-based estimator empirically outperforms the Good-Turing estimator on the actual distribution; $\textbf{Additive property estimation}$ For a broad class of additive properties, the PML plug-in estimator uses just four times the sample size required by the best estimator to achieve roughly twice its error, with exponentially higher confidence; $\boldsymbol{\alpha}\textbf{-R\'enyi entropy estimation}$ For integer $\alpha>1$, the PML plug-in estimator has optimal $k^{1-1/\alpha}$ sample complexity; for non-integer $\alpha>3/4$, the PML plug-in estimator has sample complexity lower than the state of the art; $\textbf{Identity testing}$ In testing whether an unknown distribution is equal to or at least $\varepsilon$ far from a given distribution in $\ell_1$ distance, a PML-based tester achieves the optimal sample complexity up to logarithmic factors of $k$.
Data Amplification: A Unified and Competitive Approach to Property Estimation
We design the first unified, linear-time, competitive, property estimator that for a wide class of properties and for all underlying distributions uses just $2n$ samples to achieve the performance attained by the empirical estimator with $n\sqrt{\log n}$ samples.
Data Amplification: Instance-Optimal Property Estimation
For a large variety of distribution properties including four of the most popular ones and for every underlying distribution, they achieve the accuracy that the empirical-frequency plug-in estimators would attain using a logarithmic-factor more samples.
On Learning Markov Chains
We consider two problems related to the min-max risk (expected loss) of estimating an unknown $k$-state Markov chain from its $n$ sequential samples: predicting the conditional distribution of the next sample with respect to the KL-divergence, and estimating the transition matrix with respect to a natural loss induced by KL or a more general $f$-divergence measure.
The Limits of Maxing, Ranking, and Preference Learning
We present a comprehensive understanding of three important problems in PAC preference learning: maximum selection (maxing), ranking, and estimating all pairwise preference probabilities, in the adaptive setting.
The power of absolute discounting: all-dimensional distribution estimation
Minimax optimality is too pessimistic to remedy this issue.
Maxing and Ranking with Few Assumptions
PAC maximum selection (maxing) and ranking of $n$ elements via random pairwise comparisons have diverse applications and have been studied under many models and assumptions.
A Unified Maximum Likelihood Approach for Estimating Symmetric Properties of Discrete Distributions
Symmetric distribution properties such as support size, support coverage, entropy, and proximity to uniformity, arise in many applications.
Maximum Selection and Ranking under Noisy Comparisons
We consider $(\epsilon,\delta)$-PAC maximum-selection and ranking for general probabilistic models whose comparisons probabilities satisfy strong stochastic transitivity and stochastic triangle inequality.
Near-Optimal Smoothing of Structured Conditional Probability Matrices
Utilizing the structure of a probabilistic model can significantly increase its learning speed.
A Unified Maximum Likelihood Approach for Optimal Distribution Property Estimation
The advent of data science has spurred interest in estimating properties of distributions over large alphabets.
Competitive Distribution Estimation: Why is Good-Turing Good
Second, they estimate every distribution nearly as well as the best estimator designed with prior knowledge of the exact distribution, but as all natural estimators, restricted to assign the same probability to all symbols appearing the same number of times. Specifically, for distributions over $k$ symbols and $n$ samples, we show that for both comparisons, a simple variant of Good-Turing estimator is always within KL divergence of $(3+o(1))/n^{1/3}$ from the best estimator, and that a more involved estimator is within $\tilde{\mathcal{O}}(\min(k/n, 1/\sqrt n))$.
Estimating the number of unseen species: A bird in the hand is worth $\log n $ in the bush
We derive a class of estimators that $\textit{provably}$ predict $U$ not just for constant $t>1$, but all the way up to $t$ proportional to $\log n$.
Faster Algorithms for Testing under Conditional Sampling
There has been considerable recent interest in distribution-tests whose run-time and sample requirements are sublinear in the domain-size $k$.
Competitive Distribution Estimation
We also provide an estimator that runs in linear time and incurs competitive regret of $\tilde{\mathcal{O}}(\min(k/n, 1/\sqrt n))$, and show that for natural estimators this competitive regret is inevitable.
Estimating Renyi Entropy of Discrete Distributions
It was recently shown that estimating the Shannon entropy $H({\rm p})$ of a discrete $k$-symbol distribution ${\rm p}$ requires $\Theta(k/\log k)$ samples, a number that grows near-linearly in the support size.
Universal Compression of Envelope Classes: Tight Characterization via Poisson Sampling
The Poisson-sampling technique eliminates dependencies among symbol appearances in a random sequence.
Near-optimal-sample estimators for spherical Gaussian mixtures
For mixtures of any $k$ $d$-dimensional spherical Gaussians, we derive an intuitive spectral-estimator that uses $\mathcal{O}_k\bigl(\frac{d\log^2d}{\epsilon^4}\bigr)$ samples and runs in time $\mathcal{O}_{k,\epsilon}(d^3\log^5 d)$, both significantly lower than previously known.
