Search Results for author:
Found 22 papers, 0 papers with code
Revisiting Agnostic PAC Learning
This simple algorithm is known to have an optimal error in terms of the VC-dimension of $\mathcal{H}$ and the number of samples $n$.
Majority-of-Three: The Simplest Optimal Learner?
Furthermore, we prove a near-optimal high-probability bound on this algorithm's error.
High-Probability Risk Bounds via Sequential Predictors
Online learning methods yield sequential regret bounds under minimal assumptions and provide in-expectation risk bounds for statistical learning.
Local Risk Bounds for Statistical Aggregation
In this paper, we revisit and tighten classical results in the theory of aggregation in the statistical setting by replacing the global complexity with a smaller, local one.
Optimal PAC Bounds Without Uniform Convergence
In this paper, we address this issue by providing optimal high probability risk bounds through a framework that surpasses the limitations of uniform convergence arguments.
Exploring Local Norms in Exp-concave Statistical Learning
We consider the problem of stochastic convex optimization with exp-concave losses using Empirical Risk Minimization in a convex class.
Statistically Optimal Robust Mean and Covariance Estimation for Anisotropic Gaussians
Assume that $X_{1}, \ldots, X_{N}$ is an $\varepsilon$-contaminated sample of $N$ independent Gaussian vectors in $\mathbb{R}^d$ with mean $\mu$ and covariance $\Sigma$.
Robustifying Markowitz
Markowitz mean-variance portfolios with sample mean and covariance as input parameters feature numerous issues in practice.
The One-Inclusion Graph Algorithm is not Always Optimal
In one of the first COLT open problems, Warmuth conjectured that this prediction strategy always implies an optimal high probability bound on the risk, and hence is also an optimal PAC algorithm.
A Regret-Variance Trade-Off in Online Learning
We prove that a variant of EWA either achieves a negative regret (i. e., the algorithm outperforms the best expert), or guarantees a $O(\log K)$ bound on both variance and regret.
Stability and Deviation Optimal Risk Bounds with Convergence Rate $O(1/n)$
The sharpest known high probability generalization bounds for uniformly stable algorithms (Feldman, Vondr\'{a}k, 2018, 2019), (Bousquet, Klochkov, Zhivotovskiy, 2020) contain a generally inevitable sampling error term of order $\Theta(1/\sqrt{n})$.
Distribution-Free Robust Linear Regression
In this distribution-free regression setting, we show that boundedness of the conditional second moment of the response given the covariates is a necessary and sufficient condition for achieving nontrivial guarantees.
Exponential Savings in Agnostic Active Learning through Abstention
We show that in pool-based active classification without assumptions on the underlying distribution, if the learner is given the power to abstain from some predictions by paying the price marginally smaller than the average loss $1/2$ of a random guess, exponential savings in the number of label requests are possible whenever they are possible in the corresponding realizable problem.
On Mean Estimation for Heteroscedastic Random Variables
We study the problem of estimating the common mean $\mu$ of $n$ independent symmetric random variables with different and unknown standard deviations $\sigma_1 \le \sigma_2 \le \cdots \le\sigma_n$.
Suboptimality of Constrained Least Squares and Improvements via Non-Linear Predictors
We study the problem of predicting as well as the best linear predictor in a bounded Euclidean ball with respect to the squared loss.
Proper Learning, Helly Number, and an Optimal SVM Bound
It has been recently shown by Hanneke (2016) that the optimal sample complexity of PAC learning for any VC class C is achieved by a particular improper learning algorithm, which outputs a specific majority-vote of hypotheses in C. This leaves the question of when this bound can be achieved by proper learning algorithms, which are restricted to always output a hypothesis from C. In this paper we aim to characterize the classes for which the optimal sample complexity can be achieved by a proper learning algorithm.
Robust $k$-means Clustering for Distributions with Two Moments
We consider the robust algorithms for the $k$-means clustering problem where a quantizer is constructed based on $N$ independent observations.
Fast Rates for Online Prediction with Abstention
In the setting of sequential prediction of individual $\{0, 1\}$-sequences with expert advice, we show that by allowing the learner to abstain from the prediction by paying a cost marginally smaller than $\frac 12$ (say, $0. 49$), it is possible to achieve expected regret bounds that are independent of the time horizon $T$.
Fast classification rates without standard margin assumptions
First, we consider classification with a reject option, namely Chow's reject option model, and show that by slightly lowering the impact of hard instances, a learning rate of order $O\left(\frac{d}{n}\log \frac{n}{d}\right)$ is always achievable in the agnostic setting by a specific learning algorithm.
Sharper bounds for uniformly stable algorithms
In a series of recent breakthrough papers by Feldman and Vondrak (2018, 2019), it was shown that the best known high probability upper bounds for uniformly stable learning algorithms due to Bousquet and Elisseef (2002) are sub-optimal in some natural regimes.
When are epsilon-nets small?
In many interesting situations the size of epsilon-nets depends only on $\epsilon$ together with different complexity measures.
Permutational Rademacher Complexity: a New Complexity Measure for Transductive Learning
This paper introduces a new complexity measure for transductive learning called Permutational Rademacher Complexity (PRC) and studies its properties.
