Search Results for author:
Found 13 papers, 1 papers with code
Local Risk Bounds for Statistical Aggregation
In this paper, we revisit and tighten classical results in the theory of aggregation in the statistical setting by replacing the global complexity with a smaller, local one.
Universal coding, intrinsic volumes, and metric complexity
We study sequential probability assignment in the Gaussian setting, where the goal is to predict, or equivalently compress, a sequence of real-valued observations almost as well as the best Gaussian distribution with mean constrained to a given subset of $\mathbf{R}^n$.
An elementary analysis of ridge regression with random design
In this note, we provide an elementary analysis of the prediction error of ridge regression with random design.
Distribution-Free Robust Linear Regression
In this distribution-free regression setting, we show that boundedness of the conditional second moment of the response given the covariates is a necessary and sufficient condition for achieving nontrivial guarantees.
Regularized ERM on random subspaces
We study a natural extension of classical empirical risk minimization, where the hypothesis space is a random subspace of a given space.
Asymptotics of Ridge (less) Regression under General Source Condition
We analyze the prediction error of ridge regression in an asymptotic regime where the sample size and dimension go to infinity at a proportional rate.
An improper estimator with optimal excess risk in misspecified density estimation and logistic regression
On standard examples, this bound scales as $d/n$ with $d$ the model dimension and $n$ the sample size, and critically remains valid under model misspecification.
Exact minimax risk for linear least squares, and the lower tail of sample covariance matrices
We express the minimax risk in terms of the distribution of statistical leverage scores of individual samples, and deduce a minimax lower bound of $d/(n-d+1)$ for any covariate distribution, nearly matching the risk for Gaussian design.
AMF: Aggregated Mondrian Forests for Online Learning
Using a variant of the Context Tree Weighting algorithm, we show that it is possible to efficiently perform an exact aggregation over all prunings of the trees; in particular, this enables to obtain a truly online parameter-free algorithm which is competitive with the optimal pruning of the Mondrian tree, and thus adaptive to the unknown regularity of the regression function.
On the optimality of the Hedge algorithm in the stochastic regime
Moreover, our analysis exhibits qualitative differences with other variants of the Hedge algorithm, such as the fixed-horizon version (with constant learning rate) and the one based on the so-called "doubling trick", both of which fail to adapt to the easier stochastic setting.
Minimax optimal rates for Mondrian trees and forests
Our results include consistency and convergence rates for Mondrian Trees and Forests, that turn out to be minimax optimal on the set of $s$-H\"older function with $s \in (0, 1]$ (for trees and forests) and $s \in (1, 2]$ (for forests only), assuming a proper tuning of their complexity parameter in both cases.
Universal consistency and minimax rates for online Mondrian Forests
We establish the consistency of an algorithm of Mondrian Forests, a randomized classification algorithm that can be implemented online.
Efficient tracking of a growing number of experts
By contrast, designing strategies that both achieve a near-optimal regret and maintain a reasonable number of weights is highly non-trivial.
