Search Results for author:
Found 25 papers, 8 papers with code
Do we need rebalancing strategies? A theoretical and empirical study around SMOTE and its variants
Surprisingly, for most data sets, we observe that applying no rebalancing strategy is competitive in terms of predictive performances, with tuned random forests, logistic regression or LightGBM.
Random features models: a way to study the success of naive imputation
Constant (naive) imputation is still widely used in practice as this is a first easy-to-use technique to deal with missing data.
Sparse tree-based initialization for neural networks
Dedicated neural network (NN) architectures have been designed to handle specific data types (such as CNN for images or RNN for text), which ranks them among state-of-the-art methods for dealing with these data.
Minimax rate of consistency for linear models with missing values
Missing values arise in most real-world data sets due to the aggregation of multiple sources and intrinsically missing information (sensor failure, unanswered questions in surveys...).
What’s a good imputation to predict with missing values?
In fact, we show that on perfectly imputed data the best regression function will generally be discontinuous, which makes it hard to learn.
What's a good imputation to predict with missing values?
In fact, we show that on perfectly imputed data the best regression function will generally be discontinuous, which makes it hard to learn.
SHAFF: Fast and consistent SHApley eFfect estimates via random Forests
Interpretability of learning algorithms is crucial for applications involving critical decisions, and variable importance is one of the main interpretation tools.
MDA for random forests: inconsistency, and a practical solution via the Sobol-MDA
Variable importance measures are the main tools to analyze the black-box mechanisms of random forests.
NeuMiss networks: differentiable programming for supervised learning with missing values.
We provide an upper bound on the Bayes risk of NeuMiss networks, and show that they have good predictive accuracy with both a number of parameters and a computational complexity independent of the number of missing data patterns.
Analyzing the tree-layer structure of Deep Forests
Random forests on the one hand, and neural networks on the other hand, have met great success in the machine learning community for their predictive performance.
NeuMiss networks: differentiable programming for supervised learning with missing values
We provide an upper bound on the Bayes risk of NeuMiss networks, and show that they have good predictive accuracy with both a number of parameters and a computational complexity independent of the number of missing data patterns.
Interpretable Random Forests via Rule Extraction
We introduce SIRUS (Stable and Interpretable RUle Set) for regression, a stable rule learning algorithm which takes the form of a short and simple list of rules.
Linear predictor on linearly-generated data with missing values: non consistency and solutions
In the particular Gaussian case, it can be written as a linear function of multiway interactions between the observed data and the various missing-value indicators.
Trees, forests, and impurity-based variable importance
We prove that if input variables are independent and in absence of interactions, MDI provides a variance decomposition of the output, where the contribution of each variable is clearly identified.
SIRUS: Stable and Interpretable RUle Set for Classification
State-of-the-art learning algorithms, such as random forests or neural networks, are often qualified as "black-boxes" because of the high number and complexity of operations involved in their prediction mechanism.
AMF: Aggregated Mondrian Forests for Online Learning
Using a variant of the Context Tree Weighting algorithm, we show that it is possible to efficiently perform an exact aggregation over all prunings of the trees; in particular, this enables to obtain a truly online parameter-free algorithm which is competitive with the optimal pruning of the Mondrian tree, and thus adaptive to the unknown regularity of the regression function.
On the consistency of supervised learning with missing values
A striking result is that the widely-used method of imputing with a constant, such as the mean prior to learning is consistent when missing values are not informative.
Minimax optimal rates for Mondrian trees and forests
Our results include consistency and convergence rates for Mondrian Trees and Forests, that turn out to be minimax optimal on the set of $s$-H\"older function with $s \in (0, 1]$ (for trees and forests) and $s \in (1, 2]$ (for forests only), assuming a proper tuning of their complexity parameter in both cases.
Universal consistency and minimax rates for online Mondrian Forests
We establish the consistency of an algorithm of Mondrian Forests, a randomized classification algorithm that can be implemented online.
Neural Random Forests
Given an ensemble of randomized regression trees, it is possible to restructure them as a collection of multilayered neural networks with particular connection weights.
Impact of subsampling and pruning on random forests
Random forests are ensemble learning methods introduced by Breiman (2001) that operate by averaging several decision trees built on a randomly selected subspace of the data set.
Statistics Theory Statistics Theory
A Random Forest Guided Tour
The random forest algorithm, proposed by L. Breiman in 2001, has been extremely successful as a general-purpose classification and regression method.
Random forests and kernel methods
In particular, we show that by slightly modifying their definition, random forests can be rewrit-ten as kernel methods (called KeRF for Kernel based on Random Forests) which are more interpretable and easier to analyze.
Statistics Theory Statistics Theory
On the asymptotics of random forests
The last decade has witnessed a growing interest in random forest models which are recognized to exhibit good practical performance, especially in high-dimensional settings.
Statistics Theory Statistics Theory
Consistency of random forests
What has greatly contributed to the popularity of forests is the fact that they can be applied to a wide range of prediction problems and have few parameters to tune.
