Search Results for author: Steve Hanneke

Found 47 papers, 0 papers with code

Optimal Learners for Realizable Regression: PAC Learning and Online Learning

no code implementations7 Jul 2023 Idan Attias, Steve Hanneke, Alkis Kalavasis, Amin Karbasi, Grigoris Velegkas

Additionally, in the context of online learning we provide a dimension that characterizes the minimax instance optimal cumulative loss up to a constant factor and design an optimal online learner for realizable regression, thus resolving an open question raised by Daskalakis and Golowich in STOC '22.

PAC learning regression

Universal Rates for Multiclass Learning

no code implementations5 Jul 2023 Steve Hanneke, Shay Moran, Qian Zhang

Pseudo-cubes are a structure, rooted in the work of Daniely and Shalev-Shwartz (2014), and recently shown by Brukhim, Carmon, Dinur, Moran, and Yehudayoff (2022) to characterize PAC learnability (i. e., uniform rates) for multiclass classification.

Binary Classification

Adversarial Resilience in Sequential Prediction via Abstention

no code implementations22 Jun 2023 Surbhi Goel, Steve Hanneke, Shay Moran, Abhishek Shetty

We study the problem of sequential prediction in the stochastic setting with an adversary that is allowed to inject clean-label adversarial (or out-of-distribution) examples.

Limits of Model Selection under Transfer Learning

no code implementations29 Apr 2023 Steve Hanneke, Samory Kpotufe, Yasaman Mahdaviyeh

Theoretical studies on transfer learning or domain adaptation have so far focused on situations with a known hypothesis class or model; however in practice, some amount of model selection is usually involved, often appearing under the umbrella term of hyperparameter-tuning: for example, one may think of the problem of tuning for the right neural network architecture towards a target task, while leveraging data from a related source task.

Domain Adaptation Model Selection +1

Reliable Learning for Test-time Attacks and Distribution Shift

no code implementations6 Apr 2023 Maria-Florina Balcan, Steve Hanneke, Rattana Pukdee, Dravyansh Sharma

Machine learning algorithms are often used in environments which are not captured accurately even by the most carefully obtained training data, either due to the possibility of `adversarial' test-time attacks, or on account of `natural' distribution shift.

Multiclass Online Learning and Uniform Convergence

no code implementations30 Mar 2023 Steve Hanneke, Shay Moran, Vinod Raman, Unique Subedi, Ambuj Tewari

We argue that the best expert has regret at most Littlestone dimension relative to the best concept in the class.

Binary Classification

Optimal Prediction Using Expert Advice and Randomized Littlestone Dimension

no code implementations27 Feb 2023 Yuval Filmus, Steve Hanneke, Idan Mehalel, Shay Moran

We prove an analogous result for randomized learners: we show that the optimal expected mistake bound in learning a class $\mathcal{H}$ equals its randomized Littlestone dimension, which is the largest $d$ for which there exists a tree shattered by $\mathcal{H}$ whose average depth is $2d$.

Open-Ended Question Answering

Adversarial Rewards in Universal Learning for Contextual Bandits

no code implementations14 Feb 2023 Moise Blanchard, Steve Hanneke, Patrick Jaillet

We show that optimistic universal learning for contextual bandits with adversarial rewards is impossible in general, contrary to all previously studied settings in online learning -- including standard supervised learning.

Multi-Armed Bandits

Near-optimal learning with average Hölder smoothness

no code implementations12 Feb 2023 Steve Hanneke, Aryeh Kontorovich, Guy Kornowski

We consider both the realizable and the agnostic (noisy) regression settings, proving upper and lower risk bounds in terms of the average H\"older smoothness; these rates improve upon both previously known rates even in the special case of average Lipschitz smoothness.

Contextual Bandits and Optimistically Universal Learning

no code implementations31 Dec 2022 Moise Blanchard, Steve Hanneke, Patrick Jaillet

Lastly, we consider the case of added continuity assumptions on rewards and show that these lead to universal consistency for significantly larger classes of data-generating processes.

Multi-Armed Bandits

On Optimal Learning Under Targeted Data Poisoning

no code implementations6 Oct 2022 Steve Hanneke, Amin Karbasi, Mohammad Mahmoody, Idan Mehalel, Shay Moran

In this work we aim to characterize the smallest achievable error $\epsilon=\epsilon(\eta)$ by the learner in the presence of such an adversary in both realizable and agnostic settings.

Data Poisoning

Adversarially Robust Learning: A Generic Minimax Optimal Learner and Characterization

no code implementations15 Sep 2022 Omar Montasser, Steve Hanneke, Nathan Srebro

We present a minimax optimal learner for the problem of learning predictors robust to adversarial examples at test-time.

Fine-Grained Distribution-Dependent Learning Curves

no code implementations31 Aug 2022 Olivier Bousquet, Steve Hanneke, Shay Moran, Jonathan Shafer, Ilya Tolstikhin

We solve this problem in a principled manner, by introducing a combinatorial dimension called VCL that characterizes the best $d'$ for which $d'/n$ is a strong minimax lower bound.

Learning Theory PAC learning

Adversarially Robust PAC Learnability of Real-Valued Functions

no code implementations26 Jun 2022 Idan Attias, Steve Hanneke

We study robustness to test-time adversarial attacks in the regression setting with $\ell_p$ losses and arbitrary perturbation sets.


Universally Consistent Online Learning with Arbitrarily Dependent Responses

no code implementations11 Mar 2022 Steve Hanneke

This work provides an online learning rule that is universally consistent under processes on (X, Y) pairs, under conditions only on the X process.

Robustly-reliable learners under poisoning attacks

no code implementations8 Mar 2022 Maria-Florina Balcan, Avrim Blum, Steve Hanneke, Dravyansh Sharma

Remarkably, we provide a complete characterization of learnability in this setting, in particular, nearly-tight matching upper and lower bounds on the region that can be certified, as well as efficient algorithms for computing this region given an ERM oracle.

Data Poisoning

A Characterization of Semi-Supervised Adversarially-Robust PAC Learnability

no code implementations11 Feb 2022 Idan Attias, Steve Hanneke, Yishay Mansour

This shows that there is a significant benefit in semi-supervised robust learning even in the worst-case distribution-free model, and establishes a gap between the supervised and semi-supervised label complexities which is known not to hold in standard non-robust PAC learning.

PAC learning

Universal Online Learning with Unbounded Losses: Memory Is All You Need

no code implementations21 Jan 2022 Moise Blanchard, Romain Cosson, Steve Hanneke

We resolve an open problem of Hanneke on the subject of universally consistent online learning with non-i. i. d.

Learning Theory Memorization

Transductive Robust Learning Guarantees

no code implementations20 Oct 2021 Omar Montasser, Steve Hanneke, Nathan Srebro

We study the problem of adversarially robust learning in the transductive setting.

Open Problem: Is There an Online Learning Algorithm That Learns Whenever Online Learning Is Possible?

no code implementations20 Jul 2021 Steve Hanneke

This open problem asks whether there exists an online learning algorithm for binary classification that guarantees, for all target concepts, to make a sublinear number of mistakes, under only the assumption that the (possibly random) sequence of points X allows that such a learning algorithm can exist for that sequence.

Binary Classification

A Theory of PAC Learnability of Partial Concept Classes

no code implementations18 Jul 2021 Noga Alon, Steve Hanneke, Ron Holzman, Shay Moran

In fact we exhibit easy-to-learn partial concept classes which provably cannot be captured by the traditional PAC theory.

PAC learning

Robust learning under clean-label attack

no code implementations1 Mar 2021 Avrim Blum, Steve Hanneke, Jian Qian, Han Shao

We study the problem of robust learning under clean-label data-poisoning attacks, where the attacker injects (an arbitrary set of) correctly-labeled examples to the training set to fool the algorithm into making mistakes on specific test instances at test time.

Data Poisoning PAC learning

Adversarially Robust Learning with Unknown Perturbation Sets

no code implementations3 Feb 2021 Omar Montasser, Steve Hanneke, Nathan Srebro

We study the problem of learning predictors that are robust to adversarial examples with respect to an unknown perturbation set, relying instead on interaction with an adversarial attacker or access to attack oracles, examining different models for such interactions.

Online Learning with Simple Predictors and a Combinatorial Characterization of Minimax in 0/1 Games

no code implementations2 Feb 2021 Steve Hanneke, Roi Livni, Shay Moran

More precisely, given any concept class C and any hypothesis class H, we provide nearly tight bounds (up to a log factor) on the optimal mistake bounds for online learning C using predictors from H. Our bound yields an exponential improvement over the previously best known bound by Chase and Freitag (2020).

Stable Sample Compression Schemes: New Applications and an Optimal SVM Margin Bound

no code implementations9 Nov 2020 Steve Hanneke, Aryeh Kontorovich

We analyze a family of supervised learning algorithms based on sample compression schemes that are stable, in the sense that removing points from the training set which were not selected for the compression set does not alter the resulting classifier.

Generalization Bounds Open-Ended Question Answering

Reducing Adversarially Robust Learning to Non-Robust PAC Learning

no code implementations NeurIPS 2020 Omar Montasser, Steve Hanneke, Nathan Srebro

We study the problem of reducing adversarially robust learning to standard PAC learning, i. e. the complexity of learning adversarially robust predictors using access to only a black-box non-robust learner.

PAC learning

A No-Free-Lunch Theorem for MultiTask Learning

no code implementations29 Jun 2020 Steve Hanneke, Samory Kpotufe

A perplexing fact remains in the evolving theory on the subject: while we would hope for performance bounds that account for the contribution from multiple tasks, the vast majority of analyses result in bounds that improve at best in the number $n$ of samples per task, but most often do not improve in $N$.

Domain Adaptation

Proper Learning, Helly Number, and an Optimal SVM Bound

no code implementations24 May 2020 Olivier Bousquet, Steve Hanneke, Shay Moran, Nikita Zhivotovskiy

It has been recently shown by Hanneke (2016) that the optimal sample complexity of PAC learning for any VC class C is achieved by a particular improper learning algorithm, which outputs a specific majority-vote of hypotheses in C. This leaves the question of when this bound can be achieved by proper learning algorithms, which are restricted to always output a hypothesis from C. In this paper we aim to characterize the classes for which the optimal sample complexity can be achieved by a proper learning algorithm.

PAC learning

On the Value of Target Data in Transfer Learning

no code implementations NeurIPS 2019 Steve Hanneke, Samory Kpotufe

We aim to understand the value of additional labeled or unlabeled target data in transfer learning, for any given amount of source data; this is motivated by practical questions around minimizing sampling costs, whereby, target data is usually harder or costlier to acquire than source data, but can yield better accuracy.

Transfer Learning

Universal Bayes consistency in metric spaces

no code implementations24 Jun 2019 Steve Hanneke, Aryeh Kontorovich, Sivan Sabato, Roi Weiss

This is the first learning algorithm known to enjoy this property; by comparison, the $k$-NN classifier and its variants are not generally universally Bayes-consistent, except under additional structural assumptions, such as an inner product, a norm, finite dimension, or a Besicovitch-type property.

Agnostic Sample Compression for Linear Regression

no code implementations3 Oct 2018 Steve Hanneke, Aryeh Kontorovich, Menachem Sadigurschi

We show that for p in {1, infinity}, agnostic linear regression with $\ell_p$ loss admits a bounded sample compression scheme.

Open-Ended Question Answering regression

A New Lower Bound for Agnostic Learning with Sample Compression Schemes

no code implementations21 May 2018 Steve Hanneke, Aryeh Kontorovich

We establish a tight characterization of the worst-case rates for the excess risk of agnostic learning with sample compression schemes and for uniform convergence for agnostic sample compression schemes.

Sample Compression for Real-Valued Learners

no code implementations21 May 2018 Steve Hanneke, Aryeh Kontorovich, Menachem Sadigurschi

We give an algorithmically efficient version of the learner-to-compression scheme conversion in Moran and Yehudayoff (2016).

Open-Ended Question Answering regression

Actively Avoiding Nonsense in Generative Models

no code implementations20 Feb 2018 Steve Hanneke, Adam Kalai, Gautam Kamath, Christos Tzamos

A generative model may generate utter nonsense when it is fit to maximize the likelihood of observed data.

Testing Piecewise Functions

no code implementations23 Jun 2017 Steve Hanneke, Liu Yang

We also identify the optimal dependence on the number of pieces in the query complexity of passive testing in the special case of piecewise constant functions.

Learning Whenever Learning is Possible: Universal Learning under General Stochastic Processes

no code implementations5 Jun 2017 Steve Hanneke

We are then interested in the question of whether there exist learning rules guaranteed to be universally consistent given only the assumption that universally consistent learning is possible for the given data process.

Learning Theory

Learning with Changing Features

no code implementations29 Apr 2017 Amit Dhurandhar, Steve Hanneke, Liu Yang

In particular, we propose an approach to provably determine the time instant from which the new/changed features start becoming relevant with respect to an output variable in an agnostic (supervised) learning setting.

Change Point Detection

Statistical Learning under Nonstationary Mixing Processes

no code implementations26 Dec 2015 Steve Hanneke, Liu Yang

Under these conditions, we propose a learning method, and establish that for bounded VC subgraph classes, the cumulative excess risk grows sublinearly in the number of predictions, at a quantified rate.

General Classification

Refined Error Bounds for Several Learning Algorithms

no code implementations22 Dec 2015 Steve Hanneke

This article studies the achievable guarantees on the error rates of certain learning algorithms, with particular focus on refining logarithmic factors.

Active Learning

The Optimal Sample Complexity of PAC Learning

no code implementations2 Jul 2015 Steve Hanneke

This work establishes a new upper bound on the number of samples sufficient for PAC learning in the realizable case.

PAC learning

Learning with a Drifting Target Concept

no code implementations20 May 2015 Steve Hanneke, Varun Kanade, Liu Yang

Some of the results also describe an active learning variant of this setting, and provide bounds on the number of queries for the labels of points in the sequence sufficient to obtain the stated bounds on the error rates.

Active Learning

Bounds on the Minimax Rate for Estimating a Prior over a VC Class from Independent Learning Tasks

no code implementations20 May 2015 Liu Yang, Steve Hanneke, Jaime Carbonell

We study the optimal rates of convergence for estimating a prior distribution over a VC class from a sequence of independent data sets respectively labeled by independent target functions sampled from the prior.

Transfer Learning

Minimax Analysis of Active Learning

no code implementations3 Oct 2014 Steve Hanneke, Liu Yang

This work establishes distribution-free upper and lower bounds on the minimax label complexity of active learning with general hypothesis classes, under various noise models.

Active Learning

A Compression Technique for Analyzing Disagreement-Based Active Learning

no code implementations5 Apr 2014 Yair Wiener, Steve Hanneke, Ran El-Yaniv

We introduce a new and improved characterization of the label complexity of disagreement-based active learning, in which the leading quantity is the version space compression set size.

Active Learning

Surrogate Losses in Passive and Active Learning

no code implementations16 Jul 2012 Steve Hanneke, Liu Yang

Specifically, it presents an active learning algorithm based on an arbitrary classification-calibrated surrogate loss function, along with an analysis of the number of label requests sufficient for the classifier returned by the algorithm to achieve a given risk under the 0-1 loss.

Active Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.