Search Results for author: Shay Moran

Found 46 papers, 2 papers with code

Active Learning with Label Comparisons

no code implementations10 Apr 2022 Gal Yona, Shay Moran, Gal Elidan, Amir Globerson

We show that there is a natural class where this approach is sub-optimal, and that there is a more comparison-efficient active learning scheme.

Active Learning

A Characterization of Multiclass Learnability

no code implementations3 Mar 2022 Nataly Brukhim, Daniel Carmon, Irit Dinur, Shay Moran, Amir Yehudayoff

This work resolves this problem: we characterize multiclass PAC learnability through the DS dimension, a combinatorial dimension defined by Daniely and Shalev-Shwartz (2014).

Learning Theory

Monotone Learning

no code implementations10 Feb 2022 Olivier Bousquet, Amit Daniely, Haim Kaplan, Yishay Mansour, Shay Moran, Uri Stemmer

Our transformation readily implies monotone learners in a variety of contexts: for example it extends Pestov's result to classification tasks with an arbitrary number of labels.


Multiclass Boosting and the Cost of Weak Learning

no code implementations NeurIPS 2021 Nataly Brukhim, Elad Hazan, Shay Moran, Indraneel Mukherjee, Robert E. Schapire

Here, we focus on an especially natural formulation in which the weak hypotheses are assumed to belong to an ''easy-to-learn'' base class, and the weak learner is an agnostic PAC learner for that class with respect to the standard classification loss.

Uniform Brackets, Containers, and Combinatorial Macbeath Regions

no code implementations19 Nov 2021 Kunal Dutta, Arijit Ghosh, Shay Moran

We study the connections between three seemingly different combinatorial structures - "uniform" brackets in statistics and probability theory, "containers" in online and distributed learning theory, and "combinatorial Macbeath regions", or Mnets in discrete and computational geometry.

Learning Theory

Towards a Unified Information-Theoretic Framework for Generalization

no code implementations NeurIPS 2021 Mahdi Haghifam, Gintare Karolina Dziugaite, Shay Moran, Daniel M. Roy

We further show that an inherent limitation of proper learning of VC classes contradicts the existence of a proper learner with constant CMI, and it implies a negative resolution to an open problem of Steinke and Zakynthinou (2020).

Generalization Bounds

Statistically Near-Optimal Hypothesis Selection

no code implementations17 Aug 2021 Olivier Bousquet, Mark Braverman, Klim Efremenko, Gillat Kol, Shay Moran

We derive an optimal $2$-approximation learning strategy for the Hypothesis Selection problem, outputting $q$ such that $\mathsf{TV}(p, q) \leq2 \cdot opt + \eps$, with a (nearly) optimal sample complexity of~$\tilde O(\log n/\epsilon^2)$.

Agnostic Online Learning and Excellent Sets

no code implementations12 Aug 2021 Maryanthe Malliaris, Shay Moran

We use algorithmic methods from online learning to revisit a key idea from the interaction of model theory and combinatorics, the existence of large "indivisible" sets, called "$\epsilon$-excellent," in $k$-edge stable graphs (equivalently, Littlestone classes).

online learning

A Theory of PAC Learnability of Partial Concept Classes

no code implementations18 Jul 2021 Noga Alon, Steve Hanneke, Ron Holzman, Shay Moran

In fact we exhibit easy-to-learn partial concept classes which provably cannot be captured by the traditional PAC theory.

Online Learning with Simple Predictors and a Combinatorial Characterization of Minimax in 0/1 Games

no code implementations2 Feb 2021 Steve Hanneke, Roi Livni, Shay Moran

More precisely, given any concept class C and any hypothesis class H, we provide nearly tight bounds (up to a log factor) on the optimal mistake bounds for online learning C using predictors from H. Our bound yields an exponential improvement over the previously best known bound by Chase and Freitag (2020).

online learning

Adversarial Laws of Large Numbers and Optimal Regret in Online Classification

no code implementations22 Jan 2021 Noga Alon, Omri Ben-Eliezer, Yuval Dagan, Shay Moran, Moni Naor, Eylon Yogev

Laws of large numbers guarantee that given a large enough sample from some population, the measure of any fixed sub-population is well-estimated by its frequency in the sample.

General Classification online learning

Synthetic Data Generators -- Sequential and Private

no code implementations NeurIPS 2020 Olivier Bousquet, Roi Livni, Shay Moran

We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient).

Synthetic Data Generation

A Theory of Universal Learning

no code implementations9 Nov 2020 Olivier Bousquet, Steve Hanneke, Shay Moran, Ramon van Handel, Amir Yehudayoff

How quickly can a given class of concepts be learned from examples?

On the Information Complexity of Proper Learners for VC Classes in the Realizable Case

no code implementations5 Nov 2020 Mahdi Haghifam, Gintare Karolina Dziugaite, Shay Moran, Daniel M. Roy

We provide a negative resolution to a conjecture of Steinke and Zakynthinou (2020a), by showing that their bound on the conditional mutual information (CMI) of proper learners of Vapnik--Chervonenkis (VC) classes cannot be improved from $d \log n +2$ to $O(d)$, where $n$ is the number of i. i. d.

Learning from Mixtures of Private and Public Populations

no code implementations NeurIPS 2020 Raef Bassily, Shay Moran, Anupama Nandi

Inspired by the above example, we consider a model in which the population $\mathcal{D}$ is a mixture of two sub-populations: a private sub-population $\mathcal{D}_{\sf priv}$ of private and sensitive data, and a public sub-population $\mathcal{D}_{\sf pub}$ of data with no privacy concerns.

A Limitation of the PAC-Bayes Framework

no code implementations NeurIPS 2020 Roi Livni, Shay Moran

PAC-Bayes is a useful framework for deriving generalization bounds which was introduced by McAllester ('98).

Generalization Bounds

Proper Learning, Helly Number, and an Optimal SVM Bound

no code implementations24 May 2020 Olivier Bousquet, Steve Hanneke, Shay Moran, Nikita Zhivotovskiy

It has been recently shown by Hanneke (2016) that the optimal sample complexity of PAC learning for any VC class C is achieved by a particular improper learning algorithm, which outputs a specific majority-vote of hypotheses in C. This leaves the question of when this bound can be achieved by proper learning algorithms, which are restricted to always output a hypothesis from C. In this paper we aim to characterize the classes for which the optimal sample complexity can be achieved by a proper learning algorithm.

Private Query Release Assisted by Public Data

no code implementations ICML 2020 Raef Bassily, Albert Cheu, Shay Moran, Aleksandar Nikolov, Jonathan Ullman, Zhiwei Steven Wu

In comparison, with only private samples, this problem cannot be solved even for simple query classes with VC-dimension one, and without any private samples, a larger public sample of size $d/\alpha^2$ is needed.

Closure Properties for Private Classification and Online Prediction

no code implementations10 Mar 2020 Noga Alon, Amos Beimel, Shay Moran, Uri Stemmer

Let~$\cH$ be a class of boolean functions and consider a {\it composed class} $\cH'$ that is derived from~$\cH$ using some arbitrary aggregation rule (for example, $\cH'$ may be the class of all 3-wise majority-votes of functions in $\cH$).

Classification General Classification +1

Online Agnostic Boosting via Regret Minimization

no code implementations NeurIPS 2020 Nataly Brukhim, Xinyi Chen, Elad Hazan, Shay Moran

Boosting is a widely used machine learning approach based on the idea of aggregating weak learning rules.

online learning

An Equivalence Between Private Classification and Online Prediction

no code implementations1 Mar 2020 Mark Bun, Roi Livni, Shay Moran

We prove that every concept class with finite Littlestone dimension can be learned by an (approximate) differentially-private algorithm.

Classification General Classification

Boosting Simple Learners

1 code implementation31 Jan 2020 Noga Alon, Alon Gonen, Elad Hazan, Shay Moran

(ii) Expressivity: Which tasks can be learned by boosting weak hypotheses from a bounded VC class?

Limits of Private Learning with Access to Public Data

no code implementations NeurIPS 2019 Noga Alon, Raef Bassily, Shay Moran

We consider learning problems where the training set consists of two types of examples: private and public.

Convex Set Disjointness, Distributed Learning of Halfspaces, and LP Feasibility

no code implementations8 Sep 2019 Mark Braverman, Gillat Kol, Shay Moran, Raghuvansh R. Saxena

For Convex Set Disjointness (and the equivalent task of distributed LP feasibility) we derive upper and lower bounds of $\tilde O(d^2\log n)$ and~$\Omega(d\log n)$.

Distributed Optimization

An adaptive nearest neighbor rule for classification

1 code implementation NeurIPS 2019 Akshay Balsubramani, Sanjoy Dasgupta, Yoav Freund, Shay Moran

We introduce a variant of the $k$-nearest neighbor classifier in which $k$ is chosen adaptively for each query, rather than supplied as a parameter.

Classification General Classification +1

Private Learning Implies Online Learning: An Efficient Reduction

no code implementations NeurIPS 2019 Alon Gonen, Elad Hazan, Shay Moran

We study the relationship between the notions of differentially private learning and online learning in games.

online learning

Private Center Points and Learning of Halfspaces

no code implementations27 Feb 2019 Amos Beimel, Shay Moran, Kobbi Nissim, Uri Stemmer

The building block for this learner is a differentially private algorithm for locating an approximate center point of $m>\mathrm{poly}(d, 2^{\log^*|X|})$ points -- a high dimensional generalization of the median function.

Learning to Screen

no code implementations NeurIPS 2019 Alon Cohen, Avinatan Hassidim, Haim Kaplan, Yishay Mansour, Shay Moran

(ii) In the second variant it is assumed that before the process starts, the algorithm has an access to a training set of $n$ items drawn independently from the same unknown distribution (e. g.\ data of candidates from previous recruitment seasons).

The Optimal Approximation Factor in Density Estimation

no code implementations10 Feb 2019 Olivier Bousquet, Daniel Kane, Shay Moran

We complement and extend this result by showing that: (i) the factor 3 can not be improved if one restricts the algorithm to output a density from $\mathcal{Q}$, and (ii) if one allows the algorithm to output arbitrary densities (e. g.\ a mixture of densities from $\mathcal{Q}$), then the approximation factor can be reduced to 2, which is optimal.

Density Estimation

Synthetic Data Generators: Sequential and Private

no code implementations9 Feb 2019 Olivier Bousquet, Roi Livni, Shay Moran

We study the sample complexity of private synthetic data generation over an unbounded sized class of statistical queries, and show that any class that is privately proper PAC learnable admits a private synthetic data generator (perhaps non-efficient).

Synthetic Data Generation

Unlabeled sample compression schemes and corner peelings for ample and maximum classes

no code implementations5 Dec 2018 Jérémie Chalopin, Victor Chepoi, Shay Moran, Manfred K. Warmuth

On the positive side we present a new construction of an unlabeled sample compression scheme for maximum classes.

On the Perceptron's Compression

no code implementations14 Jun 2018 Shay Moran, Ido Nachum, Itai Panasoff, Amir Yehudayoff

We study and provide exposition to several phenomena that are related to the perceptron's compression.

A Sauer-Shelah-Perles Lemma for Sumsets

no code implementations14 Jun 2018 Zeev Dvir, Shay Moran

We show that any family of subsets $A\subseteq 2^{[n]}$ satisfies $\lvert A\rvert \leq O\bigl(n^{\lceil{d}/{2}\rceil}\bigr)$, where $d$ is the VC dimension of $\{S\triangle T \,\vert\, S, T\in A\}$, and $\triangle$ is the symmetric difference operator.

Private PAC learning implies finite Littlestone dimension

no code implementations4 Jun 2018 Noga Alon, Roi Livni, Maryanthe Malliaris, Shay Moran

We show that every approximately differentially private learning algorithm (possibly improper) for a class $H$ with Littlestone dimension~$d$ requires $\Omega\bigl(\log^*(d)\bigr)$ examples.

On Communication Complexity of Classification Problems

no code implementations16 Nov 2017 Daniel M. Kane, Roi Livni, Shay Moran, Amir Yehudayoff

To naturally fit into the framework of learning theory, the players can send each other examples (as well as bits) where each example/bit costs one unit of communication.

Classification General Classification +1

A learning problem that is independent of the set theory ZFC axioms

no code implementations14 Nov 2017 Shai Ben-David, Pavel Hrubes, Shay Moran, Amir Shpilka, Amir Yehudayoff

We consider the following statistical estimation problem: given a family F of real valued functions over some domain X and an i. i. d.

General Classification

Learners that Use Little Information

no code implementations14 Oct 2017 Raef Bassily, Shay Moran, Ido Nachum, Jonathan Shafer, Amir Yehudayoff

We discuss an approach that allows us to prove upper bounds on the amount of information that algorithms reveal about their inputs, and also provide a lower bound by showing a simple concept class for which every (possibly randomized) empirical risk minimizer must reveal a lot of information.

Submultiplicative Glivenko-Cantelli and Uniform Convergence of Revenues

no code implementations NeurIPS 2017 Noga Alon, Moshe Babaioff, Yannai A. Gonczarowski, Yishay Mansour, Shay Moran, Amir Yehudayoff

In this work we derive a variant of the classic Glivenko-Cantelli Theorem, which asserts uniform convergence of the empirical Cumulative Distribution Function (CDF) to the CDF of the underlying distribution.

Near-optimal linear decision trees for k-SUM and related problems

no code implementations4 May 2017 Daniel M. Kane, Shachar Lovett, Shay Moran

We construct near optimal linear decision trees for a variety of decision problems in combinatorics and discrete geometry.

Active classification with comparison queries

no code implementations11 Apr 2017 Daniel M. Kane, Shachar Lovett, Shay Moran, Jiapeng Zhang

We identify a combinatorial dimension, called the \emph{inference dimension}, that captures the query complexity when each additional query is determined by $O(1)$ examples (such as comparison queries, each of which is determined by the two compared examples).

Active Learning Classification +1

Supervised learning through the lens of compression

no code implementations NeurIPS 2016 Ofir David, Shay Moran, Amir Yehudayoff

This work continues the study of the relationship between sample compression schemes and statistical learning, which has been mostly investigated within the framework of binary classification.

Twenty (simple) questions

no code implementations5 Nov 2016 Yuval Dagan, Yuval Filmus, Ariel Gabizon, Shay Moran

An optimal strategy for the "20 questions" game is given by a Huffman code for $\pi$: Bob's questions reveal the codeword for $x$ bit by bit.

On statistical learning via the lens of compression

no code implementations12 Oct 2016 Ofir David, Shay Moran, Amir Yehudayoff

(iv) A dichotomy for sample compression in multiclass categorization problems: If a non-trivial compression exists then a compression of logarithmic size exists.

Learning Theory

Labeled compression schemes for extremal classes

no code implementations30 May 2015 Shay Moran, Manfred K. Warmuth

We consider a generalization of maximum classes called extremal classes.

Sample compression schemes for VC classes

no code implementations24 Mar 2015 Shay Moran, Amir Yehudayoff

Sample compression schemes were defined by Littlestone and Warmuth (1986) as an abstraction of the structure underlying many learning algorithms.

Teaching and compressing for low VC-dimension

no code implementations22 Feb 2015 Shay Moran, Amir Shpilka, Avi Wigderson, Amir Yehudayoff

We further construct sample compression schemes of size $k$ for $C$, with additional information of $k \log(k)$ bits.

Cannot find the paper you are looking for? You can Submit a new open access paper.