Search Results for author:
Found 24 papers, 3 papers with code
Heavy hitters via cluster-preserving clustering
Our main innovation is an efficient reduction from the heavy hitters to a clustering problem in which each heavy hitter is encoded as some form of noisy spectral cluster in a much bigger graph, and the goal is to identify every cluster.
Fast Exact k-Means, k-Medians and Bregman Divergence Clustering in 1D
We present all the existing work that had been overlooked and compare the various solutions theoretically.
Predicting Positive and Negative Links with Noisy Queries: Theory & Practice
The {\em edge sign prediction problem} aims to predict whether an interaction between a pair of nodes will be positive or negative.
Fully Understanding the Hashing Trick
We settle this question by giving tight asymptotic bounds on the exact tradeoff between the central parameters, thus providing a complete understanding of the performance of feature hashing.
The Query Complexity of a Permutation-Based Variant of Mastermind
We study the query complexity of a permutation-based variant of the guessing game Mastermind.
Optimal Minimal Margin Maximization with Boosting
A common goal in a long line of research, is to maximize the smallest margin using as few base hypotheses as possible, culminating with the AdaBoostV algorithm by (R{\"a}tsch and Warmuth [JMLR'04]).
Optimal Learning of Joint Alignments with a Faulty Oracle
The goal is to recover $n$ discrete variables $g_i \in \{0, \ldots, k-1\}$ (up to some global offset) given noisy observations of a set of their pairwise differences $\{(g_i - g_j) \bmod k\}$; specifically, with probability $\frac{1}{k}+\delta$ for some $\delta > 0$ one obtains the correct answer, and with the remaining probability one obtains a uniformly random incorrect answer.
Margin-Based Generalization Lower Bounds for Boosted Classifiers
To date, the strongest known generalization (upper bound) is the $k$th margin bound of Gao and Zhou (2013).
Near-Tight Margin-Based Generalization Bounds for Support Vector Machines
Support Vector Machines (SVMs) are among the most fundamental tools for binary classification.
Margins are Insufficient for Explaining Gradient Boosting
We then explain the short comings of the $k$'th margin bound and prove a stronger and more refined margin-based generalization bound for boosted classifiers that indeed succeeds in explaining the performance of modern gradient boosters.
CountSketches, Feature Hashing and the Median of Three
For $t > 1$, the estimator takes the median of $2t-1$ independent estimates, and the probability that the estimate is off by more than $2 \|v\|_2/\sqrt{s}$ is exponentially small in $t$.
Compression Implies Generalization
The framework is simple and powerful enough to extend the generalization bounds by Arora et al. to also hold for the original network.
Optimality of the Johnson-Lindenstrauss Dimensionality Reduction for Practical Measures
They provided upper bounds on its quality for a wide range of practical measures and showed that indeed these are best possible in many cases.
Towards Optimal Lower Bounds for k-median and k-means Coresets
Given a set of points in a metric space, the $(k, z)$-clustering problem consists of finding a set of $k$ points called centers, such that the sum of distances raised to the power of $z$ of every data point to its closest center is minimized.
The Fast Johnson-Lindenstrauss Transform is Even Faster
In this work, we give a surprising new analysis of the Fast JL transform, showing that the $k \ln^2 n$ term in the embedding time can be improved to $(k \ln^2 n)/\alpha$ for an $\alpha = \Omega(\min\{\varepsilon^{-1}\ln(1/\varepsilon), \ln n\})$.
Optimal Weak to Strong Learning
The classic algorithm AdaBoost allows to convert a weak learner, that is an algorithm that produces a hypothesis which is slightly better than chance, into a strong learner, achieving arbitrarily high accuracy when given enough training data.
Improved Coresets for Euclidean $k$-Means
the Euclidean $k$-median problem) consists of finding $k$ centers such that the sum of squared distances (resp.
Bagging is an Optimal PAC Learner
Finally, the seminal work by Hanneke (2016) gave an algorithm with a provably optimal sample complexity.
The Impossibility of Parallelizing Boosting
The aim of boosting is to convert a sequence of weak learners into a strong learner.
AdaBoost is not an Optimal Weak to Strong Learner
AdaBoost is a classic boosting algorithm for combining multiple inaccurate classifiers produced by a weak learner, to produce a strong learner with arbitrarily high accuracy when given enough training data.
Sparse Dimensionality Reduction Revisited
In this work, we revisit sparse embeddings and identify a loophole in the lower bound.
Boosting, Voting Classifiers and Randomized Sample Compression Schemes
At the center of this paradigm lies the concept of building the strong learner as a voting classifier, which outputs a weighted majority vote of the weak learners.
Replicable Learning of Large-Margin Halfspaces
Departing from the requirement of polynomial time algorithms, using the DP-to-Replicability reduction of Bun, Gaboardi, Hopkins, Impagliazzo, Lei, Pitassi, Sorrell, and Sivakumar [STOC, 2023], we show how to obtain a replicable algorithm for large-margin halfspaces with improved sample complexity with respect to the margin parameter $\tau$, but running time doubly exponential in $1/\tau^2$ and worse sample complexity dependence on $\epsilon$ than one of our previous algorithms.
Majority-of-Three: The Simplest Optimal Learner?
Furthermore, we prove a near-optimal high-probability bound on this algorithm's error.
