Search Results for author:
Found 40 papers, 4 papers with code
Repeated Games against Budgeted Adversaries
We study repeated zero-sum games against an adversary on a budget.
Learning Eigenvectors for Free
In this extension, the alphabet of $n$ outcomes is replaced by the set of all dyads, i. e. outer products $\u\u^\top$ where $\u$ is a vector in $\R^n$ of unit length.
Putting Bayes to sleep
If the nature of the data is changing over time in that different models predict well on different segments of the data, then adaptivity is typically achieved by mixing into the weights in each round a bit of the initial prior (kind of like a weak restart).
On-line PCA with Optimal Regrets
Furthermore, we show that when considering regret bounds as function of a loss budget, EG remains optimal and strictly outperforms GD.
A Bayesian Probability Calculus for Density Matrices
Finite probability distributions are a special case where the density matrix is restricted to be diagonal.
The limits of squared Euclidean distance regularization
We conjecture that our hardness results hold for any training algorithm that is based on the squared Euclidean distance regularization (i. e. Back-propagation with the Weight Decay heuristic).
Labeled compression schemes for extremal classes
We consider a generalization of maximum classes called extremal classes.
PCA with Gaussian perturbations
We develop a simple algorithm that needs $O(kn^2)$ per trial whose regret is off by a small factor of $O(n^{1/4})$.
Low-dimensional Data Embedding via Robust Ranking
We describe a new method called t-ETE for finding a low-dimensional embedding of a set of objects in Euclidean space.
Unbiased estimates for linear regression via volume sampling
Pseudo inverse plays an important part in solving the linear least squares problem, where we try to predict a label for each column of $X$.
Two-temperature logistic regression based on the Tsallis divergence
We explain this by showing that $t_1 < 1$ caps the surrogate loss and $t_2 >1$ makes the predictive distribution have a heavy tail.
Online Dynamic Programming
We consider the problem of repeatedly solving a variant of the same dynamic programming problem in successive trials.
Subsampling for Ridge Regression via Regularized Volume Sampling
However, when labels are expensive, we are forced to select only a small subset of vectors $\mathbf{x}_i$ for which we obtain the labels $y_i$.
Leveraged volume sampling for linear regression
We then develop a new rescaled variant of volume sampling that produces an unbiased estimate which avoids this bad behavior and has at least as good a tail bound as leverage score sampling: sample size $k=O(d\log d + d/\epsilon)$ suffices to guarantee total loss at most $1+\epsilon$ times the minimum with high probability.
A more globally accurate dimensionality reduction method using triplets
We first show that the commonly used dimensionality reduction (DR) methods such as t-SNE and LargeVis poorly capture the global structure of the data in the low dimensional embedding.
Speech Recognition: Keyword Spotting Through Image Recognition
The problem of identifying voice commands has always been a challenge due to the presence of noise and variability in speed, pitch, etc.
Online Non-Additive Path Learning under Full and Partial Information
We study the problem of online path learning with non-additive gains, which is a central problem appearing in several applications, including ensemble structured prediction.
Reverse iterative volume sampling for linear regression
We can only afford to attain the responses for a small subset of the points that are then used to construct linear predictions for all points in the dataset.
Correcting the bias in least squares regression with volume-rescaled sampling
Without any assumptions on the noise, the linear least squares solution for any i. i. d.
Unlabeled sample compression schemes and corner peelings for ample and maximum classes
On the positive side we present a new construction of an unlabeled sample compression scheme for maximum classes.
Minimax experimental design: Bridging the gap between statistical and worst-case approaches to least squares regression
In the process, we develop a new algorithm for a joint sampling distribution called volume sampling, and we propose a new i. i. d.
Divergence-Based Motivation for Online EM and Combining Hidden Variable Models
Expectation-Maximization (EM) is a prominent approach for parameter estimation of hidden (aka latent) variable models.
Adaptive scale-invariant online algorithms for learning linear models
We consider online learning with linear models, where the algorithm predicts on sequentially revealed instances (feature vectors), and is compared against the best linear function (comparator) in hindsight.
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences
We introduce a temperature into the exponential function and replace the softmax output layer of neural nets by a high temperature generalization.
Unbiased estimators for random design regression
We use them to show that for any input distribution and $\epsilon>0$ there is a random design consisting of $O(d\log d+ d/\epsilon)$ points from which an unbiased estimator can be constructed whose expected square loss over the entire distribution is bounded by $1+\epsilon$ times the loss of the optimum.
An Implicit Form of Krasulina's k-PCA Update without the Orthonormality Constraint
We show that Krasulina's update corresponds to a projected gradient descent step on the Stiefel manifold of the orthonormal $k$-frames, while Oja's update amounts to a gradient descent step using the unprojected gradient.
TriMap: Large-scale Dimensionality Reduction Using Triplets
We empirically show the excellent performance of TriMap on a large variety of datasets in terms of the quality of the embedding as well as the runtime.
Reparameterizing Mirror Descent as Gradient Descent
We present a general framework for casting a mirror descent update as a gradient descent update on a different set of parameters.
A case where a spindly two-layer linear network whips any neural network with a fully connected input layer
It was conjectured that any neural network of any structure and arbitrary differentiable transfer functions at the nodes cannot learn the following problem sample efficiently when trained with gradient descent: The instances are the rows of a $d$-dimensional Hadamard matrix and the target is one of the features, i. e. very sparse.
Rank-smoothed Pairwise Learning In Perceptual Quality Assessment
Training a model on these pairwise preferences is a common deep learning approach.
Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond
Many learning tasks in machine learning can be viewed as taking a gradient step towards minimizing the average loss of a batch of examples in each training iteration.
LocoProp: Enhancing BackProp via Local Loss Optimization
Second-order methods have shown state-of-the-art performance for optimizing deep neural networks.
Step-size Adaptation Using Exponentiated Gradient Updates
In this paper, we update the step-size scale and the gain variables with exponentiated gradient updates instead.
Learning from Randomly Initialized Neural Network Features
We present the surprising result that randomly initialized neural networks are good feature extractors in expectation.
Layerwise Bregman Representation Learning with Applications to Knowledge Distillation
In this work, we propose a novel approach for layerwise representation learning of a trained neural network.
A Mechanism for Sample-Efficient In-Context Learning for Sparse Retrieval Tasks
We study the phenomenon of \textit{in-context learning} (ICL) exhibited by large language models, where they can adapt to a new learning task, given a handful of labeled examples, without any explicit parameter optimization.
Optimal Transport with Tempered Exponential Measures
In the field of optimal transport, two prominent subfields face each other: (i) unregularized optimal transport, "\`a-la-Kantorovich", which leads to extremely sparse plans but with algorithms that scale poorly, and (ii) entropic-regularized optimal transport, "\`a-la-Sinkhorn-Cuturi", which gets near-linear approximation algorithms but leads to maximally un-sparse plans.
The Tempered Hilbert Simplex Distance and Its Application To Non-linear Embeddings of TEMs
Tempered Exponential Measures (TEMs) are a parametric generalization of the exponential family of distributions maximizing the tempered entropy function among positive measures subject to a probability normalization of their power densities.
Tempered Calculus for ML: Application to Hyperbolic Model Embedding
Most mathematical distortions used in ML are fundamentally integral in nature: $f$-divergences, Bregman divergences, (regularized) optimal transport distances, integral probability metrics, geodesic distances, etc.
Noise misleads rotation invariant algorithms on sparse targets
It is well known that the class of rotation invariant algorithms are suboptimal even for learning sparse linear problems when the number of examples is below the "dimension" of the problem.
