Search Results for author:
Found 134 papers, 32 papers with code
Optimization and Analysis of the pAp@k Metric for Recommender Systems
Modern recommendation and notification systems must be robust to data imbalance, limitations on the number of recommendations/notifications, and heterogeneous engagement profiles across users.
Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts?
While the large dimensionality of input token space makes it inevitable to find adversarial prompts that can jailbreak these models, we aim to evaluate whether safety fine-tuned LLMs are safe against natural prompts which are semantically related to toxic seed prompts that elicit safe responses after alignment.
Time-Reversal Provides Unsupervised Feedback to LLMs
Large Language Models (LLMs) are typically trained to predict in the forward direction of time.
Mixture of Nested Experts: Adaptive Processing of Visual Tokens
We present Mixture of Nested Experts (MoNE), which utilizes a nested structure for experts, wherein individual experts fall on an increasing compute-accuracy curve.
LookupViT: Compressing visual information to a limited number of tokens
We show LookupViT's effectiveness on multiple domains - (a) for image-classification (ImageNet-1K and ImageNet-21K), (b) video classification (Kinetics400 and Something-Something V2), (c) image captioning (COCO-Captions) with a frozen encoder.
Gecko: Versatile Text Embeddings Distilled from Large Language Models
On the Massive Text Embedding Benchmark (MTEB), Gecko with 256 embedding dimensions outperforms all existing entries with 768 embedding size.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
In this report, we introduce the Gemini 1. 5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio.
Ranked #1 on
Visual Question Answering
on MM-Vet
HiRE: High Recall Approximate Top-$k$ Estimation for Efficient LLM Inference
Autoregressive decoding with generative Large Language Models (LLMs) on accelerators (GPUs/TPUs) is often memory-bound where most of the time is spent on transferring model parameters from high bandwidth memory (HBM) to cache.
Tandem Transformers for Inference Efficient LLMs
On the PaLM2 pretraining dataset, a tandem of PaLM2-Bison and PaLM2-Gecko demonstrates a 3. 3% improvement in next-token prediction accuracy over a standalone PaLM2-Gecko, offering a 1. 16x speedup compared to a PaLM2-Otter model with comparable downstream performance.
LLM Augmented LLMs: Expanding Capabilities through Composition
Foundational models with billions of parameters which have been trained on large corpora of data have demonstrated non-trivial skills in a variety of domains.
Dual-Encoders for Extreme Multi-Label Classification
We propose decoupled softmax loss - a simple modification to the InfoNCE loss - that overcomes the limitations of existing contrastive losses.
EHI: End-to-end Learning of Hierarchical Index for Efficient Dense Retrieval
To facilitate the effective learning of this discrete structure, EHI introduces dense path embeddings that encodes the path traversed by queries and documents within the tree.
MatFormer: Nested Transformer for Elastic Inference
Furthermore, we observe that smaller encoders extracted from a universal MatFormer-based ViT (MatViT) encoder preserve the metric-space structure for adaptive large-scale retrieval.
End-to-End Neural Network Compression via $\frac{\ell_1}{\ell_2}$ Regularized Latency Surrogates
Our algorithm is versatile and can be used with many popular compression methods including pruning, low-rank factorization, and quantization.
AdANNS: A Framework for Adaptive Semantic Search
Finally, we demonstrate that AdANNS can enable inference-time adaptivity for compute-aware search on ANNS indices built non-adaptively on matryoshka representations.
Multi-Task Differential Privacy Under Distribution Skew
We study the problem of multi-task learning under user-level differential privacy, in which $n$ users contribute data to $m$ tasks, each involving a subset of users.
iPAL: A Machine Learning Based Smart Healthcare Framework For Automatic Diagnosis Of Attention Deficit/Hyperactivity Disorder (ADHD)
In this work, multiclass classification is performed on phenotypic data using an SVM model.
Near Optimal Private and Robust Linear Regression
Under label-corruption, this is the first efficient linear regression algorithm to guarantee both $(\varepsilon,\delta)$-DP and robustness.
Optimal Algorithms for Latent Bandits with Cluster Structure
Instead, we propose LATTICE (Latent bAndiTs via maTrIx ComplEtion) which allows exploitation of the latent cluster structure to provide the minimax optimal regret of $\widetilde{O}(\sqrt{(\mathsf{M}+\mathsf{N})\mathsf{T}})$, when the number of clusters is $\widetilde{O}(1)$.
Hybrid Quantum Generative Adversarial Networks for Molecular Simulation and Drug Discovery
In molecular research, simulation \& design of molecules are key areas with significant implications for drug development, material science, and other fields.
Variational Quantum Algorithms for Chemical Simulation and Drug Discovery
A moderate protein has about 100 amino acids, and the number of combinations one needs to verify to find the stable structure is enormous.
Multi-User Reinforcement Learning with Low Rank Rewards
In this work, we consider the problem of collaborative multi-user reinforcement learning.
Sample-Efficient Personalization: Modeling User Parameters as Low Rank Plus Sparse Components
We then study the framework in the linear setting, where the problem reduces to that of estimating the sum of a rank-$r$ and a $k$-column sparse matrix using a small number of linear measurements.
Learning an Invertible Output Mapping Can Mitigate Simplicity Bias in Neural Networks
To bridge the gap between these two lines of work, we first hypothesize and verify that while SB may not altogether preclude learning complex features, it amplifies simpler features over complex ones.
Online Low Rank Matrix Completion
In each round, the algorithm recommends one item per user, for which it gets a (noisy) reward sampled from a low-rank user-item preference matrix.
Surya Namaskar: real-time advanced yoga pose recognition and correction for smart healthcare
Nowadays, yoga has gained worldwide attention because of increasing levels of stress in the modern way of life, and there are many ways or resources to learn yoga.
DAFT: Distilling Adversarially Fine-tuned Models for Better OOD Generalization
We consider the problem of OOD generalization, where the goal is to train a model that performs well on test distributions that are different from the training distribution.
Treeformer: Dense Gradient Trees for Efficient Attention Computation
Based on such hierarchical navigation, we design Treeformer which can use one of two efficient attention layers -- TF-Attention and TC-Attention.
(Nearly) Optimal Private Linear Regression via Adaptive Clipping
Compared to existing $(\epsilon, \delta)$-DP techniques which have sub-optimal error bounds, DP-AMBSSGD is able to provide nearly optimal error bounds in terms of key parameters like dimensionality $d$, number of points $N$, and the standard deviation $\sigma$ of the noise in observations.
MET: Masked Encoding for Tabular Data
Typical contrastive learning based SSL methods require instance-wise data augmentations which are difficult to design for unstructured tabular data.
DP-PCA: Statistically Optimal and Differentially Private PCA
For sub-Gaussian data, we provide nearly optimal statistical error rates even for $n=\tilde O(d)$.
Matryoshka Representation Learning
The flexibility within the learned Matryoshka Representations offer: (a) up to 14x smaller embedding size for ImageNet-1K classification at the same level of accuracy; (b) up to 14x real-world speed-ups for large-scale retrieval on ImageNet-1K and 4K; and (c) up to 2% accuracy improvements for long-tail few-shot classification, all while being as robust as the original representations.
Ranked #25 on
Image Classification
on ObjectNet
(using extra training data)
Reproducibility in Optimization: Theoretical Framework and Limits
We initiate a formal study of reproducibility in optimization.
Real-time Recognition of Yoga Poses using computer Vision for Smart Health Care
In this work, a self-assistance based yoga posture identification technique is developed, which helps users to perform Yoga with the correction feature in Real-time.
Statistically and Computationally Efficient Linear Meta-representation Learning
To cope with such data scarcity, meta-representation learning methods train across many related tasks to find a shared (lower-dimensional) representation of the data where all tasks can be solved accurately.
Differentially Private Model Personalization
We study personalization of supervised learning with user-level differential privacy.
Node-Level Differentially Private Graph Neural Networks
Graph Neural Networks (GNNs) are a popular technique for modelling graph-structured data and computing node-level representations via aggregation of information from the neighborhood of each node.
Neural Network Compatible Off-Policy Natural Actor-Critic Algorithm
Learning optimal behavior from existing data is one of the most important problems in Reinforcement Learning (RL).
Online Target Q-learning with Reverse Experience Replay: Efficiently finding the Optimal Policy for Linear MDPs
The starting point of our work is the observation that in practice, Q-learning is used with two important modifications: (i) training with two networks, called online network and target network simultaneously (online target learning, or OTL) , and (ii) experience replay (ER) (Mnih et al., 2015).
IGLU: Efficient GCN Training via Lazy Updates
Training multi-layer Graph Convolution Networks (GCN) using standard SGD techniques scales poorly as each descent step ends up updating node embeddings for a large portion of the graph.
Private Alternating Least Squares: Practical Private Matrix Completion with Tighter Rates
We study the problem of differentially private (DP) matrix completion under user-level privacy.
Robust Training in High Dimensions via Block Coordinate Geometric Median Descent
Geometric median (\textsc{Gm}) is a classical method in statistics for achieving a robust estimation of the uncorrupted data; under gross corruption, it achieves the optimal breakdown point of 0. 5.
Ranked #22 on
Image Classification
on MNIST
(Accuracy metric)
LLC: Accurate, Multi-purpose Learnt Low-dimensional Binary Codes
We further quantitatively measure the quality of our codes by applying it to the efficient image retrieval as well as out-of-distribution (OOD) detection problems.
Near-optimal Offline and Streaming Algorithms for Learning Non-Linear Dynamical Systems
In this work, we improve existing results for learning nonlinear systems in a number of ways: a) we provide the first offline algorithm that can learn non-linear dynamical systems without the mixing assumption, b) we significantly improve upon the sample complexity of existing results for mixing systems, c) in the much harder one-pass, streaming setting we study a SGD with Reverse Experience Replay ($\mathsf{SGD-RER}$) method, and demonstrate that for mixing systems, it achieves the same sample complexity as our offline algorithm, d) we justify the expansivity assumption by showing that for the popular ReLU link function -- a non-expansive but easy to learn link function with i. i. d.
Sample Efficient Linear Meta-Learning by Alternating Minimization
We show that, for a constant subspace dimension MLLAM obtains nearly-optimal estimation error, despite requiring only $\Omega(\log d)$ samples per task.
Streaming Linear System Identification with Reverse Experience Replay
Thus, we provide the first -- to the best of our knowledge -- optimal SGD-style algorithm for the classical problem of linear system identification with a first order oracle.
Do Input Gradients Highlight Discriminative Features?
We believe that the DiffROAR evaluation framework and BlockMNIST-based datasets can serve as sanity checks to audit instance-specific interpretability methods; code and data available at https://github. com/harshays/inputgradients.
Optimal Regret Algorithm for Pseudo-1d Bandit Convex Optimization
We study online learning with bandit feedback (i. e. learner has access to only zeroth-order oracle) where cost/reward functions $\f_t$ admit a "pseudo-1d" structure, i. e. $\f_t(\w) = \loss_t(\pred_t(\w))$ where the output of $\pred_t$ is one-dimensional.
Learning Accurate Decision Trees with Bandit Feedback via Quantized Gradient Descent
Decision trees provide a rich family of highly non-linear but efficient models, due to which they continue to be the go-to family of predictive models by practitioners across domains.
Everything You Wanted to Know About Noninvasive Glucose Measurement and Control
There is requirement to develop the Internet-Medical-Things (IoMT) integrated Healthcare Cyber-Physical System (H-CPS) based Smart Healthcare framework for glucose measurement with purpose of continuous health monitoring.
Medical Physics
Oasis: ILP-Guided Synthesis of Loop Invariants
Automated synthesis of inductive invariants is an important problem in software verification.
Projection Efficient Subgradient Method and Optimal Nonsmooth Frank-Wolfe Method
Further, instead of a PO if we only have a linear minimization oracle (LMO, a la Frank-Wolfe) to access the constraint set, an extension of our method, MOLES, finds a feasible $\epsilon$-suboptimal solution using $O(\epsilon^{-2})$ LMO calls and FO calls---both match known lower bounds, resolving a question left open since White (1993).
Programming by Rewards
The goal of the synthesizer is to synthesize a "decision function" $f$ which transforms the features to a decision value for the black-box component so as to maximize the expected reward $E[r \circ f (x)]$ for executing decisions $f(x)$ for various values of $x$.
Globally-convergent Iteratively Reweighted Least Squares for Robust Regression Problems
We provide the first global model recovery results for the IRLS (iteratively reweighted least squares) heuristic for robust regression problems.
Least Squares Regression with Markovian Data: Fundamental Limits and Algorithms
Our improved rate serves as one of the first results where an algorithm outperforms SGD-DD on an interesting Markov chain and also provides one of the first theoretical analyses to support the use of experience replay in practice.
The Pitfalls of Simplicity Bias in Neural Networks
Furthermore, previous settings that use SB to theoretically justify why neural networks generalize well do not simultaneously capture the non-robustness of neural networks---a widely observed phenomenon in practice [Goodfellow et al. 2014, Jo and Bengio 2017].
A Topic-Aligned Multilingual Corpus of Wikipedia Articles for Studying Information Asymmetry in Low Resource Languages
Wikipedia is the largest web-based open encyclopedia covering more than three hundred languages.
COVID-19: Strategies for Allocation of Test Kits
Thus, it is important to allocate a separate budget of test-kits per day targeted towards preventing community spread and detecting new cases early on.
DROCC: Deep Robust One-Class Classification
Classical approaches for one-class problems such as one-class SVM and isolation forest require careful feature engineering when applied to structured domains like images.
Ranked #3 on
Anomaly Detection
on UEA time-series datasets
RNNPool: Efficient Non-linear Pooling for RAM Constrained Inference
Standard Convolutional Neural Networks (CNNs) designed for computer vision tasks tend to have large intermediate activation maps.
Ranked #26 on
Face Detection
on WIDER Face (Medium)
Soft Threshold Weight Reparameterization for Learnable Sparsity
Sparsity in Deep Neural Networks (DNNs) is studied extensively with the focus of maximizing prediction accuracy given an overall parameter budget.
Rich-Item Recommendations for Rich-Users: Exploiting Dynamic and Static Side Information
In this paper, we study the problem of recommendation system where the users and items to be recommended are rich data structures with multiple entity types and with multiple sources of side-information in the form of graphs.
Shallow RNN: Accurate Time-series Classification on Resource Constrained Devices
The second layer consumes the output of the first layer using a second RNN thus capturing long dependencies.
Provable Non-linear Inductive Matrix Completion
Inductive matrix completion (IMC) method is a standard approach for this problem where the given query as well as the items are embedded in a common low-dimensional space.
On Scaling Data-Driven Loop Invariant Inference
Automated synthesis of inductive invariants is an important problem in software verification.
GesturePod: Enabling On-device Gesture-based Interaction for White Cane Users
Our in-lab study shows that GesturePod achieves 92% gesture recognition accuracy and can help perform common smartphone tasks faster.
Ranked #1 on
Gesture Recognition
on GesturePod
Learning Functions over Sets via Permutation Adversarial Networks
In this paper, we consider the problem of learning functions over sets, i. e., functions that are invariant to permutations of input set items.
Efficient Algorithms for Smooth Minimax Optimization
This paper studies first order methods for solving smooth minimax optimization problems $\min_x \max_y g(x, y)$ where $g(\cdot,\cdot)$ is smooth and $g(x,\cdot)$ is concave for each $x$.
Universality Patterns in the Training of Neural Networks
This paper proposes and demonstrates a surprising pattern in the training of neural networks: there is a one to one relation between the values of any pair of losses (such as cross entropy, mean squared error, 0/1 error etc.)
Making the Last Iterate of SGD Information Theoretically Optimal
While classical theoretical analysis of SGD for convex problems studies (suffix) \emph{averages} of iterates and obtains information theoretically optimal bounds on suboptimality, the \emph{last point} of SGD is, by far, the most preferred choice in practice.
Adaptive Hard Thresholding for Near-optimal Consistent Robust Regression
We provide a nearly linear time estimator which consistently estimates the true regression vector, even with $1-o(1)$ fraction of corruptions.
SGD without Replacement: Sharper Rates for General Smooth Convex Functions
For {\em small} $K$, we show \sgdwor can achieve same convergence rate as \sgd for {\em general smooth strongly-convex} functions.
FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network
FastRNN addresses these limitations by adding a residual connection that does not constrain the range of the singular values explicitly and has only two extra scalar parameters.
Multiple Instance Learning for Efficient Sequential Data Classification on Resource-constrained Devices
We propose a method, EMI-RNN, that exploits these observations by using a multiple instance learning formulation along with an early prediction technique to learn a model that achieves better accuracy compared to baseline models, while simultaneously reducing computation by a large fraction.
Support Recovery for Orthogonal Matching Pursuit: Upper and Lower bounds
This paper studies the problem of sparse regression where the goal is to learn a sparse vector that best optimizes a given objective function.
PennyLane: Automatic differentiation of hybrid quantum-classical computations
PennyLane's core feature is the ability to compute gradients of variational quantum circuits in a way that is compatible with classical techniques such as backpropagation.
Nonlinear Inductive Matrix Completion based on One-layer Neural Networks
A standard approach to modeling this problem is Inductive Matrix Completion where the predicted rating is modeled as an inner product of the user and the item features projected onto a latent space.
Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples
In this work, we propose Neural Guided Deductive Search (NGDS), a hybrid synthesis technique that combines the best of both symbolic logic techniques and statistical models.
On the insufficiency of existing momentum schemes for Stochastic Optimization
Extensive empirical results in this paper show that ASGD has performance gains over HB, NAG, and SGD.
Smoothed analysis for low-rank solutions to semidefinite programs in quadratic penalty form
Semidefinite programs (SDP) are important in learning and combinatorial optimization with numerous applications.
Differentially Private Matrix Completion Revisited
We provide the first provably joint differentially private algorithm with formal utility guarantees for the problem of user-level privacy-preserving collaborative filtering.
Non-convex Optimization for Machine Learning
The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non-convex problems.
Consistent Robust Regression
We present the first efficient and provably consistent estimator for the robust regression problem.
A Markov Chain Theory Approach to Characterizing the Minimax Optimality of Stochastic Gradient Descent (for Least Squares)
This work provides a simplified proof of the statistical minimax optimality of (iterate averaged) stochastic gradient descent (SGD), for the special case of least squares.
Leveraging Distributional Semantics for Multi-Label Learning
Our approach is novel in that it highlights interesting connections between label embedding methods used for multi-label learning and paragraph/document embedding methods commonly used for learning representations of text data.
FlashProfile: A Framework for Synthesizing Data Profiles
However, manual inspection of data to identify the different formats is infeasible in standard big-data scenarios.
Nearly Optimal Robust Matrix Completion
Finally, an application of our result to the robust PCA problem (low-rank+sparse matrix separation) leads to nearly linear time (in matrix dimensions) algorithm for the same; existing state-of-the-art methods require quadratic time.
Active Heteroscedastic Regression
In this work, we consider a theoretical analysis of the label requirement of active learning for regression under a heteroscedastic noise model, where the noise depends on the instance.
ProtoNN: Compressed and Accurate kNN for Resource-scarce Devices
Such applications demand prediction models with small storage and computational complexity that do not compromise significantly on accuracy.
Learning Mixture of Gaussians with Streaming Data
However, by using a streaming version of the classical (soft-thresholding-based) EM method that exploits the Gaussian distribution explicitly, we show that for a mixture of two Gaussians the true means can be estimated consistently, with estimation error decreasing at nearly optimal rate, and tending to $0$ for $N\rightarrow \infty$.
Recovery Guarantees for One-hidden-layer Neural Networks
For activation functions that are also smooth, we show $\mathit{local~linear~convergence}$ guarantees of gradient descent under a resampling rule.
Accelerating Stochastic Gradient Descent For Least Squares Regression
There is widespread sentiment that it is not possible to effectively utilize fast gradient methods (e. g. Nesterov's acceleration, conjugate gradient, heavy ball) for the purposes of stochastic optimization due to their instability and error accumulation, a notion made precise in d'Aspremont 2008 and Devolder, Glineur, and Nesterov 2014.
Thresholding based Efficient Outlier Robust PCA
That is, given a data matrix $M^*$, where $(1-\alpha)$ fraction of the points are noisy samples from a low-dimensional subspace while $\alpha$ fraction of the points can be arbitrary outliers, the goal is to recover the subspace accurately.
Structured Sparse Regression via Greedy Hard Thresholding
Several learning applications require solving high-dimensional regression problems where the relevant features belong to a small number of (overlapping) groups.
Mixed Linear Regression with Multiple Components
Furthermore, our empirical results indicate that even with random initialization, our approach converges to the global optima in linear time, providing speed-up of up to two orders of magnitude.
Parallelizing Stochastic Gradient Descent for Least Squares Regression: mini-batching, averaging, and model misspecification
In particular, this work provides a sharp analysis of: (1) mini-batching, a method of averaging many samples of a stochastic gradient to both reduce the variance of the stochastic gradient estimate and for parallelizing SGD and (2) tail-averaging, a method involving averaging the final few iterates of SGD to decrease the variance in SGD's final iterate.
Efficient and Consistent Robust Time Series Analysis
We illustrate our methods on synthetic datasets and show that our methods indeed are able to consistently recover the optimal parameters despite a large fraction of points being corrupted.
Nearly-optimal Robust Matrix Completion
Finally, an application of our result to the robust PCA problem (low-rank+sparse matrix separation) leads to nearly linear time (in matrix dimensions) algorithm for the same; existing state-of-the-art methods require quadratic time.
Regret Bounds for Non-decomposable Metrics with Missing Labels
We consider the problem of recommending relevant labels (items) for a given data point (user).
Streaming PCA: Matching Matrix Bernstein and Near-Optimal Finite Sample Guarantees for Oja's Algorithm
This work provides improved guarantees for streaming principle component analysis (PCA).
Structured Sparse Regression via Greedy Hard-Thresholding
Several learning applications require solving high-dimensional regression problems where the relevant features belong to a small number of (overlapping) groups.
Sparse Local Embeddings for Extreme Multi-label Classification
The objective in extreme multi-label learning is to train a classifier that can automatically tag a novel data point with the most relevant subset of labels from an extremely large label set.
Predtron: A Family of Online Algorithms for General Prediction Problems
We offer a general framework to derive mistake driven online algorithms and associated loss bounds.
Alternating Minimization for Regression Problems with Vector-valued Outputs
In regression problems involving vector-valued outputs (or equivalently, multiple responses), it is well known that the maximum likelihood estimator (MLE), which takes noise covariance structure into account, can be significantly more accurate than the ordinary least squares (OLS) estimator.
Tensor vs Matrix Methods: Robust Tensor Decomposition under Block Sparse Perturbations
Robust tensor CP decomposition involves decomposing a tensor into low rank and sparse components.
Locally Non-linear Embeddings for Extreme Multi-label Learning
Embedding based approaches make training and prediction tractable by assuming that the training label matrix is low-rank and hence the effective number of labels can be reduced by projecting the high dimensional label vectors onto a low dimensional linear subspace.
Extreme Multi-Label Classification
General Classification
+2
Robust Regression via Hard Thresholding
In this work, we study a simple hard-thresholding algorithm called TORRENT which, under mild conditions on X, can recover w* exactly even if b corrupts the response variables in an adversarial manner, i. e. both the support and entries of b are selected adversarially after observing X and w*.
Surrogate Functions for Maximizing Precision at the Top
At the heart of our results is a family of truly upper bounding surrogates for prec@k. These surrogates are motivated in a principled manner and enjoy attractive properties such as consistency to prec@k under various natural margin/noise conditions.
Optimizing Non-decomposable Performance Measures: A Tale of Two Classes
Modern classification problems frequently present mild to severe label imbalance as well as specific requirements on classification characteristics, and require optimizing performance measures that are non-decomposable over the dataset, such as F-measure.
To Drop or Not to Drop: Robustness, Consistency and Differential Privacy Properties of Dropout
Moreover, using the above mentioned stability properties of dropout, we design dropout based differentially private algorithms for solving ERMs.
Provable Submodular Minimization using Wolfe's Algorithm
In 1976, Wolfe proposed an algorithm to find the minimum Euclidean norm point in a polytope, and in 1980, Fujishige showed how Wolfe's algorithm can be used for SFM.
Fast Exact Matrix Completion with Finite Samples
In this paper, we present a fast iterative algorithm that solves the matrix completion problem by observing $O(nr^5 \log^3 n)$ entries, which is independent of the condition number and the desired accuracy.
Non-convex Robust PCA
In contrast, existing methods for robust PCA, which are based on convex optimization, have $O(m^2n)$ complexity per iteration, and take $O(1/\epsilon)$ iterations, i. e., exponentially more iterations for the same accuracy.
Online and Stochastic Gradient Methods for Non-decomposable Loss Functions
In this work we initiate a study of online learning techniques for such non-decomposable loss functions with an aim to enable incremental learning as well as design scalable solvers for batch problems.
On Iterative Hard Thresholding Methods for High-dimensional M-Estimation
Our results rely on a general analysis framework that enables us to analyze several popular hard thresholding style algorithms (such as HTP, CoSaMP, SP) in the high dimensional regression setting.
Tighter Low-rank Approximation via Sampling the Leveraged Element
The first is a new method to directly compute a low-rank approximation (in efficient factored form) to the product of two given matrices; it computes a small random set of entries of the product, and then executes weighted alternating minimization (as before) on these.
Provable Tensor Factorization with Missing Data
We show that under certain standard assumptions, our method can recover a three-mode $n\times n\times n$ dimensional rank-$r$ tensor exactly from $O(n^{3/2} r^5 \log^4 n)$ randomly sampled entries.
Universal Matrix Completion
The problem of low-rank matrix completion has recently generated a lot of interest leading to several results that offer exact solutions to the problem.
Learning Mixtures of Discrete Product Distributions using Spectral Decompositions
The main challenge in learning mixtures of discrete product distributions is that these low-rank tensors cannot be obtained directly from the sample moments.
Learning Sparsely Used Overcomplete Dictionaries via Alternating Minimization
Alternating minimization is a popular heuristic for sparse coding, where the dictionary and the coefficients are estimated in alternate steps, keeping the other fixed.
Large-scale Multi-label Learning with Missing Labels
The multi-label classification problem has generated significant interest in recent years.
Memory Limited, Streaming PCA
Standard algorithms require $O(p^2)$ memory; meanwhile no algorithm can do better than $O(kp)$ memory, since this is what the output itself requires.
Provable Inductive Matrix Completion
In addition to inductive matrix completion, we show that two other low-rank estimation problems can be studied in our framework: a) general low-rank matrix sensing using rank-1 measurements, and b) multi-label regression with missing labels.
Phase Retrieval using Alternating Minimization
Empirically, we demonstrate that alternating minimization performs similar to recently proposed convex techniques for this problem (which are based on "lifting" to a convex matrix problem) in sample complexity and robustness to noise.
On the Generalization Ability of Online Learning Algorithms for Pairwise Loss Functions
We are also able to analyze a class of memory efficient online learning algorithms for pairwise learning problems that use only a bounded subset of past training samples to update the hypothesis at each step.
Multilabel Classification using Bayesian Compressed Sensing
The two key benefits of the model are that a) it can naturally handle datasets that have missing labels and b) it can also measure uncertainty in prediction.
Supervised Learning with Similarity Functions
a given supervised learning task and then adapt a well-known landmarking technique to provide efficient algorithms for supervised learning using ''good'' similarity functions.
Orthogonal Matching Pursuit with Replacement
Our proof techniques are novel and flexible enough to also permit the tightest known analysis of popular iterative algorithms such as CoSaMP and Subspace Pursuit.
Similarity-based Learning via Data Driven Embeddings
We propose a landmarking-based approach to obtaining a classifier from such learned goodness criteria.
Inductive Regularized Learning of Kernel Functions
Our result shows that the learned kernel matrices parameterize a linear transformation kernel function and can be applied inductively to new data points.
Hashing Hyperplane Queries to Near Points with Applications to Large-Scale Active Learning
Our first approach maps the data to two-bit binary keys that are locality-sensitive for the angle between the hyperplane normal and a database point.
Matrix Completion from Power-Law Distributed Samples
In this paper, we propose a graph theoretic approach to matrix completion that solves the problem for more realistic sampling models.