Search Results for author:
Found 47 papers, 5 papers with code
Associative Memory via a Sparse Recovery Model
An associative memory is a structure learned from a dataset $\mathcal{M}$ of vectors (signals) in a way such that, given a noisy version of one of the vectors as input, the nearest valid vector from $\mathcal{M}$ (nearest neighbor) is provided as output, preferably via a fast iterative algorithm.
Low rank approximation and decomposition of large matrices using error correcting codes
In this paper, we show how matrices from error correcting codes can be used to find such low rank approximations and matrix decompositions, and extend the framework to linear least squares regression problems.
Clustering Via Crowdsourcing
A major contribution of this paper is to reduce the query complexity to linear or even sublinear in $n$ when mild side information is provided by a machine, and even in presence of crowd errors which are not correctable via resampling.
Associative Memory using Dictionary Learning and Expander Decoding
Designing an associative memory requires addressing two main tasks: 1) learning phase: given a dataset, learn a concise representation of the dataset in the form of a graphical model (or a neural network), 2) recall phase: given a noisy version of a message vector from the dataset, output the correct message vector via a neurally feasible algorithm over the network learnt during the learning phase.
Efficient Rank Aggregation via Lehmer Codes
We propose a novel rank aggregation method based on converting permutations into their corresponding Lehmer codes or other subdiagonal images.
A Theoretical Analysis of First Heuristics of Crowdsourced Entity Resolution
Entity resolution (ER) is the task of identifying all records in a database that refer to the same underlying entity, and are therefore duplicates of each other.
Clustering with Noisy Queries
In this paper, we provide the first information theoretic lower bound on the number of queries for clustering with noisy oracle in both situations.
Query Complexity of Clustering with Side Information
A natural noisy model is where similarity values are drawn independently from some arbitrary probability distribution $f_+$ when the underlying pair of elements belong to the same cluster, and from some $f_-$ otherwise.
Multilabel Classification with Group Testing and Codes
In this work, we propose a novel approach based on group testing to solve such large multilabel classification problems with sparse label vectors.
The Geometric Block Model
To capture the inherent geometric features of many community detection problems, we propose to use a new random graph model of communities that we call a Geometric Block Model.
Semisupervised Clustering, AND-Queries and Locally Encodable Source Coding
In this paper, we show that a recently popular model of semisupervised clustering is equivalent to locally encodable source coding.
Representation Learning and Recovery in the ReLU Model
Given a set of observation vectors $\mathbf{y}^i \in \mathbb{R}^d, i =1, 2, \dots , n$, we aim to recover $d\times k$ matrix $A$ and the latent vectors $\{\mathbf{c}^i\} \subset \mathbb{R}^k$ under the model $\mathbf{y}^i = \mathrm{ReLU}(A\mathbf{c}^i +\mathbf{b})$, where $\mathbf{b}\in \mathbb{R}^d$ is a random bias.
Connectivity in Random Annulus Graphs and the Geometric Block Model
Our next contribution is in using the connectivity of random annulus graphs to provide necessary and sufficient conditions for efficient recovery of communities for {\em the geometric block model} (GBM).
Robust Gradient Descent via Moment Encoding with LDPC Codes
We, instead, propose to encode the second-moment of the data with a low density parity-check (LDPC) code.
High Dimensional Discrete Integration over the Hypergrid
Recently Ermon et al. (2013) pioneered a way to practically compute approximations to large scale counting or discrete integration problems by using random hashes.
Semisupervised Clustering by Queries and Locally Encodable Source Coding
In this paper, we show that a recently popular model of semi-supervised clustering is equivalent to locally encodable source coding.
Learning Network Parameters in the ReLU Model
Rectified linear units, or ReLUs, have become a preferred activation function for artificial neural networks.
Same-Cluster Querying for Overlapping Clusters
In this paper, we look at the more practical scenario of overlapping clusters, and provide upper bounds (with algorithms) on the sufficient number of queries.
Sample Complexity of Learning Mixtures of Sparse Linear Regressions
In the problem of learning mixtures of linear regressions, the goal is to learn a collection of signal vectors from a sequence of (possibly noisy) linear measurements, where each measurement is evaluated on an unknown signal drawn uniformly from this collection.
vqSGD: Vector Quantized Stochastic Gradient Descent
In this work, we present a family of vector quantization schemes \emph{vqSGD} (Vector-Quantized Stochastic Gradient Descent) that provide an asymptotic reduction in the communication cost with convergence guarantees in first-order distributed optimization.
Communication-Efficient and Byzantine-Robust Distributed Learning with Error Feedback
Moreover, we analyze the compressed gradient descent algorithm with error feedback (proposed in \cite{errorfeed}) in a distributed setting and in the presence of Byzantine worker machines.
Sample Complexity of Learning Mixture of Sparse Linear Regressions
Ourtechniques are quite different from those in the previous work: for the noiselesscase, we rely on a property of sparse polynomials and for the noisy case, we providenew connections to learning Gaussian mixtures and use ideas from the theory of
Algebraic and Analytic Approaches for Parameter Learning in Mixture Models
Our second approach uses algebraic and combinatorial tools and applies to binomial mixtures with shared trial parameter $N$ and differing success parameters, as well as to mixtures of geometric distributions.
Reliable Distributed Clustering with Redundant Data Assignment
In this paper, we present distributed generalized clustering algorithms that can handle large scale data across multiple machines in spite of straggling or unreliable machines.
Distributed Newton Can Communicate Less and Resist Byzantine Workers
We develop a distributed second order optimization algorithm that is communication-efficient as well as robust against Byzantine failures of the worker machines.
Multilabel Classification by Hierarchical Partitioning and Data-dependent Grouping
We then present a hierarchical partitioning approach that exploits the label hierarchy in large scale problems to divide up the large label space and create smaller sub-problems, which can then be solved independently via the grouping approach.
Recovery of Sparse Signals from a Mixture of Linear Samples
Mixture of linear regressions is a popular learning theoretic model that is used widely to represent heterogeneous data.
Recovery of sparse linear classifiers from mixture of responses
We look at a hitherto unstudied problem of query complexity upper bound of recovering all the hyperplanes, especially for the case when the hyperplanes are sparse.
Session-Aware Query Auto-completion using Extreme Multi-label Ranking
Previous queries in the user session can provide useful context for the user's intent and can be leveraged to suggest auto-completions that are more relevant while adhering to the user's prefix.
Escaping Saddle Points in Distributed Newton's Method with Communication Efficiency and Byzantine Resilience
Moreover, we validate our theoretical findings with experiments using standard datasets and several types of Byzantine attacks, and obtain an improvement of $25\%$ with respect to first order methods in iteration complexity.
Fuzzy Clustering with Similarity Queries
In particular, we provide algorithms for fuzzy clustering in this setting that asks $O(\mathsf{poly}(k)\log n)$ similarity queries and run with polynomial-time-complexity, where $n$ is the number of items.
Support Recovery of Sparse Signals from a Mixture of Linear Measurements
In this work, we study the number of measurements sufficient for recovering the supports of all the component vectors in a mixture in both these models.
Support Recovery in Universal One-bit Compressed Sensing
With universality, it is known that $\tilde{\Theta}(k^2)$ 1bCS measurements are necessary and sufficient for support recovery (where $k$ denotes the sparsity).
Lower Bounds on the Total Variation Distance Between Mixtures of Two Gaussians
Mixtures of high dimensional Gaussian distributions have been studied extensively in statistics and learning theory.
Random Subgraph Detection Using Queries
Specifically, we show that any (possibly randomized) algorithm must make $\mathsf{Q} = \Omega(\frac{n^2}{k^2\chi^4(p||q)}\log^2n)$ adaptive queries (on expectation) to the adjacency matrix of the graph to detect the planted subgraph with probability more than $1/2$, where $\chi^2(p||q)$ is the Chi-Square distance.
Support Recovery in Mixture Models with Sparse Parameters
Sparsity of parameter vectors is a natural constraint in variety of settings, and support recovery is a major step towards parameter estimation.
On Learning Mixture of Linear Regressions in the Non-Realizable Setting
In this paper we show that a version of the popular alternating minimization (AM) algorithm finds the best fit lines in a dataset even when a realizable model is not assumed, under some regularity conditions on the dataset and the initial points, and thereby provides a solution for the ERM.
Decentralized Competing Bandits in Non-Stationary Matching Markets
We propose and analyze a decentralized and asynchronous learning algorithm, namely Decentralized Non-stationary Competing Bandits (\texttt{DNCB}), where the agents play (restrictive) successive elimination type learning algorithms to learn their preference over the arms.
Community Recovery in the Geometric Block Model
We show that a simple triangle-counting algorithm to detect communities in the geometric block model is near-optimal.
Binary Iterative Hard Thresholding Converges with Optimal Number of Measurements for 1-Bit Compressed Sensing
Note that, this dependence on $k$ and $\epsilon$ is optimal for any recovery method in 1-bit compressed sensing.
An Improved Algorithm for Clustered Federated Learning
\texttt{SR-FCA} treats each user as a singleton cluster as an initialization, and then successively refine the cluster estimation via exploiting similar users belonging to the same cluster.
Improved Support Recovery in Universal One-bit Compressed Sensing
A {\em universal} measurement matrix for 1bCS refers to one set of measurements that work for all sparse signals.
Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks
To fully unleash the potential of Async-HFL in converging speed under system heterogeneities and stragglers, we design device selection at the gateway level and device-gateway association at the cloud level.
Optimization using Parallel Gradient Evaluations on Multiple Parameters
We propose a first-order method for convex optimization, where instead of being restricted to the gradient from a single parameter, gradients from multiple parameters can be used during each step of gradient descent.
On the sample complexity of parameter estimation in logistic regression with normal design
The logistic regression model is one of the most popular data generation model in noisy binary classification problems.
Optimal Compression of Unit Norm Vectors in the High Distortion Regime
We approach this problem in a worst-case scenario, without any prior information on the vector, but allowing for the use of randomized compression maps.
Robust 1-bit Compressed Sensing with Iterative Hard Thresholding
In particular, we analyze the Binary Iterative Hard Thresholding (BIHT) algorithm, a proximal gradient descent on a properly defined loss function used for 1-bit compressed sensing, in this noisy setting.
