Search Results for author:
Found 44 papers, 3 papers with code
Optimization using Parallel Gradient Evaluations on Multiple Parameters
We propose a first-order method for convex optimization, where instead of being restricted to the gradient from a single parameter, gradients from multiple parameters can be used during each step of gradient descent.
Async-HFL: Efficient and Robust Asynchronous Federated Learning in Hierarchical IoT Networks
To fully unleash the potential of Async-HFL in converging speed under system heterogeneities and stragglers, we design device selection at the gateway level and device-gateway association at the cloud level.
Improved Support Recovery in Universal One-bit Compressed Sensing
A {\em universal} measurement matrix for 1bCS refers to one set of measurements that work for all sparse signals.
An Improved Algorithm for Clustered Federated Learning
\texttt{SR-FCA} treats each user as a singleton cluster as an initialization, and then successively refine the cluster estimation via exploiting similar users belonging to the same cluster.
Binary Iterative Hard Thresholding Converges with Optimal Number of Measurements for 1-Bit Compressed Sensing
Note that, this dependence on $k$ and $\epsilon$ is optimal for any recovery method in 1-bit compressed sensing.
Community Recovery in the Geometric Block Model
We show that a simple triangle-counting algorithm to detect communities in the geometric block model is near-optimal.
Decentralized Competing Bandits in Non-Stationary Matching Markets
We propose and analyze a decentralized and asynchronous learning algorithm, namely Decentralized Non-stationary Competing Bandits (\texttt{DNCB}), where the agents play (restrictive) successive elimination type learning algorithms to learn their preference over the arms.
On Learning Mixture of Linear Regressions in the Non-Realizable Setting
In this paper we show that a version of the popular alternating minimization (AM) algorithm finds the best fit lines in a dataset even when a realizable model is not assumed, under some regularity conditions on the dataset and the initial points, and thereby provides a solution for the ERM.
Support Recovery in Mixture Models with Sparse Parameters
Sparsity of parameter vectors is a natural constraint in variety of settings, and support recovery is a major step towards parameter estimation.
Random Subgraph Detection Using Queries
Specifically, we show that any (possibly randomized) algorithm must make $\mathsf{Q} = \Omega(\frac{n^2}{k^2\chi^4(p||q)}\log^2n)$ adaptive queries (on expectation) to the adjacency matrix of the graph to detect the planted subgraph with probability more than $1/2$, where $\chi^2(p||q)$ is the Chi-Square distance.
Lower Bounds on the Total Variation Distance Between Mixtures of Two Gaussians
Mixtures of high dimensional Gaussian distributions have been studied extensively in statistics and learning theory.
Support Recovery in Universal One-bit Compressed Sensing
With universality, it is known that $\tilde{\Theta}(k^2)$ 1bCS measurements are necessary and sufficient for support recovery (where $k$ denotes the sparsity).
Support Recovery of Sparse Signals from a Mixture of Linear Measurements
In this work, we study the number of measurements sufficient for recovering the supports of all the component vectors in a mixture in both these models.
Fuzzy Clustering with Similarity Queries
In particular, we provide algorithms for fuzzy clustering in this setting that asks $O(\mathsf{poly}(k)\log n)$ similarity queries and run with polynomial-time-complexity, where $n$ is the number of items.
Escaping Saddle Points in Distributed Newton's Method with Communication Efficiency and Byzantine Resilience
Moreover, we validate our theoretical findings with experiments using standard datasets and several types of Byzantine attacks, and obtain an improvement of $25\%$ with respect to first order methods in iteration complexity.
Session-Aware Query Auto-completion using Extreme Multi-label Ranking
Previous queries in the user session can provide useful context for the user's intent and can be leveraged to suggest auto-completions that are more relevant while adhering to the user's prefix.
Recovery of sparse linear classifiers from mixture of responses
We look at a hitherto unstudied problem of query complexity upper bound of recovering all the hyperplanes, especially for the case when the hyperplanes are sparse.
Recovery of Sparse Signals from a Mixture of Linear Samples
Mixture of linear regressions is a popular learning theoretic model that is used widely to represent heterogeneous data.
Multilabel Classification by Hierarchical Partitioning and Data-dependent Grouping
We then present a hierarchical partitioning approach that exploits the label hierarchy in large scale problems to divide up the large label space and create smaller sub-problems, which can then be solved independently via the grouping approach.
Distributed Newton Can Communicate Less and Resist Byzantine Workers
We develop a distributed second order optimization algorithm that is communication-efficient as well as robust against Byzantine failures of the worker machines.
Reliable Distributed Clustering with Redundant Data Assignment
In this paper, we present distributed generalized clustering algorithms that can handle large scale data across multiple machines in spite of straggling or unreliable machines.
Algebraic and Analytic Approaches for Parameter Learning in Mixture Models
Our second approach uses algebraic and combinatorial tools and applies to binomial mixtures with shared trial parameter $N$ and differing success parameters, as well as to mixtures of geometric distributions.
Sample Complexity of Learning Mixture of Sparse Linear Regressions
Ourtechniques are quite different from those in the previous work: for the noiselesscase, we rely on a property of sparse polynomials and for the noisy case, we providenew connections to learning Gaussian mixtures and use ideas from the theory of
Communication-Efficient and Byzantine-Robust Distributed Learning with Error Feedback
Moreover, we analyze the compressed gradient descent algorithm with error feedback (proposed in \cite{errorfeed}) in a distributed setting and in the presence of Byzantine worker machines.
vqSGD: Vector Quantized Stochastic Gradient Descent
In this work, we present a family of vector quantization schemes \emph{vqSGD} (Vector-Quantized Stochastic Gradient Descent) that provide an asymptotic reduction in the communication cost with convergence guarantees in first-order distributed optimization.
Sample Complexity of Learning Mixtures of Sparse Linear Regressions
In the problem of learning mixtures of linear regressions, the goal is to learn a collection of signal vectors from a sequence of (possibly noisy) linear measurements, where each measurement is evaluated on an unknown signal drawn uniformly from this collection.
Same-Cluster Querying for Overlapping Clusters
In this paper, we look at the more practical scenario of overlapping clusters, and provide upper bounds (with algorithms) on the sufficient number of queries.
Learning Network Parameters in the ReLU Model
Rectified linear units, or ReLUs, have become a preferred activation function for artificial neural networks.
Semisupervised Clustering by Queries and Locally Encodable Source Coding
In this paper, we show that a recently popular model of semi-supervised clustering is equivalent to locally encodable source coding.
High Dimensional Discrete Integration over the Hypergrid
Recently Ermon et al. (2013) pioneered a way to practically compute approximations to large scale counting or discrete integration problems by using random hashes.
Robust Gradient Descent via Moment Encoding with LDPC Codes
We, instead, propose to encode the second-moment of the data with a low density parity-check (LDPC) code.
Connectivity in Random Annulus Graphs and the Geometric Block Model
Our next contribution is in using the connectivity of random annulus graphs to provide necessary and sufficient conditions for efficient recovery of communities for {\em the geometric block model} (GBM).
Representation Learning and Recovery in the ReLU Model
Given a set of observation vectors $\mathbf{y}^i \in \mathbb{R}^d, i =1, 2, \dots , n$, we aim to recover $d\times k$ matrix $A$ and the latent vectors $\{\mathbf{c}^i\} \subset \mathbb{R}^k$ under the model $\mathbf{y}^i = \mathrm{ReLU}(A\mathbf{c}^i +\mathbf{b})$, where $\mathbf{b}\in \mathbb{R}^d$ is a random bias.
Semisupervised Clustering, AND-Queries and Locally Encodable Source Coding
In this paper, we show that a recently popular model of semisupervised clustering is equivalent to locally encodable source coding.
The Geometric Block Model
To capture the inherent geometric features of many community detection problems, we propose to use a new random graph model of communities that we call a Geometric Block Model.
Multilabel Classification with Group Testing and Codes
In this work, we propose a novel approach based on group testing to solve such large multilabel classification problems with sparse label vectors.
Query Complexity of Clustering with Side Information
A natural noisy model is where similarity values are drawn independently from some arbitrary probability distribution $f_+$ when the underlying pair of elements belong to the same cluster, and from some $f_-$ otherwise.
Clustering with Noisy Queries
In this paper, we provide the first information theoretic lower bound on the number of queries for clustering with noisy oracle in both situations.
A Theoretical Analysis of First Heuristics of Crowdsourced Entity Resolution
Entity resolution (ER) is the task of identifying all records in a database that refer to the same underlying entity, and are therefore duplicates of each other.
Efficient Rank Aggregation via Lehmer Codes
We propose a novel rank aggregation method based on converting permutations into their corresponding Lehmer codes or other subdiagonal images.
Associative Memory using Dictionary Learning and Expander Decoding
Designing an associative memory requires addressing two main tasks: 1) learning phase: given a dataset, learn a concise representation of the dataset in the form of a graphical model (or a neural network), 2) recall phase: given a noisy version of a message vector from the dataset, output the correct message vector via a neurally feasible algorithm over the network learnt during the learning phase.
Clustering Via Crowdsourcing
A major contribution of this paper is to reduce the query complexity to linear or even sublinear in $n$ when mild side information is provided by a machine, and even in presence of crowd errors which are not correctable via resampling.
Low rank approximation and decomposition of large matrices using error correcting codes
In this paper, we show how matrices from error correcting codes can be used to find such low rank approximations and matrix decompositions, and extend the framework to linear least squares regression problems.
Associative Memory via a Sparse Recovery Model
An associative memory is a structure learned from a dataset $\mathcal{M}$ of vectors (signals) in a way such that, given a noisy version of one of the vectors as input, the nearest valid vector from $\mathcal{M}$ (nearest neighbor) is provided as output, preferably via a fast iterative algorithm.
