Search Results for author:
Found 65 papers, 8 papers with code
Optimization Can Learn Johnson Lindenstrauss Embeddings
A first answer is no: as we show, the distance-preserving objective of JL has a non-convex landscape over the space of projection matrices, with many bad stationary points.
Semantic Image Inversion and Editing using Rectified Stochastic Differential Equations
Although Diffusion Models (DMs) have recently dominated the field of generative modeling for images, their inversion presents faithfulness and editability challenges due to nonlinearities in drift and diffusion.
RL in Latent MDPs is Tractable: Online Guarantees via Off-Policy Evaluation
Our result builds off a new perspective on the role of off-policy evaluation guarantees and coverage coefficients in LMDPs, a perspective, that has been overlooked in the context of exploration in partially observed environments.
RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Control
Existing training-free approaches exhibit difficulties in (a) style extraction from reference images in the absence of additional style or content text descriptions, (b) unwanted content leakage from reference style images, and (c) effective composition of style and content.
Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion
To our best knowledge, this is the first work to offer an efficient second-order approximation in solving inverse problems using latent diffusion and editing real-world images with corruptions.
Prospective Side Information for Latent MDPs
In such an environment, the latent information remains fixed throughout each episode, since the identity of the user does not change during an interaction.
Optimizing Solution-Samplers for Combinatorial Problems: The Landscape of Policy-Gradient Methods
Deep Neural Networks and Reinforcement Learning methods have empirically shown great promise in tackling challenging combinatorial problems.
Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models
We present the first framework to solve linear inverse problems leveraging pre-trained latent diffusion models.
Finite-Time Logarithmic Bayes Regret Upper Bounds
In a multi-armed bandit, we obtain $O(c_\Delta \log n)$ and $O(c_h \log^2 n)$ upper bounds for an upper confidence bound algorithm, where $c_h$ and $c_\Delta$ are constants depending on the prior distribution and the gaps of bandit instances sampled from it, respectively.
Understanding Lexical Biases when Identifying Gang-related Social Media Communications
Individuals involved in gang-related activity use mainstream social media including Facebook and Twitter to express taunts and threats as well as grief and memorializing.
Beyond Uniform Smoothness: A Stopped Analysis of Adaptive SGD
Despite the richness, an emerging line of works achieves the $\widetilde{\mathcal{O}}(\frac{1}{\sqrt{T}})$ rate of convergence when the noise of the stochastic gradients is deterministically and uniformly bounded.
A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models
To the best of our knowledge, this is the first linear convergence result for a diffusion based image inpainting algorithm.
Reward-Mixing MDPs with a Few Latent Contexts are Learnable
We consider episodic reinforcement learning in reward-mixing Markov decision processes (RMMDPs): at the beginning of every episode nature randomly picks a latent reward model among $M$ candidates and an agent interacts with the MDP throughout the episode for $H$ time steps.
Tractable Optimality in Episodic Latent MABs
Then, through a method-of-moments approach, we design a procedure that provably learns a near-optimal policy with $O(\texttt{poly}(A) + \texttt{poly}(M, H)^{\min(M, H)})$ interactions.
Non-Stationary Bandits under Recharging Payoffs: Improved Planning with Sublinear Regret
Even assuming prior knowledge of the mean payoff functions, computing an optimal planning in the above model is NP-hard, while the state-of-the-art is a $1/4$-approximation algorithm for the case where at most one arm can be played per round.
Contextual Pandora's Box
Pandora's Box is a fundamental stochastic optimization problem, where the decision-maker must find a good alternative while minimizing the search cost of exploring the value of each alternative.
The Power of Adaptivity in SGD: Self-Tuning Step Sizes with Unbounded Gradients and Affine Variance
We study convergence rates of AdaGrad-Norm as an exemplar of adaptive stochastic gradient methods (SGD), where the step sizes change based on observed stochastic gradients, for minimizing non-convex, smooth objectives.
Coordinated Attacks against Contextual Bandits: Fundamental Limits and Defense Mechanisms
This parallelization gain is fundamentally altered by the presence of adversarial users: unless there are super-polynomial number of users, we show a lower bound of $\tilde{\Omega}(\min(S, A) \cdot \alpha^2 / \epsilon^2)$ {\it per-user} interactions to learn an $\epsilon$-optimal policy for the good users.
Reinforcement Learning in Reward-Mixing MDPs
We study the problem of learning a near optimal policy for two reward-mixing MDPs.
Combinatorial Blocking Bandits with Stochastic Delays
Recent work has considered natural variations of the multi-armed bandit problem, where the reward distribution of each arm is a special function of the time passed since its last pulling.
Recurrent Submodular Welfare and Matroid Blocking Semi-Bandits
A recent line of research focuses on the study of stochastic multi-armed bandits (MAB), in the case where temporal correlations of specific structure are imposed between the player's actions and the reward distributions of the arms.
Recoverability Landscape of Tree Structured Markov Random Fields under Symmetric Noise
Furthermore, we present a polynomial time, sample efficient algorithm that recovers the exact tree when this is possible, or up to the unidentifiability as promised by our characterization, when full recoverability is impossible.
RL for Latent MDPs: Regret Guarantees and a Lower Bound
In this work, we consider the regret minimization problem for reinforcement learning in latent Markov Decision Processes (LMDP).
Recurrent Submodular Welfare and Matroid Blocking Bandits
A recent line of research focuses on the study of the stochastic multi-armed bandits problem (MAB), in the case where temporal correlations of specific structure are imposed between the player's actions and the reward distributions of the arms (Kleinberg and Immorlica [FOCS18], Basu et al. [NeurIPS19]).
On the computational and statistical complexity of over-parameterized matrix sensing
We consider solving the low rank matrix sensing problem with Factorized Gradient Descend (FGD) method when the true rank is unknown and over-specified, which we refer to as over-parameterized matrix sensing.
Second Order Optimality in Decentralized Non-Convex Optimization via Perturbed Gradient Tracking
In this paper we study the problem of escaping from saddle points and achieving second-order optimality in a decentralized setting where a group of agents collaborate to minimize their aggregate objective function.
Robust Structured Statistical Estimation via Conditional Gradient Type Methods
However, the existing CG type methods are not robust to data corruption.
Robust Compressed Sensing using Generative Models
In analogy to classical compressed sensing, here we assume a generative model as a prior, that is, we assume the vector is represented by a deep generative model $G: \mathbb{R}^k \rightarrow \mathbb{R}^n$.
Robust Estimation of Tree Structured Ising Models
We consider the task of learning Ising models when the signs of different random variables are flipped independently with possibly unequal, unknown probabilities.
On the Minimax Optimality of the EM Algorithm for Learning Two-Component Mixed Linear Regression
In the low SNR regime where the SNR is below $\mathcal{O}((d/n)^{1/4})$, we show that EM converges to a $\mathcal{O}((d/n)^{1/4})$ neighborhood of the true parameters, after $\mathcal{O}((n/d)^{1/2})$ iterations.
Contextual Blocking Bandits
Assuming knowledge of the context distribution and the mean reward of each arm-context pair, we cast the problem as an online bipartite matching problem, where the right-vertices (contexts) arrive stochastically and the left-vertices (arms) are blocked for a finite number of rounds each time they are matched.
The EM Algorithm gives Sample-Optimality for Learning Mixtures of Well-Separated Gaussians
A fundamental previous result established that separation of $\Omega(\sqrt{\log k})$ is necessary and sufficient for identifiability of the parameters with polynomial sample complexity (Regev and Vijayaraghavan, 2017).
Primal-Dual Block Generalized Frank-Wolfe
We propose a generalized variant of Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems.
Communication-Efficient Asynchronous Stochastic Frank-Wolfe over Nuclear-norm Balls
Large-scale machine learning training suffers from two prior challenges, specifically for nuclear-norm constrained problems with distributed systems: the synchronization slowdown due to the straggling workers, and high communication costs.
Mix and Match: An Optimistic Tree-Search Approach for Learning Models from Mixture Distributions
We consider a covariate shift problem where one has access to several different training datasets for the same learning problem and a small validation set which possibly differs from all the individual training distributions.
More Supervision, Less Computation: Statistical-Computational Tradeoffs in Weakly Supervised Learning
We consider the weakly supervised binary classification problem where the labels are randomly flipped with probability $1- {\alpha}$.
Learning Mixtures of Graphs from Epidemic Cascades
When the conditions are met, i. e., when the graphs are connected with at least three edges, we give an efficient algorithm for learning the weights of both graphs with optimal sample complexity (up to log factors).
Primal-Dual Block Frank-Wolfe
We propose a variant of the Frank-Wolfe algorithm for solving a class of sparse/low-rank optimization problems.
EM Converges for a Mixture of Many Linear Regressions
In particular, our results imply exact recovery as $\sigma \rightarrow 0$, in contrast to most previous local convergence results for EM, where the statistical error scaled with the norm of parameters.
Learning Graphs from Noisy Epidemic Cascades
Finally, we give a polynomial time algorithm for learning the weights of general bounded-degree graphs in the limited-noise setting.
Robust estimation of tree structured Gaussian Graphical Model
If we observe realizations of the variables, we can compute the covariance matrix, and it is well known that the support of the inverse covariance matrix corresponds to the edges of the graphical model.
High Dimensional Robust $M$-Estimation: Arbitrary Corruption and Heavy Tails
We define a natural condition we call the Robust Descent Condition (RDC), and show that if a gradient estimator satisfies the RDC, then Robust Hard Thresholding (IHT using this gradient estimator), is guaranteed to obtain good statistical rates.
Global Convergence of EM Algorithm for Mixtures of Two Component Linear Regression
Recent results established that EM enjoys global convergence for Gaussian Mixture Models.
Applications of Common Entropy for Causal Inference
We study the problem of discovering the simplest latent variable that can make two observed discrete variables conditionally independent.
High Dimensional Robust Sparse Regression
Our algorithm recovers the true sparse parameters with sub-linear sample complexity, in the presence of a constant fraction of arbitrary corruptions.
Approximate Newton-based statistical inference using only stochastic gradients
We present a novel statistical inference framework for convex empirical risk minimization, using approximate stochastic Newton steps.
The Stochastic Firefighter Problem
We devise a simple policy that only vaccinates neighbors of infected nodes and is optimal on regular trees and on general graphs for a sufficiently large budget.
Statistical inference using SGD
We present a novel method for frequentist statistical inference in $M$-estimation problems, based on stochastic gradient descent (SGD) with a fixed step size: we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling.
Non-square matrix sensing without spurious local minima via the Burer-Monteiro approach
We consider the non-square matrix sensing problem, under restricted isometry property (RIP) assumptions.
Solving a Mixture of Many Random Linear Equations by Tensor Decomposition and Alternating Minimization
We give a tractable algorithm for the mixed linear equation problem, and show that under some technical conditions, our algorithm is guaranteed to solve the problem exactly with sample complexity linear in the dimension, and polynomial in $k$, the number of components.
Finding Low-Rank Solutions via Non-Convex Matrix Factorization, Efficiently and Provably
We study such parameterization for optimization of generic convex objectives $f$, and focus on first-order, gradient descent algorithmic solutions.
Provable Burer-Monteiro factorization for a class of norm-constrained matrix problems
We study the projected gradient descent method on low-rank matrix problems with a strongly convex objective.
Fast Algorithms for Robust PCA via Gradient Descent
For the partially observed case, we show the complexity of our algorithm is no more than $\mathcal{O}(r^4d \log d \log(1/\varepsilon))$.
Regularized EM Algorithms: A Unified Framework and Statistical Guarantees
In particular, regularizing the M-step using the state-of-the-art high-dimensional prescriptions (e. g., Wainwright (2014)) is not guaranteed to provide this balance.
Optimal linear estimation under unknown nonlinear transform
This model is known as the single-index model in statistics, and, among other things, it represents a significant generalization of one-bit compressed sensing.
Binary Embedding: Fundamental Limits and Fast Algorithm
Binary embedding is a nonlinear dimension reduction methodology where high dimensional data are embedded into the Hamming cube while preserving the structure of the original space.
Data Structures and Algorithms Information Theory Information Theory
Greedy Subspace Clustering
We consider the problem of subspace clustering: given points that lie on or near the union of many low-dimensional linear subspaces, recover the subspaces.
Localized epidemic detection in networks with overwhelming noise
Our algorithm requires only local-neighbor knowledge of this graph, and in a broad array of settings that we describe, succeeds even when false negatives and false positives make up an overwhelming fraction of the data available.
A Convex Formulation for Mixed Regression with Two Components: Minimax Optimal Rates
We consider the mixed regression problem with two components, under adversarial and stochastic noise.
Alternating Minimization for Mixed Linear Regression
Mixed linear regression involves the recovery of two (or more) unknown vectors from unlabeled linear measurements; that is, where each sample comes from exactly one of the vectors, but we do not know which one.
Memory Limited, Streaming PCA
Standard algorithms require $O(p^2)$ memory; meanwhile no algorithm can do better than $O(kp)$ memory, since this is what the output itself requires.
Orthogonal Matching Pursuit with Noisy and Missing Data: Low and High Dimensional Results
Many models for sparse regression typically assume that the covariates are known completely, and without noise.
Matrix completion with column manipulation: Near-optimal sample-robustness-rank tradeoffs
Moreover, we show by an information-theoretic argument that our guarantees are nearly optimal in terms of the fraction of sampled entries on the authentic columns, the fraction of corrupted columns, and the rank of the underlying matrix.
Robust PCA via Outlier Pursuit
Singular Value Decomposition (and Principal Component Analysis) is one of the most widely used techniques for dimensionality reduction: successful and efficiently computable, it is nevertheless plagued by a well-known, well-documented sensitivity to outliers.
Robust Regression and Lasso
We generalize this robust formulation to consider more general uncertainty sets, which all lead to tractable convex optimization problems.
