Search Results for author:
Found 21 papers, 2 papers with code
Theoretical limitations of multi-layer Transformer
We also introduce a new proof technique that finds a certain $\textit{indistinguishable}$ $\textit{decomposition}$ of all possible inputs iteratively for proving lower bounds in this model.
The complexity of approximate (coarse) correlated equilibrium for incomplete information games
We study the iteration complexity of decentralized learning of approximate correlated equilibria in incomplete information games.
On Limitations of the Transformer Architecture
What are the root causes of hallucinations in large language models (LLMs)?
The sample complexity of multi-distribution learning
Multi-distribution learning generalizes the classic PAC learning to handle data coming from multiple distributions.
Fast swap regret minimization and applications to approximate correlated equilibria
We give a simple and computationally efficient algorithm that, for any constant $\varepsilon>0$, obtains $\varepsilon T$-swap regret within only $T = \mathsf{polylog}(n)$ rounds; this is an exponential improvement compared to the super-linear number of rounds required by the state-of-the-art algorithm, and resolves the main open problem of [Blum and Mansour 2007].
The complexity of non-stationary reinforcement learning
The problem of continual learning in the domain of reinforcement learning, often called non-stationary reinforcement learning, has been identified as an important challenge to the application of reinforcement learning.
Memory-Query Tradeoffs for Randomized Convex Optimization
We show that any randomized first-order algorithm which minimizes a $d$-dimensional, $1$-Lipschitz convex function over the unit ball must either use $\Omega(d^{2-\delta})$ bits of memory or make $\Omega(d^{1+\delta/6-o(1)})$ queries, for any constant $\delta\in (0, 1)$ and when the precision $\epsilon$ is quasipolynomially small in $d$.
Near Optimal Memory-Regret Tradeoff for Online Learning
In the experts problem, on each of $T$ days, an agent needs to follow the advice of one of $n$ ``experts''.
Online Prediction in Sub-linear Space
We provide the first sub-linear space and sub-linear regret algorithm for online learning with expert advice (against an oblivious adversary), addressing an open question raised recently by Srinivas, Woodruff, Xu and Zhou (STOC 2022).
Memory Bounds for Continual Learning
We make novel uses of communication complexity to establish that any continual learner, even an improper one, needs memory that grows linearly with $k$, strongly suggesting that the problem is intractable.
Continual learning: a feature extraction formalization, an efficient algorithm, and fundamental obstructions
When the features are linear, we design an efficient gradient-based algorithm $\mathsf{DPGD}$, that is guaranteed to perform well on the current environment, as well as avoid catastrophic forgetting.
The Complexity of Dynamic Least-Squares Regression
We settle the complexity of dynamic least-squares regression (LSR), where rows and labels $(\mathbf{A}^{(t)}, \mathbf{b}^{(t)})$ can be adaptively inserted and/or deleted, and the goal is to efficiently maintain an $\epsilon$-approximate solution to $\min_{\mathbf{x}^{(t)}} \| \mathbf{A}^{(t)} \mathbf{x}^{(t)} - \mathbf{b}^{(t)} \|_2$ for all $t\in [T]$.
On the Complexity of Dynamic Submodular Maximization
We study dynamic algorithms for the problem of maximizing a monotone submodular function over a stream of $n$ insertions and deletions.
Shuffle Private Stochastic Convex Optimization
In shuffle privacy, each user sends a collection of randomized messages to a trusted shuffler, the shuffler randomly permutes these messages, and the resulting shuffled collection of messages must satisfy differential privacy.
Self-Attention Networks Can Process Bounded Hierarchical Languages
Despite their impressive performance in NLP, self-attention networks were recently proved to be limited for processing formal languages with hierarchical structure, such as $\mathsf{Dyck}_k$, the language consisting of well-nested parentheses of $k$ types.
MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training
Recent advances by practitioners in the deep learning community have breathed new life into Locality Sensitive Hashing (LSH), using it to reduce memory and time bottlenecks in neural network (NN) training.
MixCon: Adjusting the Separability of Data Representations for Harder Data Recovery
To address the issue that deep neural networks (DNNs) are vulnerable to model inversion attacks, we design an objective function, which adjusts the separability of the hidden data representations, as a way to control the trade-off between data utility and vulnerability to inversion attacks.
Training (Overparametrized) Neural Networks in Near-Linear Time
The slow convergence rate and pathological curvature issues of first-order gradient methods for training deep neural networks, initiated an ongoing effort for developing faster $\mathit{second}$-$\mathit{order}$ optimization algorithms beyond SGD, without compromising the generalization error.
Adaptive Greedy versus Non-adaptive Greedy for Influence Maximization
On the other side, we prove that in any submodular cascade, the adaptive greedy algorithm always outputs a $(1-1/e)$-approximation to the expected number of adoptions in the optimal non-adaptive seed choice.
Social and Information Networks
On Adaptivity Gaps of Influence Maximization under the Independent Cascade Model with Full Adoption Feedback
In this paper, we study the adaptivity gap of the influence maximization problem under independent cascade model when full-adoption feedback is available.
Social and Information Networks
Adaptive Influence Maximization with Myopic Feedback
We study the adaptive influence maximization problem with myopic feedback under the independent cascade model: one sequentially selects k nodes as seeds one by one from a social network, and each selected seed returns the immediate neighbors it activates as the feedback available for later selections, and the goal is to maximize the expected number of total activated nodes, referred as the influence spread.
Social and Information Networks
