Search Results for author:
Found 49 papers, 12 papers with code
No-regret Algorithms for Fair Resource Allocation
The objective is to allocate resources equitably among several agents in an online fashion so that the difference of the aggregate $\alpha$-fair utilities of the agents between an optimal static clairvoyant allocation and that of the online policy grows sub-linearly with time.
Sample Constrained Treatment Effect Estimation
In particular, we study sample-constrained treatment effect estimation, where we must select a subset of $s \ll n$ individuals from the population to experiment on.
Direct Embedding of Temporal Network Edges via Time-Decayed Line Graphs
We present a simple method that avoids both shortcomings: construct the line graph of the network, which includes a node for each interaction, and weigh the edges of this graph based on the difference in time between interactions.
Simplified Graph Convolution with Heterophily
Like SGC, ASGC is not a deep model, and hence is fast, scalable, and interpretable; further, we can prove performance guarantees on natural synthetic data models.
Sublinear Time Approximation of Text Similarity Matrices
Approximation methods reduce this quadratic complexity, often by using a small subset of exactly computed similarities to approximate the remainder of the complete pairwise similarity matrix.
Active Linear Regression for $\ell_p$ Norms and Beyond
By combining this with our techniques for $\ell_p$ regression, we obtain an active regression algorithm making $\tilde O(d^{1+\max\{1, p/2\}}/\mathrm{poly}(\epsilon))$ queries for such loss functions, including the Tukey and Huber losses, answering another question of [CD21].
Exact Representation of Sparse Networks with Symmetric Nonnegative Embeddings
Many models for undirected graphs are based on factorizing the graph's adjacency matrix; these models find a vector representation of each node such that the predicted probability of a link between two nodes increases with the similarity (dot product) of their associated vectors.
On the Power of Edge Independent Graph Models
We prove that subject to a bounded overlap condition, which ensures that the model does not simply memorize a single graph, edge independent models are inherently limited in their ability to generate graphs with high triangle and other subgraph densities.
Coresets for Classification -- Simplified and Strengthened
It also does not depend on the specific loss function, so a single coreset can be used in multiple training scenarios.
DeepWalking Backwards: From Embeddings Back to Graphs
Our findings are a step towards a more rigorous understanding of exactly what information embeddings encode about the input graph, and why this information is useful for learning tasks.
Faster Kernel Matrix Algebra via Density Estimation
In particular, we consider estimating the sum of kernel matrix entries, along with its top eigenvalue and eigenvector.
Faster Kernel Interpolation for Gaussian Processes
Structured kernel interpolation (SKI) is among the most scalable methods: by placing inducing points on a dense grid and using structured matrix algebra, SKI achieves per-iteration time of O(n + m log m) for approximate inference.
Intervention Efficient Algorithms for Approximate Learning of Causal Graphs
Our goal is to recover the directions of all causal or ancestral relations in $G$, via a minimum cost set of interventions.
Estimation of Shortest Path Covariance Matrices
Such matrices generalize Toeplitz and circulant covariance matrices and are widely applied in signal processing applications, where the covariance between two measurements depends on the (shortest path) distance between them in time or space.
Model-specific Data Subsampling with Influence Functions
Model selection requires repeatedly evaluating models on a given dataset and measuring their relative performances.
Hutch++: Optimal Stochastic Trace Estimation
This improves on the ubiquitous Hutchinson's estimator, which requires $O(1/\epsilon^2)$ matrix-vector products.
Subspace Embeddings Under Nonlinear Transformations
We consider low-distortion embeddings for subspaces under \emph{entrywise nonlinear transformations}.
Fourier Sparse Leverage Scores and Approximate Kernel Learning
Bounding Fourier sparse leverage scores under various measures is of pure mathematical interest in approximation theory, and our work extends existing results for the uniform measure [Erd17, CP19a].
Node Embeddings and Exact Low-Rank Representations of Complex Networks
In this work we show that the results of Seshadhri et al. are intimately connected to the model they use rather than the low-dimensional structure of complex networks.
InfiniteWalk: Deep Network Embeddings as Laplacian Embeddings with a Nonlinearity
We study the objective in the limit as T goes to infinity, which allows us to simplify the expression of Qiu et al. We prove that this limiting objective corresponds to factoring a simple transformation of the pseudoinverse of the graph Laplacian, linking DeepWalk to extensive prior work in spectral graph embeddings.
Efficient Intervention Design for Causal Discovery with Latents
We consider recovering a causal graph in presence of latent variables, where we seek to minimize the cost of interventions used in the recovery process.
Projection-Cost-Preserving Sketches: Proof Strategies and Constructions
In this note we illustrate how common matrix approximation methods, such as random projection and random sampling, yield projection-cost-preserving sketches, as introduced in [FSS13, CEM+15].
Importance Sampling via Local Sensitivity
Unfortunately, sensitivity sampling is difficult to apply since (1) it is unclear how to efficiently compute the sensitivity scores and (2) the sample size required is often impractically large.
Toward a Characterization of Loss Functions for Distribution Learning
We aim to understand this fact, taking an axiomatic approach to the design of loss functions for learning distributions.
Sample Efficient Toeplitz Covariance Estimation
In addition to results that hold for any Toeplitz $T$, we further study the important setting when $T$ is close to low-rank, which is often the case in practice.
Learning to Prune: Speeding up Repeated Computations
We present an algorithm that learns to maximally prune the search space on repeated computations, thereby reducing runtime while provably outputting the correct solution each period with high probability.
Winner-Take-All Computation in Spiking Neural Networks
We provide efficient constructions of WTA circuits in our stochastic spiking neural network model, as well as lower bounds in terms of the number of auxiliary neurons required to drive convergence to WTA in a given number of steps.
Simple Heuristics Yield Provable Algorithms for Masked Low-Rank Approximation
In particular, for rank $k' > k$ depending on the $public\ coin\ partition\ number$ of $W$, the heuristic outputs rank-$k'$ $L$ with cost$(L) \leq OPT + \epsilon \|A\|_F^2$.
A Universal Sampling Method for Reconstructing Signals with Simple Fourier Transforms
We formalize this intuition by showing that, roughly, a continuous signal from a given class can be approximately reconstructed using a number of samples proportional to the *statistical dimension* of the allowed power spectrum of that class.
Inferring Networks From Random Walk-Based Node Similarities
In this work we consider a privacy threat to a social network in which an attacker has access to a subset of random walk-based node similarities, such as effective resistances (i. e., commute times) or personalized PageRank scores.
A Basic Compositional Model for Spiking Neural Networks
We define two operators on SNNs: a $composition$ $operator$, which supports modeling of SNNs as combinations of smaller SNNs, and a $hiding$ $operator$, which reclassifies some output behavior of an SNN as internal.
Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees
Qualitatively, our results are twofold: on the one hand, we show that random Fourier feature approximation can provably speed up kernel ridge regression under reasonable assumptions.
Learning Networks from Random Walk-Based Node Similarities
In this work we consider a privacy threat to a social network in which an attacker has access to a subset of random walk-based node similarities, such as effective resistances (i. e., commute times) or personalized PageRank scores.
Minimizing Polarization and Disagreement in Social Networks
We perform an empirical study of our proposed methods on synthetic and real-world data that verify their value as mining tools to better understand the trade-off between of disagreement and polarization.
Recursive Sampling for the Nystrom Method
We give the first algorithm for kernel Nystrom approximation that runs in linear time in the number of training points and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions.
Is Input Sparsity Time Possible for Kernel Low-Rank Approximation?
Low-rank approximation is a common tool used to accelerate kernel methods: the $n \times n$ kernel matrix $K$ is approximated via a rank-$k$ matrix $\tilde K$ which can be stored in much less space and processed more quickly.
Stability of the Lanczos Method for Matrix Function Approximation
In exact arithmetic, the method's error after $k$ iterations is bounded by the error of the best degree-$k$ polynomial uniformly approximating $f(x)$ on the range $[\lambda_{min}(A), \lambda_{max}(A)]$.
Data Structures and Algorithms Numerical Analysis
Neuro-RAM Unit with Applications to Similarity Testing and Compression in Spiking Neural Networks
Randomization allows us to solve this task with a very compact network, using $O \left (\frac{\sqrt{n}\log n}{\epsilon}\right)$ auxiliary neurons, which is sublinear in the input size.
Spectrum Approximation Beyond Fast Matrix Multiplication: Algorithms and Hardness
We thus effectively compute a histogram of the spectrum, which can stand in for the true singular values in many applications.
Sublinear Time Low-Rank Approximation of Positive Semidefinite Matrices
We show how to compute a relative-error low-rank approximation to any positive semidefinite (PSD) matrix in sublinear time, i. e., for any $n \times n$ PSD matrix $A$, in $\tilde O(n \cdot poly(k/\epsilon))$ time we output a rank-$k$ matrix $B$, in factored form, for which $\|A-B\|_F^2 \leq (1+\epsilon)\|A-A_k\|_F^2$, where $A_k$ is the best rank-$k$ approximation to $A$.
Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks
In this paper, we focus on the important `winner-take-all' (WTA) problem, which is analogous to a neural leader election unit: a network consisting of $n$ input neurons and $n$ corresponding output neurons must converge to a state in which a single output corresponding to a firing input (the `winner') fires, while all other outputs remain silent.
Faster Eigenvector Computation via Shift-and-Invert Preconditioning
We give faster algorithms and improved sample complexities for estimating the top eigenvector of a matrix $\Sigma$ -- i. e. computing a unit vector $x$ such that $x^T \Sigma x \ge (1-\epsilon)\lambda_1(\Sigma)$: Offline Eigenvector Estimation: Given an explicit $A \in \mathbb{R}^{n \times d}$ with $\Sigma = A^TA$, we show how to compute an $\epsilon$ approximate top eigenvector in time $\tilde O([nnz(A) + \frac{d*sr(A)}{gap^2} ]* \log 1/\epsilon )$ and $\tilde O([\frac{nnz(A)^{3/4} (d*sr(A))^{1/4}}{\sqrt{gap}} ] * \log 1/\epsilon )$.
Recursive Sampling for the Nyström Method
We give the first algorithm for kernel Nystr\"om approximation that runs in *linear time in the number of training points* and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions.
Principal Component Projection Without Principal Component Analysis
To achieve our results, we first observe that ridge regression can be used to obtain a "smooth projection" onto the top principal components.
Input Sparsity Time Low-Rank Approximation via Ridge Leverage Score Sampling
Our method is based on a recursive sampling scheme for computing a representative subset of $A$'s columns, which is then used to find a low-rank approximation.
Robust Shift-and-Invert Preconditioning: Faster and More Sample Efficient Algorithms for Eigenvector Computation
Combining our algorithm with previous work to initialize $x_0$, we obtain a number of improved sample complexity and runtime results.
Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition
We give the first provable runtime improvement on Simultaneous Iteration: a simple randomized block Krylov method, closely related to the classic Block Lanczos algorithm, gives the same guarantees in just $\tilde{O}(1/\sqrt{\epsilon})$ iterations and performs substantially better experimentally.
Dimensionality Reduction for k-Means Clustering and Low Rank Approximation
We show how to approximate a data matrix $\mathbf{A}$ with a much smaller sketch $\mathbf{\tilde A}$ that can be used to solve a general class of constrained k-rank approximation problems to within $(1+\epsilon)$ error.
Uniform Sampling for Matrix Approximation
In addition to an improved understanding of uniform sampling, our main proof introduces a structural result of independent interest: we show that every matrix can be made to have low coherence by reweighting a small subset of its rows.
