Search Results for author: Cameron Musco

Found 44 papers, 10 papers with code

Active Sampling for Linear Regression Beyond the $\ell_2$ Norm

no code implementations9 Nov 2021 Cameron Musco, Christopher Musco, David P. Woodruff, Taisuke Yasuda

Our result resolves a recent open question of Chen and Derezi\'{n}ski, who gave near optimal bounds for the $\ell_1$ norm, and suboptimal bounds for $\ell_p$ regression with $p\in(1, 2)$.

An Interpretable Graph Generative Model with Heterophily

no code implementations4 Nov 2021 Sudhanshu Chanpuriya, Ryan A. Rossi, Anup Rao, Tung Mai, Nedim Lipka, Zhao Song, Cameron Musco

These models output the probabilities of edges existing between all pairs of nodes, and the probability of a link between two nodes increases with the dot product of vectors associated with the nodes.

Link Prediction

On the Power of Edge Independent Graph Models

1 code implementation NeurIPS 2021 Sudhanshu Chanpuriya, Cameron Musco, Konstantinos Sotiropoulos, Charalampos E. Tsourakakis

We prove that subject to a bounded overlap condition, which ensures that the model does not simply memorize a single graph, edge independent models are inherently limited in their ability to generate graphs with high triangle and other subgraph densities.

Coresets for Classification -- Simplified and Strengthened

no code implementations NeurIPS 2021 Tung Mai, Anup B. Rao, Cameron Musco

It also does not depend on the specific loss function, so a single coreset can be used in multiple training scenarios.

Active Learning Classification

DeepWalking Backwards: From Embeddings Back to Graphs

1 code implementation17 Feb 2021 Sudhanshu Chanpuriya, Cameron Musco, Konstantinos Sotiropoulos, Charalampos E. Tsourakakis

Our findings are a step towards a more rigorous understanding of exactly what information embeddings encode about the input graph, and why this information is useful for learning tasks.

Faster Kernel Matrix Algebra via Density Estimation

no code implementations16 Feb 2021 Arturs Backurs, Piotr Indyk, Cameron Musco, Tal Wagner

In particular, we consider estimating the sum of kernel matrix entries, along with its top eigenvalue and eigenvector.

Density Estimation

Faster Kernel Interpolation for Gaussian Processes

no code implementations28 Jan 2021 Mohit Yadav, Daniel Sheldon, Cameron Musco

Structured kernel interpolation (SKI) is among the most scalable methods: by placing inducing points on a dense grid and using structured matrix algebra, SKI achieves per-iteration time of O(n + m log m) for approximate inference.

Gaussian Processes

Intervention Efficient Algorithms for Approximate Learning of Causal Graphs

no code implementations27 Dec 2020 Raghavendra Addanki, Andrew Mcgregor, Cameron Musco

Our goal is to recover the directions of all causal or ancestral relations in $G$, via a minimum cost set of interventions.

Estimation of Shortest Path Covariance Matrices

no code implementations19 Nov 2020 Raj Kumar Maity, Cameron Musco

Such matrices generalize Toeplitz and circulant covariance matrices and are widely applied in signal processing applications, where the covariance between two measurements depends on the (shortest path) distance between them in time or space.

Model-specific Data Subsampling with Influence Functions

no code implementations20 Oct 2020 Anant Raj, Cameron Musco, Lester Mackey, Nicolo Fusi

Model selection requires repeatedly evaluating models on a given dataset and measuring their relative performances.

Model Selection

Hutch++: Optimal Stochastic Trace Estimation

1 code implementation19 Oct 2020 Raphael A. Meyer, Cameron Musco, Christopher Musco, David P. Woodruff

This improves on the ubiquitous Hutchinson's estimator, which requires $O(1/\epsilon^2)$ matrix-vector products.

Subspace Embeddings Under Nonlinear Transformations

no code implementations5 Oct 2020 Aarshvi Gajjar, Cameron Musco

We consider low-distortion embeddings for subspaces under \emph{entrywise nonlinear transformations}.

Fourier Sparse Leverage Scores and Approximate Kernel Learning

no code implementations NeurIPS 2020 Tamás Erdélyi, Cameron Musco, Christopher Musco

Bounding Fourier sparse leverage scores under various measures is of pure mathematical interest in approximation theory, and our work extends existing results for the uniform measure [Erd17, CP19a].

Active Learning

Node Embeddings and Exact Low-Rank Representations of Complex Networks

1 code implementation NeurIPS 2020 Sudhanshu Chanpuriya, Cameron Musco, Konstantinos Sotiropoulos, Charalampos E. Tsourakakis

In this work we show that the results of Seshadhri et al. are intimately connected to the model they use rather than the low-dimensional structure of complex networks.

InfiniteWalk: Deep Network Embeddings as Laplacian Embeddings with a Nonlinearity

1 code implementation29 May 2020 Sudhanshu Chanpuriya, Cameron Musco

We study the objective in the limit as T goes to infinity, which allows us to simplify the expression of Qiu et al. We prove that this limiting objective corresponds to factoring a simple transformation of the pseudoinverse of the graph Laplacian, linking DeepWalk to extensive prior work in spectral graph embeddings.

Learning Word Embeddings Multi-Label Classification

Efficient Intervention Design for Causal Discovery with Latents

no code implementations ICML 2020 Raghavendra Addanki, Shiva Prasad Kasiviswanathan, Andrew Mcgregor, Cameron Musco

We consider recovering a causal graph in presence of latent variables, where we seek to minimize the cost of interventions used in the recovery process.

Causal Discovery

Projection-Cost-Preserving Sketches: Proof Strategies and Constructions

no code implementations17 Apr 2020 Cameron Musco, Christopher Musco

In this note we illustrate how common matrix approximation methods, such as random projection and random sampling, yield projection-cost-preserving sketches, as introduced in [FSS13, CEM+15].

Importance Sampling via Local Sensitivity

no code implementations4 Nov 2019 Anant Raj, Cameron Musco, Lester Mackey

Unfortunately, sensitivity sampling is difficult to apply since (1) it is unclear how to efficiently compute the sensitivity scores and (2) the sample size required is often impractically large.

Toward a Characterization of Loss Functions for Distribution Learning

no code implementations NeurIPS 2019 Nika Haghtalab, Cameron Musco, Bo Waggoner

We aim to understand this fact, taking an axiomatic approach to the design of loss functions for learning distributions.

Density Estimation

Sample Efficient Toeplitz Covariance Estimation

no code implementations14 May 2019 Yonina C. Eldar, Jerry Li, Cameron Musco, Christopher Musco

In addition to results that hold for any Toeplitz $T$, we further study the important setting when $T$ is close to low-rank, which is often the case in practice.

Learning to Prune: Speeding up Repeated Computations

no code implementations26 Apr 2019 Daniel Alabi, Adam Tauman Kalai, Katrina Ligett, Cameron Musco, Christos Tzamos, Ellen Vitercik

We present an algorithm that learns to maximally prune the search space on repeated computations, thereby reducing runtime while provably outputting the correct solution each period with high probability.

Winner-Take-All Computation in Spiking Neural Networks

no code implementations25 Apr 2019 Nancy Lynch, Cameron Musco, Merav Parter

We provide efficient constructions of WTA circuits in our stochastic spiking neural network model, as well as lower bounds in terms of the number of auxiliary neurons required to drive convergence to WTA in a given number of steps.

Simple Heuristics Yield Provable Algorithms for Masked Low-Rank Approximation

no code implementations22 Apr 2019 Cameron Musco, Christopher Musco, David P. Woodruff

In particular, for rank $k' > k$ depending on the $public\ coin\ partition\ number$ of $W$, the heuristic outputs rank-$k'$ $L$ with cost$(L) \leq OPT + \epsilon \|A\|_F^2$.

Low-Rank Matrix Completion Tensor Decomposition

A Universal Sampling Method for Reconstructing Signals with Simple Fourier Transforms

no code implementations20 Dec 2018 Haim Avron, Michael Kapralov, Cameron Musco, Christopher Musco, Ameya Velingker, Amir Zandieh

We formalize this intuition by showing that, roughly, a continuous signal from a given class can be approximately reconstructed using a number of samples proportional to the *statistical dimension* of the allowed power spectrum of that class.

Inferring Networks From Random Walk-Based Node Similarities

1 code implementation NeurIPS 2018 Jeremy Hoskins, Cameron Musco, Christopher Musco, Babis Tsourakakis

In this work we consider a privacy threat to a social network in which an attacker has access to a subset of random walk-based node similarities, such as effective resistances (i. e., commute times) or personalized PageRank scores.

Anomaly Detection Graph Clustering +2

A Basic Compositional Model for Spiking Neural Networks

no code implementations12 Aug 2018 Nancy Lynch, Cameron Musco

This paper presents a formal, mathematical foundation for modeling and reasoning about the behavior of $synchronous$, $stochastic$ $Spiking$ $Neural$ $Networks$ $(SNNs)$.

Random Fourier Features for Kernel Ridge Regression: Approximation Bounds and Statistical Guarantees

no code implementations ICML 2017 Haim Avron, Michael Kapralov, Cameron Musco, Christopher Musco, Ameya Velingker, Amir Zandieh

Qualitatively, our results are twofold: on the one hand, we show that random Fourier feature approximation can provably speed up kernel ridge regression under reasonable assumptions.

Learning Networks from Random Walk-Based Node Similarities

1 code implementation23 Jan 2018 Jeremy G. Hoskins, Cameron Musco, Christopher Musco, Charalampos E. Tsourakakis

In this work we consider a privacy threat to a social network in which an attacker has access to a subset of random walk-based node similarities, such as effective resistances (i. e., commute times) or personalized PageRank scores.

Anomaly Detection Graph Clustering +2

Minimizing Polarization and Disagreement in Social Networks

3 code implementations28 Dec 2017 Cameron Musco, Christopher Musco, Charalampos E. Tsourakakis

We perform an empirical study of our proposed methods on synthetic and real-world data that verify their value as mining tools to better understand the trade-off between of disagreement and polarization.

Recommendation Systems

Recursive Sampling for the Nystrom Method

no code implementations NeurIPS 2017 Cameron Musco, Christopher Musco

We give the first algorithm for kernel Nystrom approximation that runs in linear time in the number of training points and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions.

Is Input Sparsity Time Possible for Kernel Low-Rank Approximation?

no code implementations NeurIPS 2017 Cameron Musco, David P. Woodruff

Low-rank approximation is a common tool used to accelerate kernel methods: the $n \times n$ kernel matrix $K$ is approximated via a rank-$k$ matrix $\tilde K$ which can be stored in much less space and processed more quickly.

Stability of the Lanczos Method for Matrix Function Approximation

1 code implementation25 Aug 2017 Cameron Musco, Christopher Musco, Aaron Sidford

In exact arithmetic, the method's error after $k$ iterations is bounded by the error of the best degree-$k$ polynomial uniformly approximating $f(x)$ on the range $[\lambda_{min}(A), \lambda_{max}(A)]$.

Data Structures and Algorithms Numerical Analysis

Neuro-RAM Unit with Applications to Similarity Testing and Compression in Spiking Neural Networks

no code implementations5 Jun 2017 Nancy Lynch, Cameron Musco, Merav Parter

Randomization allows us to solve this task with a very compact network, using $O \left (\frac{\sqrt{n}\log n}{\epsilon}\right)$ auxiliary neurons, which is sublinear in the input size.

Spectrum Approximation Beyond Fast Matrix Multiplication: Algorithms and Hardness

no code implementations13 Apr 2017 Cameron Musco, Praneeth Netrapalli, Aaron Sidford, Shashanka Ubaru, David P. Woodruff

We thus effectively compute a histogram of the spectrum, which can stand in for the true singular values in many applications.

Sublinear Time Low-Rank Approximation of Positive Semidefinite Matrices

no code implementations11 Apr 2017 Cameron Musco, David P. Woodruff

We show how to compute a relative-error low-rank approximation to any positive semidefinite (PSD) matrix in sublinear time, i. e., for any $n \times n$ PSD matrix $A$, in $\tilde O(n \cdot poly(k/\epsilon))$ time we output a rank-$k$ matrix $B$, in factored form, for which $\|A-B\|_F^2 \leq (1+\epsilon)\|A-A_k\|_F^2$, where $A_k$ is the best rank-$k$ approximation to $A$.

Computational Tradeoffs in Biological Neural Networks: Self-Stabilizing Winner-Take-All Networks

no code implementations6 Oct 2016 Nancy Lynch, Cameron Musco, Merav Parter

In this paper, we focus on the important `winner-take-all' (WTA) problem, which is analogous to a neural leader election unit: a network consisting of $n$ input neurons and $n$ corresponding output neurons must converge to a state in which a single output corresponding to a firing input (the `winner') fires, while all other outputs remain silent.

Distributed Computing

Faster Eigenvector Computation via Shift-and-Invert Preconditioning

no code implementations26 May 2016 Dan Garber, Elad Hazan, Chi Jin, Sham M. Kakade, Cameron Musco, Praneeth Netrapalli, Aaron Sidford

We give faster algorithms and improved sample complexities for estimating the top eigenvector of a matrix $\Sigma$ -- i. e. computing a unit vector $x$ such that $x^T \Sigma x \ge (1-\epsilon)\lambda_1(\Sigma)$: Offline Eigenvector Estimation: Given an explicit $A \in \mathbb{R}^{n \times d}$ with $\Sigma = A^TA$, we show how to compute an $\epsilon$ approximate top eigenvector in time $\tilde O([nnz(A) + \frac{d*sr(A)}{gap^2} ]* \log 1/\epsilon )$ and $\tilde O([\frac{nnz(A)^{3/4} (d*sr(A))^{1/4}}{\sqrt{gap}} ] * \log 1/\epsilon )$.

Stochastic Optimization

Recursive Sampling for the Nyström Method

1 code implementation24 May 2016 Cameron Musco, Christopher Musco

We give the first algorithm for kernel Nystr\"om approximation that runs in *linear time in the number of training points* and is provably accurate for all kernel matrices, without dependence on regularity or incoherence conditions.

Principal Component Projection Without Principal Component Analysis

no code implementations22 Feb 2016 Roy Frostig, Cameron Musco, Christopher Musco, Aaron Sidford

To achieve our results, we first observe that ridge regression can be used to obtain a "smooth projection" onto the top principal components.

Input Sparsity Time Low-Rank Approximation via Ridge Leverage Score Sampling

no code implementations23 Nov 2015 Michael B. Cohen, Cameron Musco, Christopher Musco

Our method is based on a recursive sampling scheme for computing a representative subset of $A$'s columns, which is then used to find a low-rank approximation.

Robust Shift-and-Invert Preconditioning: Faster and More Sample Efficient Algorithms for Eigenvector Computation

no code implementations29 Oct 2015 Chi Jin, Sham M. Kakade, Cameron Musco, Praneeth Netrapalli, Aaron Sidford

Combining our algorithm with previous work to initialize $x_0$, we obtain a number of improved sample complexity and runtime results.

Stochastic Optimization

Randomized Block Krylov Methods for Stronger and Faster Approximate Singular Value Decomposition

no code implementations NeurIPS 2015 Cameron Musco, Christopher Musco

We give the first provable runtime improvement on Simultaneous Iteration: a simple randomized block Krylov method, closely related to the classic Block Lanczos algorithm, gives the same guarantees in just $\tilde{O}(1/\sqrt{\epsilon})$ iterations and performs substantially better experimentally.

Dimensionality Reduction for k-Means Clustering and Low Rank Approximation

no code implementations24 Oct 2014 Michael B. Cohen, Sam Elder, Cameron Musco, Christopher Musco, Madalina Persu

We show how to approximate a data matrix $\mathbf{A}$ with a much smaller sketch $\mathbf{\tilde A}$ that can be used to solve a general class of constrained k-rank approximation problems to within $(1+\epsilon)$ error.

Dimensionality Reduction

Uniform Sampling for Matrix Approximation

no code implementations21 Aug 2014 Michael B. Cohen, Yin Tat Lee, Cameron Musco, Christopher Musco, Richard Peng, Aaron Sidford

In addition to an improved understanding of uniform sampling, our main proof introduces a structural result of independent interest: we show that every matrix can be made to have low coherence by reweighting a small subset of its rows.

Cannot find the paper you are looking for? You can Submit a new open access paper.