Search Results for author:
Found 86 papers, 5 papers with code
Local Signal Adaptivity: Provable Feature Learning in Neural Networks Beyond Kernels
We therefore propose the "local signal adaptivity" (LSA) phenomenon as one explanation for the superiority of neural networks over kernel methods.
Integrating Rankings into Quantized Scores in Peer Review
Our approach addresses the two aforementioned challenges by: (i) ensuring that rankings are incorporated into the updates scores in the same manner for all papers, thereby mitigating arbitrariness, and (ii) allowing to seamlessly use existing interfaces and workflows designed for scores.
PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review
Our fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the commonly used objective of maximizing the total quality over all papers.
Gradient Descent Learns One-hidden-layer CNN: Don't be Afraid of Spurious Local Minima
We consider the problem of learning a one-hidden-layer neural network with non-overlapping convolutional layer and ReLU activation, i. e., $f(\mathbf{Z}, \mathbf{w}, \mathbf{a}) = \sum_j a_j\sigma(\mathbf{w}^T\mathbf{Z}_j)$, in which both the convolutional weights $\mathbf{w}$ and the output weights $\mathbf{a}$ are parameters to be learned.
Regression with Comparisons: Escaping the Curse of Dimensionality with Ordinal Information
In supervised learning, we typically leverage a fully labeled dataset to design methods for function estimation or prediction.
Robust Nonparametric Regression under Huber's $ε$-contamination Model
We first show that a simple local binning median step can effectively remove the adversary noise and this median estimator is minimax optimal up to absolute constants over the H\"{o}lder function class with smoothness parameters smaller than or equal to 1.
How Many Samples are Needed to Estimate a Convolutional or Recurrent Neural Network?
It is widely believed that the practical success of Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) owes to the fact that CNNs and RNNs use a more compact parametric representation than their Fully-Connected Neural Network (FNN) counterparts, and consequently require fewer training examples to accurately estimate their parameters.
Local White Matter Architecture Defines Functional Brain Dynamics
If communication efficiency is fundamentally constrained by the integrity along the entire length of a white matter bundle, then variability in the functional dynamics of brain networks should be associated with variability in the local connectome.
Optimization of Smooth Functions with Noisy Observations: Local Minimax Rates
In this setup, an algorithm is allowed to adaptively query the underlying function at different locations and receives noisy evaluations of function values at the queried points (i. e. the algorithm has access to zeroth-order information).
Stochastic Zeroth-order Optimization in High Dimensions
We consider the problem of optimizing a high-dimensional convex function using stochastic zeroth-order queries.
Towards Understanding the Generalization Bias of Two Layer Convolutional Linear Classifiers with Gradient Descent
A major challenge in understanding the generalization of deep learning is to explain why (stochastic) gradient descent can exploit the network architecture to find solutions that have good generalization performance when using high capacity models.
Provably Correct Algorithms for Matrix Column Subset Selection with Selectively Sampled Data
We consider the problem of matrix column subset selection, which selects a subset of columns from an input matrix such that the input can be well approximated by the span of the selected columns.
On Computationally Tractable Selection of Experiments in Measurement-Constrained Regression Models
We derive computationally tractable methods to select a small subset of experiment settings from a large pool of given design points.
Near-Optimal Discrete Optimization for Experimental Design: A Regret Minimization Approach
The experimental design problem concerns the selection of k points from a potentially large design pool of p-dimensional vectors, so as to maximize the statistical efficiency regressed on the selected k design points.
Gradient Descent Can Take Exponential Time to Escape Saddle Points
Although gradient descent (GD) almost always escapes saddle points asymptotically [Lee et al., 2016], this paper shows that even with fairly natural random initialization schemes and non-pathological functions, GD can be significantly slowed down by saddle points, taking exponential time to escape.
Hypothesis Transfer Learning via Transformation Functions
We consider the Hypothesis Transfer Learning (HTL) problem where one incorporates a hypothesis trained on the source domain into the learning procedure of the target domain.
On the Power of Truncated SVD for General High-rank Matrix Estimation Problems
This observation leads to many interesting results on general high-rank matrix estimation problems, which we briefly summarize below ($A$ is an $n\times n$ high-rank PSD matrix and $A_k$ is the best rank-$k$ approximation of $A$): (1) High-rank matrix completion: By observing $\Omega(\frac{n\max\{\epsilon^{-4}, k^2\}\mu_0^2\|A\|_F^2\log n}{\sigma_{k+1}(A)^2})$ elements of $A$ where $\sigma_{k+1}\left(A\right)$ is the $\left(k+1\right)$-th singular value of $A$ and $\mu_0$ is the incoherence, the truncated SVD on a zero-filled matrix satisfies $\|\widehat{A}_k-A\|_F \leq (1+O(\epsilon))\|A-A_k\|_F$ with high probability.
Rate Optimal Estimation and Confidence Intervals for High-dimensional Regression with Missing Covariates
We consider the problems of estimation and of constructing component-wise confidence intervals in a sparse high-dimensional linear regression model when some covariates of the design matrix are missing completely at random.
Noise-Tolerant Interactive Learning from Pairwise Comparisons
We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive.
Computationally Efficient Robust Estimation of Sparse Functionals
Many conventional statistical procedures are extremely sensitive to seemingly minor deviations from modeling assumptions.
Subspace Learning from Extremely Compressed Measurements
Our theoretical results show that even a constant number of measurements per column suffices to approximate the principal subspace to arbitrary precision, provided that the number of vectors is large.
A Theoretical Analysis of Noisy Sparse Subspace Clustering on Dimensionality-Reduced Data
Subspace clustering is the problem of partitioning unlabeled data points into a number of clusters so that data points within one cluster lie approximately on a low-dimensional linear subspace.
Data Poisoning Attacks on Factorization-Based Collaborative Filtering
Recommendation and collaborative filtering systems are important in modern information and e-commerce applications.
Graph Connectivity in Noisy Sparse Subspace Clustering
A line of recent work (4, 19, 24, 20) provided strong theoretical guarantee for sparse subspace clustering (4), the state-of-the-art algorithm for subspace clustering, on both noiseless and noisy data sets.
Active Learning Algorithms for Graphical Model Selection
The problem of learning the structure of a high dimensional graphical model from data has received considerable attention in recent years.
A statistical perspective of sampling scores for linear regression
In this paper, we consider a statistical problem of learning a linear model from noisy samples.
Classification accuracy as a proxy for two sample testing
We prove two results that hold for all classifiers in any dimensions: if its true error remains $\epsilon$-better than chance for some $\epsilon>0$ as $d, n \to \infty$, then (a) the permutation-based test is consistent (has power approaching to one), (b) a computationally efficient test based on a Gaussian approximation of the null distribution is also consistent.
Minimax Lower Bounds for Linear Independence Testing
Linear independence testing is a fundamental information-theoretic and statistical problem that can be posed as follows: given $n$ points $\{(X_i, Y_i)\}^n_{i=1}$ from a $p+q$ dimensional multivariate distribution where $X_i \in \mathbb{R}^p$ and $Y_i \in\mathbb{R}^q$, determine whether $a^T X$ and $b^T Y$ are uncorrelated for every $a \in \mathbb{R}^p, b\in \mathbb{R}^q$ or not.
Signal Representations on Graphs: Tools and Applications
For each class, we provide an explicit definition of the graph signals and construct a corresponding graph dictionary with desirable properties.
Noise-adaptive Margin-based Active Learning and Lower Bounds under Tsybakov Noise Condition
We present a simple noise-robust margin-based active learning algorithm to find homogeneous (passing the origin) linear separators and analyze its error convergence when labels are corrupted by noise.
Extreme Compressive Sampling for Covariance Estimation
This paper studies the problem of estimating the covariance of a collection of vectors using only highly compressed measurements of each vector.
Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing
We formally characterize the power of popular tests for GDA like the Maximum Mean Discrepancy with the Gaussian kernel (gMMD) and bandwidth-dependent variants of the Energy Distance with the Euclidean norm (eED) in the high-dimensional MDA regime.
Signal Recovery on Graphs: Random versus Experimentally Designed Sampling
We study signal recovery on graphs based on two sampling strategies: random sampling and experimentally designed sampling.
An Analysis of Active Learning With Uniform Feature Noise
For larger $\sigma$, the \textit{unflattening} of the regression function on convolution with uniform noise, along with its local antisymmetry around the threshold, together yield a behaviour where noise \textit{appears} to be beneficial.
Algorithmic Connections Between Active Learning and Stochastic Convex Optimization
Combining these two parts yields an algorithm that solves stochastic convex optimization of uniformly convex and smooth functions using only noisy gradient signs by repeatedly performing active learning, achieves optimal rates and is adaptive to all unknown convexity and smoothness parameters.
On the Decreasing Power of Kernel and Distance based Nonparametric Hypothesis Tests in High Dimensions
This paper is about two related decision theoretic problems, nonparametric two-sample testing and independence testing.
On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives
The current literature is split into two kinds of tests - those which are consistent without any assumptions about how the distributions may differ (\textit{general} alternatives), and those which are designed to specifically test easier alternatives, like a difference in means (\textit{mean-shift} alternatives).
Confidence sets for persistence diagrams
Persistent homology is a method for probing topological properties of point clouds and functions.
On the Power of Adaptivity in Matrix Completion and Approximation
We show that adaptive sampling allows one to eliminate standard incoherence assumptions on the matrix row space that are necessary for passive sampling procedures.
Feature Selection For High-Dimensional Clustering
We provide explicit bounds on the error rate of the resulting clustering.
Efficient Sparse Clustering of High-Dimensional Non-spherical Gaussian Mixtures
We consider the problem of clustering data points in high dimensions, i. e. when the number of data points may be much smaller than the number of dimensions.
FuSSO: Functional Shrinkage and Selection Operator
We present the FuSSO, a functional analogue to the LASSO, that efficiently finds a sparse set of functional input covariates to regress a real-valued response against.
Recovering Graph-Structured Activations using Adaptive Compressive Measurements
We study the localization of a cluster of activated vertices in a graph, from adaptively designed compressive measurements.
Near-optimal Anomaly Detection in Graphs using Lovasz Extended Scan Statistic
The detection of anomalous activity in graphs is a statistical problem that arises in many applications, such as network surveillance, disease outbreak detection, and activity monitoring in social networks.
Low-Rank Matrix and Tensor Completion via Adaptive Sampling
In the absence of noise, we show that one can exactly recover a $n \times n$ matrix of rank $r$ from merely $\Omega(n r^{3/2}\log(r))$ matrix entries.
Tight Lower Bounds for Homology Inference
In this note we use a different construction based on the direct analysis of the likelihood ratio test to show that the upper bound of Niyogi, Smale and Weinberger is in fact tight, thus establishing rate optimal asymptotic minimax bounds for the problem.
Cluster Trees on Manifolds
In this paper we investigate the problem of estimating the cluster tree for a density $f$ supported on or near a smooth $d$-dimensional manifold $M$ isometrically embedded in $\mathbb{R}^D$.
Recovering Block-structured Activations Using Compressive Measurements
We consider the problems of detection and localization of a contiguous block of weak activation in a large matrix, from a small number of noisy, possibly adaptive, compressive (linear) measurements.
Minimax Theory for High-dimensional Gaussian Mixtures with Sparse Mean Separation
While several papers have investigated computationally and statistically efficient methods for learning Gaussian mixtures, precise minimax bounds for their statistical performance as well as fundamental limits in high-dimensional settings are not well-understood.
Density-sensitive semisupervised inference
Semisupervised methods are techniques for using labeled data $(X_1, Y_1),\ldots,(X_n, Y_n)$ together with unlabeled data $X_{n+1},\ldots, X_N$ to make predictions.
On the Bootstrap for Persistence Diagrams and Landscapes
Persistent homology probes topological properties from point clouds and functions.
Algebraic Topology Computational Geometry Applications
Gradient Descent Provably Optimizes Over-parameterized Neural Networks
One of the mysteries in the success of neural networks is randomly initialized first order methods like gradient descent can achieve zero training loss even though the objective function is non-convex and non-smooth.
Efficient Load Sampling for Worst-Case Structural Analysis Under Force Location Uncertainty
Our approach uses a computationally tractable experimental design method to select number of sample force locations based on geometry only, without inspecting the stress response that requires computationally expensive finite-element analysis.
How Many Samples are Needed to Estimate a Convolutional Neural Network?
We show that for an $m$-dimensional convolutional filter with linear activation acting on a $d$-dimensional input, the sample complexity of achieving population prediction error of $\epsilon$ is $\widetilde{O(m/\epsilon^2)$, whereas the sample-complexity for its FNN counterpart is lower bounded by $\Omega(d/\epsilon^2)$ samples.
Noise-Tolerant Interactive Learning Using Pairwise Comparisons
We study the problem of interactively learning a binary classifier using noisy labeling and pairwise comparison oracles, where the comparison oracle answers which one in the given two instances is more likely to be positive.
Differentially private subspace clustering
Subspace clustering is an unsupervised learning problem that aims at grouping data points into multiple ``clusters'' so that data points in a single cluster lie approximately on a low-dimensional linear subspace.
Minimax Localization of Structural Information in Large Noisy Matrices
We consider the problem of identifying a sparse set of relevant columns and rows in a large data matrix with highly corrupted entries.
Noise Thresholds for Spectral Clustering
Although spectral clustering has enjoyed considerable empirical success in machine learning, its theoretical properties are not yet fully developed.
Identifying graph-structured activation patterns in networks
We consider the problem of identifying an activation pattern in a complex, large-scale network that is embedded in very noisy measurements.
Unlabeled data: Now it helps, now it doesn't
We show that there are large classes of problems for which SSL can significantly outperform supervised learning, in finite sample regimes and sometimes also in terms of error convergence rates.
Uncorrelation and Evenness: a New Diversity-Promoting Regularizer
Latent space models (LSMs) provide a principled and effective way to extract hidden patterns from observed data.
Near-Optimal Design of Experiments via Regret Minimization
We consider computationally tractable methods for the experimental design problem, where k out of n design points of dimension p are selected so that certain optimality criteria are approximately satisfied.
Nonparametric Regression with Comparisons: Escaping the Curse of Dimensionality with Ordinal Information
Finally, we present experiments that show the efficacy of RR and investigate its robustness to various sources of noise and model-misspecification.
Thresholding Bandit Problem with Both Duels and Pulls
The Thresholding Bandit Problem (TBP) aims to find the set of arms with mean rewards greater than a given threshold.
Active Learning for Graph Neural Networks via Node Feature Propagation
Graph Neural Networks (GNNs) for prediction tasks like node classification or edge prediction have received increasing attention in recent machine learning from graphically structured data.
Zeroth Order Non-convex optimization with Dueling-Choice Bandits
We consider a novel setting of zeroth order non-convex optimization, where in addition to querying the function value at a given point, we can also duel two points and get the point with the larger function value.
Preference-based Reinforcement Learning with Finite-Time Guarantees
If preferences are stochastic, and the preference probability relates to the hidden reward values, we present algorithms for PbRL, both with and without a simulator, that are able to identify the best policy up to accuracy $\varepsilon$ with high probability.
Two-Sample Testing on Ranked Preference Data and the Role of Modeling Assumptions
We also provide testing algorithms and associated sample complexity bounds for the problem of two-sample testing with partial (or total) ranking data. Furthermore, we empirically evaluate our results via extensive simulations as well as two real-world datasets consisting of pairwise comparisons.
AlphaNet: Improving Long-Tail Classification By Combining Classifiers
Methods in long-tail learning focus on improving performance for data-poor (rare) classes; however, performance for such classes remains much lower than performance for more data-rich (frequent) classes.
Catch Me if I Can: Detecting Strategic Behaviour in Peer Assessment
We consider the issue of strategic behaviour in various peer-assessment tasks, including peer grading of exams or homeworks and peer review in hiring or promotions.
A Large Scale Randomized Controlled Trial on Herding in Peer-Review Discussions
Peer review is the backbone of academia and humans constitute a cornerstone of this process, being responsible for reviewing papers and making the final acceptance/rejection decisions.
A Novice-Reviewer Experiment to Address Scarcity of Qualified Reviewers in Large Conferences
Conference peer review constitutes a human-computation process whose importance cannot be overstated: not only it identifies the best submissions for acceptance, but, ultimately, it impacts the future of the whole research area by promoting some ideas and restraining others.
Prior and Prejudice: The Novice Reviewers' Bias against Resubmissions in Conference Peer Review
Modern machine learning and computer science conferences are experiencing a surge in the number of submissions that challenges the quality of peer review as the number of competent reviewers is growing at a much slower rate.
Smooth Bandit Optimization: Generalization to Hölder Space
We also study adaptation to unknown function smoothness over a continuous scale of H\"older spaces indexed by $\alpha$, with a bandit model selection approach applied with our proposed two-layer algorithms.
Active Learning Graph Neural Networks via Node Feature Propagation
Graph Neural Networks (GNNs) for prediction tasks like node classification or edge prediction have received increasing attention in recent machine learning from graphically structured data.
Best Arm Identification under Additive Transfer Bandits
We consider a variant of the best arm identification (BAI) problem in multi-armed bandits (MAB) in which there are two sets of arms (source and target), and the objective is to determine the best target arm while only pulling source arms.
Complete Policy Regret Bounds for Tallying Bandits
Policy regret is a well established notion of measuring the performance of an online learning algorithm against an adaptive adversary.
The Virtues of Laziness in Model-based RL: A Unified Objective and Algorithms
We propose a novel approach to addressing two fundamental challenges in Model-based Reinforcement Learning (MBRL): the computational expense of repeatedly finding a good policy in the learned model, and the objective mismatch between model fitting and policy computation.
Predicting the Initial Conditions of the Universe using a Deterministic Neural Network
Finding the initial conditions that led to the current state of the universe is challenging because it involves searching over an intractable input space of initial conditions, along with modeling their evolution via tools such as N-body simulations which are computationally expensive.
Adaptation to Misspecified Kernel Regularity in Kernelised Bandits
In continuum-armed bandit problems where the underlying function resides in a reproducing kernel Hilbert space (RKHS), namely, the kernelised bandit problems, an important open problem remains of how well learning algorithms can adapt if the regularity of the associated kernel function is unknown.
Weighted Tallying Bandits: Overcoming Intractability via Repeated Exposure Optimality
Motivated by this, a significant line of work has formalized settings where an action's loss is a function of the number of times that action was recently played in the prior $m$ timesteps, where $m$ corresponds to a bound on human memory capacity.
Deep learning powered real-time identification of insects using citizen science data
InsectNet can guide citizen science data collection, especially for invasive species where early detection is crucial.
Specifying and Solving Robust Empirical Risk Minimization Problems Using CVXPY
We consider robust empirical risk minimization (ERM), where model parameters are chosen to minimize the worst-case empirical loss when each data point varies over a given convex uncertainty set.
Goodhart's Law Applies to NLP's Explanation Benchmarks
First, we show that we can inflate a model's comprehensiveness and sufficiency scores dramatically without altering its predictions or explanations on in-distribution test inputs.
Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs
Vision tasks are characterized by the properties of locality and translation invariance.
