Search Results for author:
Found 15 papers, 4 papers with code
The Power of External Memory in Increasing Predictive Model Capacity
One way of introducing sparsity into deep networks is by attaching an external table of parameters that is sparsely looked up at different layers of the network.
A Theoretical View on Sparsely Activated Networks
After representing LSH-based sparse networks with our model, we prove that sparse networks can match the approximation power of dense networks on Lipschitz functions.
Do More Negative Samples Necessarily Hurt in Contrastive Learning?
Recent investigations in noise contrastive estimation suggest, both empirically as well as theoretically, that while having more "negative samples" in the contrastive loss improves downstream classification performance initially, beyond a threshold, it hurts downstream performance due to a "collision-coverage" trade-off.
Statistical Estimation from Dependent Data
We consider a general statistical estimation problem wherein binary labels across different observations are not independent conditioned on their feature vectors, but dependent, capturing settings where e. g. these observations are collected on a spatial domain, a temporal domain, or a social network, which induce dependencies.
For Manifold Learning, Deep Neural Networks can be Locality Sensitive Hash Functions
We provide theoretical and empirical evidence that neural representations can be viewed as LSH-like functions that map each input to an embedding that is a function of solely the informative $\gamma$ and invariant to $\theta$, effectively recovering the manifold identifier $\gamma$.
Minimax Estimation of Conditional Moment Models
We develop an approach for estimating models described via conditional moment restrictions, with a prototypical application being non-parametric instrumental variable regression.
Learning Ising models from one or multiple samples
As corollaries of our main theorem, we derive bounds when the model's interaction matrix is a (sparse) linear combination of known matrices, or it belongs to a finite set, or to a high-dimensional manifold.
Logistic-Regression with peer-group effects via inference in higher order Ising models
In this work we study extensions of these to models with higher-order sufficient statistics, modeling behavior on a social network with peer-group effects.
Learning from weakly dependent data under Dobrushin's condition
Indeed, we show that the standard complexity measures of Gaussian and Rademacher complexities and VC dimension are sufficient measures of complexity for the purposes of bounding the generalization error and learning rates of hypothesis classes in our setting.
Regression from Dependent Observations
The standard linear and logistic regression models assume that the response variables are independent, but share the same linear relationship to their corresponding vectors of covariates.
HOGWILD!-Gibbs can be PanAccurate
Hence, the expectation of any function that is Lipschitz with respect to a power of the Hamming distance, can be estimated with a bias that grows logarithmically in $n$.
From Soft Classifiers to Hard Decisions: How fair can we be?
We study the feasibility of achieving various fairness properties by post-processing calibrated scores, and then show that deferring post-processors allow for more fairness conditions to hold on the final decision.
Concentration of Multilinear Functions of the Ising Model with Applications to Network Data
We prove near-tight concentration of measure for polynomial functions of the Ising model under high temperature.
Testing Symmetric Markov Chains from a Single Trajectory
We initiate the study of Markov chain testing, assuming access to a single trajectory of a Markov Chain.
Testing Ising Models
Given samples from an unknown multivariate distribution $p$, is it possible to distinguish whether $p$ is the product of its marginals versus $p$ being far from every product distribution?
