Search Results for author:
Found 15 papers, 5 papers with code
Towards a theory of model distillation
Distillation is the task of replacing a complicated machine learning model with a simpler model that approximates the original [BCNM06, HVD15].
PROPANE: Prompt design as an inverse problem
Carefully-designed prompts are key to inducing desired behavior in Large Language Models (LLMs).
When can transformers reason with abstract symbols?
We investigate the capabilities of transformer models on relational reasoning tasks.
Transformers learn through gradual rank increase
Our experiments support the theory and also show that phenomenon can occur in practice without the simplifying assumptions.
Tight conditions for when the NTK approximation is valid
We study when the neural tangent kernel (NTK) approximation is valid for training a model with the square loss.
SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics
For $d$-dimensional uniform Boolean or isotropic Gaussian data, our main conjecture states that the time complexity to learn a function $f$ with low-dimensional support is $\tilde\Theta (d^{\max(\mathrm{Leap}(f), 2)})$.
GULP: a prediction-based metric between representations
Comparing the representations learned by different neural networks has recently emerged as a key tool to understand various architectures and ultimately optimize them.
On the non-universality of deep learning: quantifying the cost of symmetry
We prove limitations on what neural networks trained by noisy gradient descent (GD) can efficiently learn.
The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks
It is currently known how to characterize functions that neural networks can learn with SGD for two extremal parameterizations: neural networks in the linear regime, and neural networks with no structural constraints.
The staircase property: How hierarchical structure can guide deep learning
This paper identifies a structural property of data distributions that enables deep neural networks to learn hierarchically.
Chow-Liu++: Optimal Prediction-Centric Learning of Tree Ising Models
In this paper, we introduce a new algorithm that carefully combines elements of the Chow-Liu algorithm with tree metric reconstruction methods to efficiently and optimally learn tree Ising models under a prediction-centric loss.
Wasserstein barycenters are NP-hard to compute
Moreover, our hardness results for computing Wasserstein barycenters extend to approximate computation, to seemingly simple cases of the problem, and to averaging probability distributions in other Optimal Transport metrics.
Hardness results for Multimarginal Optimal Transport problems
We demonstrate this toolkit by using it to establish the intractability of a number of MOT problems studied in the literature that have resisted previous algorithmic efforts.
Polynomial-time algorithms for Multimarginal Optimal Transport problems with structure
We illustrate this ease-of-use by developing poly(n, k) time algorithms for three general classes of MOT cost structures: (1) graphical structure; (2) set-optimization structure; and (3) low-rank plus sparse structure.
Wasserstein barycenters can be computed in polynomial time in fixed dimension
Computing Wasserstein barycenters is a fundamental geometric problem with widespread applications in machine learning, statistics, and computer graphics.
