Search Results for author:
Found 19 papers, 1 papers with code
Simple Mechanisms for Representing, Indexing and Manipulating Concepts
Deep networks typically learn concepts via classifiers, which involves setting up a model and training it via gradient descent to fit the concept-labeled data.
The Power of External Memory in Increasing Predictive Model Capacity
One way of introducing sparsity into deep networks is by attaching an external table of parameters that is sparsely looked up at different layers of the network.
A Theoretical View on Sparsely Activated Networks
After representing LSH-based sparse networks with our model, we prove that sparse networks can match the approximation power of dense networks on Lipschitz functions.
A Unified Cascaded Encoder ASR Model for Dynamic Model Sizes
In this paper, we propose a dynamic cascaded encoder Automatic Speech Recognition (ASR) model, which unifies models for different deployment scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Provable hierarchical lifelong learning with a sketch-based modular architecture
We propose a modular architecture for lifelong learning of hierarchically structured tasks.
One Network Fits All? Modular versus Monolithic Task Formulations in Neural Networks
Can deep learning solve multiple tasks simultaneously, even when they are unrelated and very different?
For Manifold Learning, Deep Neural Networks can be Locality Sensitive Hash Functions
We provide theoretical and empirical evidence that neural representations can be viewed as LSH-like functions that map each input to an embedding that is a function of solely the informative $\gamma$ and invariant to $\theta$, effectively recovering the manifold identifier $\gamma$.
Learning the gravitational force law and other analytic functions
We present experimental evidence that the many-body gravitational force function is easier to learn with ReLU networks as compared to networks with exponential activations.
How does the Mind store Information?
How we store information in our mind has been a major intriguing open question.
Recursive Sketches for Modular Deep Learning
The sketch summarizes essential information about the inputs and outputs of the network and can be used to quickly identify key components and summary statistics of the inputs.
On the Learnability of Deep Random Networks
In this paper we study the learnability of deep random networks from both theoretical and practical points of view.
Recovering the Lowest Layer of Deep Networks with High Threshold Activations
Giving provable guarantees for learning neural networks is a core challenge of machine learning theory.
Algorithms for $\ell_p$ Low-Rank Approximation
We consider the problem of approximating a given matrix by a low-rank matrix so as to minimize the entrywise $\ell_p$-approximation error, for any $p \geq 1$; the case $p = 2$ is the classical SVD problem.
Convergence Results for Neural Networks via Electrodynamics
Iterating, we show that gradient descent can be used to learn the entire network one node at a time.
Sparse Matrix Factorization
We investigate the problem of factorizing a matrix into several sparse matrices and propose an algorithm for this under randomness and sparsity assumptions.
A Differential Equations Approach to Optimizing Regret Trade-offs
To obtain our main result, we show that the optimal payoff functions have to satisfy the Hermite differential equation, and hence are given by the solutions to this equation.
Optimal amortized regret in every interval
In this paper we show a randomized algorithm that in an amortized sense gets a regret of $O(\sqrt x)$ for any interval when the sequence is partitioned into intervals arbitrarily.
Fractal structures in Adversarial Prediction
In this work we study how "fractal-like" processes arise in a prediction game where an adversary is generating a sequence of bits and an algorithm is trying to predict them.
Prediction strategies without loss
Moreover, for {\em any window of size $n$} the regret of our algorithm to any expert never exceeds $O(\sqrt{n(\log N+\log T)})$, where $N$ is the number of experts and $T$ is the time horizon, while maintaining the essentially zero loss property.
