Search Results for author:
Found 7 papers, 5 papers with code
Gradient Descent Fails to Learn High-frequency Functions and Modular Arithmetic
In the novel framework, the hardness of a class is usually quantified by the variance of the gradient with respect to a random choice of a target function.
Intractability of Learning the Discrete Logarithm with Gradient-Based Methods
The discrete logarithm problem is a fundamental challenge in number theory with significant implications for cryptographic protocols.
Long-Tail Theory under Gaussian Mixtures
We suggest a simple Gaussian mixture model for data generation that complies with Feldman's long tail theory (2020).
Autoencoders for a manifold learning problem with a Jacobian rank constraint
This constraint is included in the objective function as a new term, namely a squared Ky-Fan $k$-antinorm of the Jacobian function.
Speeding Up Entmax
Softmax is the de facto standard in modern neural networks for language processing when it comes to normalizing logits.
The Rediscovery Hypothesis: Language Models Need to Meet Linguistics
In this paper, we study whether linguistic knowledge is a necessary condition for the good performance of modern language models, which we call the \textit{rediscovery hypothesis}.
Semantics- and Syntax-related Subvectors in the Skip-gram Embeddings
We show that the skip-gram embedding of any word can be decomposed into two subvectors which roughly correspond to semantic and syntactic roles of the word.
