Search Results for author:
Found 12 papers, 6 papers with code
Salsa Fresca: Angular Embeddings and Pre-Training for ML Attacks on Learning With Errors
Our architecture improvements enable scaling to larger-dimension LWE problems: this work is the first instance of ML attacks recovering sparse binary secrets in dimension $n=1024$, the smallest dimension used in practice for homomorphic encryption applications of LWE where sparse binary secrets are proposed.
Learning the greatest common divisor: explaining transformer predictions
Models trained from uniform operands only learn a handful of GCD (up to $38$ GCD $\leq100$).
Length Generalization in Arithmetic Transformers
We find that relative position embeddings enable length generalization for simple tasks, such as addition: models trained on $5$-digit numbers can perform $15$-digit sums.
Contrastive Distillation Is a Sample-Efficient Self-Supervised Loss Policy for Transfer Learning
Traditional approaches to RL have focused on learning decision policies directly from episodic decisions, while slowly and implicitly learning the semantics of compositional representations needed for generalization.
What is my math transformer doing? -- Three results on interpretability and generalization
This paper investigates the failure cases and out-of-distribution behavior of transformers trained on matrix inversion and eigenvalue decomposition.
SALSA: Attacking Lattice Cryptography with Transformers
Currently deployed public-key cryptosystems will be vulnerable to attacks by full-scale quantum computers.
End-to-end symbolic regression with transformers
Symbolic regression, the task of predicting the mathematical expression of a function from the observation of its values, is a difficult task which usually involves a two-step procedure: predicting the "skeleton" of the expression up to the choice of numerical constants, then fitting the constants by optimizing a non-convex loss function.
Deep Symbolic Regression for Recurrent Sequences
Symbolic regression, i. e. predicting a function from the observation of its values, is well-known to be a challenging task.
A deep language model to predict metabolic network equilibria
We show that deep learning models, and especially architectures like the Transformer, originally intended for natural language, can be trained on randomly generated datasets to predict to very high accuracy both the qualitative and quantitative features of metabolic networks.
Linear algebra with transformers
Transformers can learn to perform numerical computations from examples only.
Learning advanced mathematical computations from examples
Using transformers over large generated datasets, we train models to learn mathematical properties of differential systems, such as local stability, behavior at infinity and controllability.
Deep Learning for Symbolic Mathematics
Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data.
