Search Results for author:
Found 37 papers, 27 papers with code
Magistral
We introduce Magistral, Mistral's first reasoning model and our own scalable reinforcement learning (RL) pipeline.
Pixtral 12B
Unlike many open-source models, Pixtral is also a cutting-edge text model for its size, and does not compromise on natural language performance to excel in multimodal tasks.
Mixtral of Experts
In particular, Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and multilingual benchmarks.
Ranked #11 on
Multi-task Language Understanding
on MML
Mistral 7B
We introduce Mistral 7B v0. 1, a 7-billion-parameter language model engineered for superior performance and efficiency.
Ranked #1 on
answerability prediction
on PeerQA
LLaMA: Open and Efficient Foundation Language Models
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters.
Ranked #3 on
Few-Shot Learning
on MedConceptsQA
Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search
Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data.
Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
In this work, we introduce Draft, Sketch, and Prove (DSP), a method that maps informal proofs to formal proof sketches, and uses the sketches to guide an automated prover by directing its search to easier sub-problems.
Ranked #3 on
Automated Theorem Proving
on miniF2F-valid
(Pass@100 metric)
HyperTree Proof Search for Neural Theorem Proving
With a similar computational budget, we improve the state of the art on the Lean-based miniF2F-curriculum dataset from 31% to 42% proving accuracy.
Ranked #1 on
Automated Theorem Proving
on Metamath set.mm
(Pass@32 metric)
End-to-end symbolic regression with transformers
Symbolic regression, the task of predicting the mathematical expression of a function from the observation of its values, is a difficult task which usually involves a two-step procedure: predicting the "skeleton" of the expression up to the choice of numerical constants, then fitting the constants by optimizing a non-convex loss function.
Deep Symbolic Regression for Recurrent Sequences
Symbolic regression, i. e. predicting a function from the observation of its values, is well-known to be a challenging task.
Leveraging Automated Unit Tests for Unsupervised Code Translation
With little to no parallel data available for programming languages, unsupervised methods are well-suited to source code translation.
DOBF: A Deobfuscation Pre-Training Objective for Programming Languages
Recent advances in self-supervised learning have dramatically improved the state of the art on a wide variety of tasks.
Learning advanced mathematical computations from examples
Using transformers over large generated datasets, we train models to learn mathematical properties of differential systems, such as local stability, behavior at infinity and controllability.
Unsupervised Translation of Programming Languages
We train our model on source code from open source GitHub projects, and show that it can translate functions between C++, Java, and Python with high accuracy.
Deep Learning for Symbolic Mathematics
Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data.
The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali--English and Sinhala--English
For machine translation, a vast majority of language pairs in the world are considered low-resource because they have little parallel data available.
Large Memory Layers with Product Keys
In our experiments we consider a dataset with up to 30 billion words, and we plug our memory layer in a state-of-the-art transformer-based architecture.
Augmenting Self-attention with Persistent Memory
More precisely, we augment the self-attention layers with persistent memory vectors that play a similar role as the feed-forward layer.
Ranked #5 on
Language Modelling
on Text8
Multiple-Attribute Text Rewriting
The dominant approach to unsupervised "style transfer" in text is based on the idea of learning a latent representation, which is independent of the attributes specifying its "style".
The FLoRes Evaluation Datasets for Low-Resource Machine Translation: Nepali-English and Sinhala-English
For machine translation, a vast majority of language pairs in the world are considered low-resource because they have little parallel data available.
Cross-lingual Language Model Pretraining
On unsupervised machine translation, we obtain 34. 3 BLEU on WMT'16 German-English, improving the previous state of the art by more than 9 BLEU.
Multiple-Attribute Text Style Transfer
The dominant approach to unsupervised "style transfer" in text is based on the idea of learning a latent representation, which is independent of the attributes specifying its "style".
Phrase-Based \& Neural Unsupervised Machine Translation
Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs.
XNLI: Evaluating Cross-lingual Sentence Representations
State-of-the-art natural language processing systems rely on supervision in the form of annotated data to learn competent models.
Ranked #6 on
Natural Language Inference
on XNLI French
Cross-Lingual Natural Language Inference
Machine Translation
+2
What you can cram into a single \$\&!\#* vector: Probing sentence embeddings for linguistic properties
Although much effort has recently been devoted to training high-quality sentence embeddings, we still have a poor understanding of what they are capturing.
What you can cram into a single vector: Probing sentence embeddings for linguistic properties
Although much effort has recently been devoted to training high-quality sentence embeddings, we still have a poor understanding of what they are capturing.
Phrase-Based & Neural Unsupervised Machine Translation
Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs.
Ranked #2 on
Machine Translation
on WMT2016 English-Russian
Fader Networks:Manipulating Images by Sliding Attributes
This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space.
Unsupervised Machine Translation Using Monolingual Corpora Only
By learning to reconstruct in both languages from this shared feature space, the model effectively learns to translate without using any labeled data.
Ranked #7 on
Machine Translation
on WMT2016 German-English
Word Translation Without Parallel Data
We finally describe experiments on the English-Esperanto low-resource language pair, on which there only exists a limited amount of parallel data, to show the potential impact of our method in fully unsupervised machine translation.
Ranked #2 on
Word Alignment
on en-es
Fader Networks: Manipulating Images by Sliding Attributes
This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space.
Playing FPS Games with Deep Reinforcement Learning
Advances in deep reinforcement learning have allowed autonomous agents to perform well on Atari games, often outperforming humans, using only raw pixels to make their decisions.
Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning
We introduce polyglot language models, recurrent neural network models trained to predict symbol sequences in many different languages using shared representations of symbols and conditioning on typological information about the language to be predicted.
Neural Architectures for Named Entity Recognition
State-of-the-art named entity recognition systems rely heavily on hand-crafted features and domain-specific knowledge in order to learn effectively from the small, supervised training corpora that are available.
Ranked #11 on
Named Entity Recognition (NER)
on CoNLL++
Massively Multilingual Word Embeddings
We introduce new methods for estimating and evaluating embeddings of words in more than fifty languages in a single shared embedding space.
