Search Results for author: Guillaume Lample

Found 35 papers, 26 papers with code

Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs

3 code implementations21 Oct 2022 Albert Q. Jiang, Sean Welleck, Jin Peng Zhou, Wenda Li, Jiacheng Liu, Mateja Jamnik, Timothée Lacroix, Yuhuai Wu, Guillaume Lample

In this work, we introduce Draft, Sketch, and Prove (DSP), a method that maps informal proofs to formal proof sketches, and uses the sketches to guide an automated prover by directing its search to easier sub-problems.

Ranked #3 on Automated Theorem Proving on miniF2F-valid (Pass@100 metric)

Automated Theorem Proving Language Modelling

HyperTree Proof Search for Neural Theorem Proving

no code implementations23 May 2022 Guillaume Lample, Marie-Anne Lachaux, Thibaut Lavril, Xavier Martinet, Amaury Hayat, Gabriel Ebner, Aurélien Rodriguez, Timothée Lacroix

With a similar computational budget, we improve the state of the art on the Lean-based miniF2F-curriculum dataset from 31% to 42% proving accuracy.

Automated Theorem Proving

End-to-end symbolic regression with transformers

3 code implementations22 Apr 2022 Pierre-Alexandre Kamienny, Stéphane d'Ascoli, Guillaume Lample, François Charton

Symbolic regression, the task of predicting the mathematical expression of a function from the observation of its values, is a difficult task which usually involves a two-step procedure: predicting the "skeleton" of the expression up to the choice of numerical constants, then fitting the constants by optimizing a non-convex loss function.

regression Symbolic Regression

Deep Symbolic Regression for Recurrent Sequences

no code implementations12 Jan 2022 Stéphane d'Ascoli, Pierre-Alexandre Kamienny, Guillaume Lample, François Charton

Symbolic regression, i. e. predicting a function from the observation of its values, is well-known to be a challenging task.

regression Symbolic Regression

Target Conditioning for One-to-Many Generation

no code implementations Findings of the Association for Computational Linguistics 2020 Marie-Anne Lachaux, Armand Joulin, Guillaume Lample

In this paper, we propose to explicitly model this one-to-many mapping by conditioning the decoder of a NMT model on a latent variable that represents the domain of target sentences.

Decoder Diversity +4

Learning advanced mathematical computations from examples

1 code implementation ICLR 2021 François Charton, Amaury Hayat, Guillaume Lample

Using transformers over large generated datasets, we train models to learn mathematical properties of differential systems, such as local stability, behavior at infinity and controllability.

Unsupervised Translation of Programming Languages

9 code implementations NeurIPS 2020 Marie-Anne Lachaux, Baptiste Roziere, Lowik Chanussot, Guillaume Lample

We train our model on source code from open source GitHub projects, and show that it can translate functions between C++, Java, and Python with high accuracy.

Code Translation Translation +1

Deep Learning for Symbolic Mathematics

7 code implementations ICLR 2020 Guillaume Lample, François Charton

Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data.

Large Memory Layers with Product Keys

8 code implementations NeurIPS 2019 Guillaume Lample, Alexandre Sablayrolles, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou

In our experiments we consider a dataset with up to 30 billion words, and we plug our memory layer in a state-of-the-art transformer-based architecture.

Language Modelling

Augmenting Self-attention with Persistent Memory

2 code implementations2 Jul 2019 Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample, Herve Jegou, Armand Joulin

More precisely, we augment the self-attention layers with persistent memory vectors that play a similar role as the feed-forward layer.

Language Modelling Translation

Multiple-Attribute Text Rewriting

no code implementations ICLR 2019 Guillaume Lample, Sandeep Subramanian, Eric Smith, Ludovic Denoyer, Marc'Aurelio Ranzato, Y-Lan Boureau

The dominant approach to unsupervised "style transfer" in text is based on the idea of learning a latent representation, which is independent of the attributes specifying its "style".

Attribute Disentanglement +2

Cross-lingual Language Model Pretraining

16 code implementations NeurIPS 2019 Guillaume Lample, Alexis Conneau

On unsupervised machine translation, we obtain 34. 3 BLEU on WMT'16 German-English, improving the previous state of the art by more than 9 BLEU.

Language Modelling Natural Language Understanding +2

Multiple-Attribute Text Style Transfer

3 code implementations1 Nov 2018 Sandeep Subramanian, Guillaume Lample, Eric Michael Smith, Ludovic Denoyer, Marc'Aurelio Ranzato, Y-Lan Boureau

The dominant approach to unsupervised "style transfer" in text is based on the idea of learning a latent representation, which is independent of the attributes specifying its "style".

Attribute Disentanglement +3

Phrase-Based \& Neural Unsupervised Machine Translation

no code implementations EMNLP 2018 Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, Marc{'}Aurelio Ranzato

Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs.

Denoising NMT +3

What you can cram into a single vector: Probing sentence embeddings for linguistic properties

6 code implementations3 May 2018 Alexis Conneau, German Kruszewski, Guillaume Lample, Loïc Barrault, Marco Baroni

Although much effort has recently been devoted to training high-quality sentence embeddings, we still have a poor understanding of what they are capturing.

General Classification Sentence +2

Phrase-Based & Neural Unsupervised Machine Translation

15 code implementations EMNLP 2018 Guillaume Lample, Myle Ott, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato

Machine translation systems achieve near human-level performance on some languages, yet their effectiveness strongly relies on the availability of large amounts of parallel sentences, which hinders their applicability to the majority of language pairs.

NMT Sentence +2

Fader Networks:Manipulating Images by Sliding Attributes

no code implementations NeurIPS 2017 Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, Marc'Aurelio Ranzato

This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space.

Attribute Decoder

Unsupervised Machine Translation Using Monolingual Corpora Only

14 code implementations ICLR 2018 Guillaume Lample, Alexis Conneau, Ludovic Denoyer, Marc'Aurelio Ranzato

By learning to reconstruct in both languages from this shared feature space, the model effectively learns to translate without using any labeled data.

Sentence Translation +1

Word Translation Without Parallel Data

19 code implementations ICLR 2018 Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou

We finally describe experiments on the English-Esperanto low-resource language pair, on which there only exists a limited amount of parallel data, to show the potential impact of our method in fully unsupervised machine translation.

Cross-Lingual Word Embeddings Translation +4

Fader Networks: Manipulating Images by Sliding Attributes

3 code implementations1 Jun 2017 Guillaume Lample, Neil Zeghidour, Nicolas Usunier, Antoine Bordes, Ludovic Denoyer, Marc'Aurelio Ranzato

This paper introduces a new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space.

Attribute Decoder

Playing FPS Games with Deep Reinforcement Learning

7 code implementations18 Sep 2016 Guillaume Lample, Devendra Singh Chaplot

Advances in deep reinforcement learning have allowed autonomous agents to perform well on Atari games, often outperforming humans, using only raw pixels to make their decisions.

Game of Doom Q-Learning +2

Polyglot Neural Language Models: A Case Study in Cross-Lingual Phonetic Representation Learning

no code implementations NAACL 2016 Yulia Tsvetkov, Sunayana Sitaram, Manaal Faruqui, Guillaume Lample, Patrick Littell, David Mortensen, Alan W. black, Lori Levin, Chris Dyer

We introduce polyglot language models, recurrent neural network models trained to predict symbol sequences in many different languages using shared representations of symbols and conditioning on typological information about the language to be predicted.

Representation Learning

Neural Architectures for Named Entity Recognition

43 code implementations NAACL 2016 Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, Chris Dyer

State-of-the-art named entity recognition systems rely heavily on hand-crafted features and domain-specific knowledge in order to learn effectively from the small, supervised training corpora that are available.

Named Entity Recognition

Massively Multilingual Word Embeddings

1 code implementation5 Feb 2016 Waleed Ammar, George Mulcaire, Yulia Tsvetkov, Guillaume Lample, Chris Dyer, Noah A. Smith

We introduce new methods for estimating and evaluating embeddings of words in more than fifty languages in a single shared embedding space.

Multilingual Word Embeddings Text Categorization

Cannot find the paper you are looking for? You can Submit a new open access paper.