Generative Language Modeling for Automated Theorem Proving

7 Sep 2020  ·  Stanislas Polu, Ilya Sutskever ·

We explore the application of transformer-based language models to automated theorem proving. This work is motivated by the possibility that a major limitation of automated theorem provers compared to humans -- the generation of original mathematical terms -- might be addressable via generation from language models. We present an automated prover and proof assistant, GPT-f, for the Metamath formalization language, and analyze its performance. GPT-f found new short proofs that were accepted into the main Metamath library, which is to our knowledge, the first time a deep-learning based system has contributed proofs that were adopted by a formal mathematics community.

PDF Abstract

Datasets


  Add Datasets introduced or used in this paper
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Automated Theorem Proving Metamath set.mm GPT-f Percentage correct 56.2 # 1

Methods


No methods listed for this paper. Add relevant methods here