Natural Language Deduction with Incomplete Information

1 code implementation • 1 Nov 2022 • Zayne Sprague, Kaj Bostrom, Swarat Chaudhuri, Greg Durrett

A growing body of work studies how to answer a question or verify a claim by generating a natural language "proof": a chain of deductive inferences yielding the answer based on a set of premises.

Text Generation

Natural Language Deduction through Search over Statement Compositions

no code implementations • 16 Jan 2022 • Kaj Bostrom, Zayne Sprague, Swarat Chaudhuri, Greg Durrett

In settings from fact-checking to question answering, we frequently want to know whether a collection of evidence (premises) entails a hypothesis.

Fact Checking Question Answering

Flexible Generation of Natural Language Deductions

1 code implementation • EMNLP 2021 • Kaj Bostrom, Xinyu Zhao, Swarat Chaudhuri, Greg Durrett

Natural language is an attractive representation for this purpose -- it is both highly expressive and easy for humans to understand.

Byte Pair Encoding is Suboptimal for Language Model Pretraining

no code implementations • Findings of the Association for Computational Linguistics 2020 • Kaj Bostrom, Greg Durrett

We analyze differences between BPE and unigram LM tokenization, finding that the latter method recovers subword units that align more closely with morphology and avoids problems stemming from BPE's greedy construction procedure.

Language Modelling