Search Results for author:
Found 7 papers, 5 papers with code
Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways
We pretrained our masked language models with three ingredients: an initial pretraining with music data, training on shorter sequences before training on longer ones, and masking specific tokens to target some of the BLiMP subtasks.
MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning
We evaluate a range of LLMs and prompting techniques on this dataset and characterize the gaps that remain for techniques like chain-of-thought to perform robust reasoning.
Deductive Additivity for Planning of Natural Language Proofs
Specifically, we evaluate whether embedding spaces exhibit a property we call deductive additivity: the sum of premise statement embeddings should be close to embeddings of conclusions based on those premises.
Natural Language Deduction with Incomplete Information
A growing body of work studies how to answer a question or verify a claim by generating a natural language "proof": a chain of deductive inferences yielding the answer based on a set of premises.
Natural Language Deduction through Search over Statement Compositions
In settings from fact-checking to question answering, we frequently want to know whether a collection of evidence (premises) entails a hypothesis.
Flexible Generation of Natural Language Deductions
Natural language is an attractive representation for this purpose -- it is both highly expressive and easy for humans to understand.
Byte Pair Encoding is Suboptimal for Language Model Pretraining
We analyze differences between BPE and unigram LM tokenization, finding that the latter method recovers subword units that align more closely with morphology and avoids problems stemming from BPE's greedy construction procedure.
