Search Results for author:
Found 7 papers, 3 papers with code
Hierarchical Transformers Are More Efficient Language Models
Transformer models yield impressive results on many NLP and sequence modeling tasks.
Ranked #4 on
Image Generation
on ImageNet 32x32
(bpd metric)
Focused Transformer: Contrastive Training for Context Scaling
This novel approach enhances the structure of the (key, value) space, enabling an extension of the context length.
Analysing The Impact of Sequence Composition on Language Model Pre-Training
In this work, we find that applying causal masking can lead to the inclusion of distracting information from previous documents during pre-training, which negatively impacts the performance of the models on language modelling and downstream tasks.
Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers
Thor increases a language model's success rate on the PISA dataset from $39\%$ to $57\%$, while solving $8. 2\%$ of problems neither language models nor automated theorem provers are able to solve on their own.
Ranked #2 on
Automated Theorem Proving
on miniF2F-test
Magnushammer: A Transformer-Based Approach to Premise Selection
By combining \method with a language-model-based automated theorem prover, we further improve the state-of-the-art proof success rate from $57. 0\%$ to $71. 0\%$ on the PISA benchmark using $4$x fewer parameters.
Explaining Competitive-Level Programming Solutions using LLMs
In this paper, we approach competitive-level programming problem-solving as a composite task of reasoning and code generation.
Structured Packing in LLM Training Improves Long Context Utilization
Recent developments in long-context large language models have attracted considerable attention.
