Search Results for author: Szymon Tworkowski

Found 7 papers, 3 papers with code

Analysing The Impact of Sequence Composition on Language Model Pre-Training

1 code implementation21 Feb 2024 Yu Zhao, Yuanbin Qu, Konrad Staniszewski, Szymon Tworkowski, Wei Liu, Piotr Miłoś, Yuxiang Wu, Pasquale Minervini

In this work, we find that applying causal masking can lead to the inclusion of distracting information from previous documents during pre-training, which negatively impacts the performance of the models on language modelling and downstream tasks.

In-Context Learning Language Modelling +1

Explaining Competitive-Level Programming Solutions using LLMs

no code implementations11 Jul 2023 Jierui Li, Szymon Tworkowski, Yingying Wu, Raymond Mooney

In this paper, we approach competitive-level programming problem-solving as a composite task of reasoning and code generation.

Code Generation Explanation Generation

Thor: Wielding Hammers to Integrate Language Models and Automated Theorem Provers

no code implementations22 May 2022 Albert Q. Jiang, Wenda Li, Szymon Tworkowski, Konrad Czechowski, Tomasz Odrzygóźdź, Piotr Miłoś, Yuhuai Wu, Mateja Jamnik

Thor increases a language model's success rate on the PISA dataset from $39\%$ to $57\%$, while solving $8. 2\%$ of problems neither language models nor automated theorem provers are able to solve on their own.

Automated Theorem Proving

Cannot find the paper you are looking for? You can Submit a new open access paper.