no code implementations • 23 Dec 2024 • Łukasz Borchmann
ARC Challenge appears more difficult than ARC Easy for modern LLMs primarily due to an evaluation setup that prevents direct comparison of answer choices rather than inherent complexity.
no code implementations • 18 Nov 2024 • Marek Wydmuch, Łukasz Borchmann, Filip Graliński
Though large language models (LLMs) have demonstrated exceptional performance across numerous problems, their application to predictive tasks in relational databases remains largely unexplored.
1 code implementation • 30 Oct 2024 • Michał Pietruszka, Łukasz Borchmann, Aleksander Jędrosz, Paweł Morawiecki
We present a benchmark for large language models designed to tackle one of the most knowledge-intensive tasks in data science: writing feature engineering code, which requires domain knowledge in addition to a deep understanding of the underlying problem and data structure.
no code implementations • 8 Aug 2024 • Łukasz Borchmann, Michał Pietruszka, Wojciech Jaśkowski, Dawid Jurkiewicz, Piotr Halama, Paweł Józiak, Łukasz Garncarek, Paweł Liskowski, Karolina Szyndler, Andrzej Gretkowski, Julita Ołtusek, Gabriela Nowakowska, Artur Zawłocki, Łukasz Duhr, Paweł Dyda, Michał Turski
The vast portion of workloads employing LLMs involves answering questions grounded on PDF or scan content.
no code implementations • 28 May 2024 • Łukasz Borchmann
Evaluation is followed by analyses that suggest possible contamination of textual GPT-4 models and indicate the significant performance drop for lengthy documents.
1 code implementation • ICCV 2023 • Jordy Van Landeghem, Rubén Tito, Łukasz Borchmann, Michał Pietruszka, Paweł Józiak, Rafał Powalski, Dawid Jurkiewicz, Mickaël Coustaty, Bertrand Ackaert, Ernest Valveny, Matthew Blaschko, Sien Moens, Tomasz Stanisławek
We call on the Document AI (DocAI) community to reevaluate current methodologies and embrace the challenge of creating more practically-oriented benchmarks.
no code implementations • 8 Jun 2022 • Michał Pietruszka, Michał Turski, Łukasz Borchmann, Tomasz Dwojak, Gabriela Pałka, Karolina Szyndler, Dawid Jurkiewicz, Łukasz Garncarek
The output structure of database-like tables, consisting of values structured in horizontal rows and vertical columns identifiable by name, can cover a wide range of NLP tasks.
1 code implementation • 18 Feb 2021 • Rafał Powalski, Łukasz Borchmann, Dawid Jurkiewicz, Tomasz Dwojak, Michał Pietruszka, Gabriela Pałka
We address the challenging problem of Natural Language Comprehension beyond plain-text documents by introducing the TILT neural network architecture which simultaneously learns layout information, visual features, and textual semantics.
Ranked #7 on
Visual Question Answering (VQA)
on InfographicVQA
(using extra training data)
1 code implementation • CONLL 2020 • Tomasz Dwojak, Michał Pietruszka, Łukasz Borchmann, Jakub Chłędowski, Filip Graliński
This paper investigates various Transformer architectures on the WikiReading Information Extraction and Machine Reading Comprehension dataset.
no code implementations • 27 Oct 2020 • Łukasz Borchmann, Dawid Jurkiewicz, Filip Graliński, Tomasz Górecki
The paper presents a novel method of finding a fragment in a long temporal sequence similar to the set of shorter sequences.
Ranked #2 on
Semantic Retrieval
on Contract Discovery
1 code implementation • 8 Oct 2020 • Michał Pietruszka, Łukasz Borchmann, Filip Graliński
We propose a differentiable successive halving method of relaxing the top-k operator, rendering gradient-based optimization possible.
1 code implementation • ACL 2022 • Michał Pietruszka, Łukasz Borchmann, Łukasz Garncarek
A reduction of quadratic time and memory complexity to sublinear was achieved due to a robust trainable top-$k$ operator.
Ranked #2 on
Text Summarization
on arXiv Summarization Dataset
no code implementations • 15 Jun 2020 • Tomasz Dwojak, Michał Pietruszka, Łukasz Borchmann, Filip Graliński, Jakub Chłędowski
In this paper, we investigate the Dual-source Transformer architecture on the WikiReading information extraction and machine reading comprehension dataset.
no code implementations • SEMEVAL 2020 • Dawid Jurkiewicz, Łukasz Borchmann, Izabela Kosmala, Filip Graliński
This paper presents the winning system for the propaganda Technique Classification (TC) task and the second-placed system for the propaganda Span Identification (SI) task.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Łukasz Borchmann, Dawid Wiśniewski, Andrzej Gretkowski, Izabela Kosmala, Dawid Jurkiewicz, Łukasz Szałkiewicz, Gabriela Pałka, Karol Kaczmarek, Agnieszka Kaliska, Filip Graliński
We propose a new shared task of semantic retrieval from legal texts, in which a so-called contract discovery is to be performed, where legal clauses are extracted from documents, given a few examples of similar clauses from other legal acts.
Ranked #1 on
Semantic Retrieval
on Contract Discovery