no code implementations • 7 Oct 2024 • Si-An Chen, Lesly Miculicich, Julian Martin Eisenschlos, Zifeng Wang, Zilong Wang, Yanfei Chen, Yasuhisa Fujii, Hsuan-Tien Lin, Chen-Yu Lee, Tomas Pfister
Recent advancements in language models (LMs) have notably enhanced their ability to reason with tabular data, primarily through program-aided mechanisms that manipulate and analyze tables.
no code implementations • 3 Jun 2024 • Julian Martin Eisenschlos, Hernán Maina, Guido Ivetta, Luciana Benotti
We perform the first in-depth analysis of calibration methods and metrics for VQA with in-context learning LMMs.
no code implementations • 29 May 2024 • Syrine Krichene, Francesco Piccinno, Fangyu Liu, Julian Martin Eisenschlos
Chart-to-summary generation can help explore data, communicate insights, and help the visually impaired people.
no code implementations • 13 May 2024 • Mubashara Akhtar, Chenxi Pang, Andreea Marzoca, Yasemin Altun, Julian Martin Eisenschlos
Language models, potentially augmented with tool usage such as retrieval are becoming the go-to means of answering questions.
no code implementations • 9 Jan 2024 • Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, Tomas Pfister
We propose the Chain-of-Table framework, where tabular data is explicitly used in the reasoning chain as a proxy for intermediate thoughts.
Ranked #3 on Table-based Fact Verification on TabFact
no code implementations • 24 May 2023 • Xingchen Wan, Ruoxi Sun, Hootan Nakhost, Hanjun Dai, Julian Martin Eisenschlos, Sercan O. Arik, Tomas Pfister
A hallmark of modern large language models (LLMs) is their impressive general zero-shot and few-shot abilities, often elicited through in-context learning (ICL) via prompting.
no code implementations • 24 May 2023 • Jeremy R. Cole, Michael J. Q. Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, Jacob Eisenstein
We investigate question answering from this perspective, focusing on answering a subset of questions with a high degree of accuracy, from a set of questions in which many are inherently ambiguous.
no code implementations • 1 Mar 2023 • Jeremy R. Cole, Palak Jain, Julian Martin Eisenschlos, Michael J. Q. Zhang, Eunsol Choi, Bhuwan Dhingra
We propose representing factual changes between paired documents as question-answer pairs, where the answer to the same question differs between two versions.
1 code implementation • 20 Dec 2022 • Fangyu Liu, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, Yasemin Altun
Compared with a SOTA model finetuned on more than >28k data points, DePlot+LLM with just one-shot prompting achieves a 24. 0% improvement over finetuned SOTA on human-written queries from the task of chart QA.
Chart Question Answering Factual Inconsistency Detection in Chart Captioning +3
1 code implementation • 19 Dec 2022 • Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, Julian Martin Eisenschlos
Visual language data such as plots, charts, and infographics are ubiquitous in the human world.
Ranked #2 on Visual Question Answering on PlotQA-D2
no code implementations • 23 Nov 2022 • Aashna Jena, Vivek Gupta, Manish Shrivastava, Julian Martin Eisenschlos
Creating challenging tabular inference data is essential for learning complex reasoning.
no code implementations • 17 Oct 2022 • Ewa Andrejczuk, Julian Martin Eisenschlos, Francesco Piccinno, Syrine Krichene, Yasemin Altun
Encoder-only transformer models have been successfully applied to different table understanding tasks, as in TAPAS (Herzig et al., 2020).
no code implementations • 14 Oct 2022 • Iulia-Maria Comsa, Julian Martin Eisenschlos, Srini Narayanan
We propose a benchmark to assess the capability of large language models to reason with conventional metaphors.
no code implementations • 26 Sep 2022 • Fangyu Liu, Julian Martin Eisenschlos, Jeremy R. Cole, Nigel Collier
Language models (LMs) trained on raw texts have no direct access to the physical world.
no code implementations • 25 Sep 2022 • Julian Martin Eisenschlos, Jeremy R. Cole, Fangyu Liu, William W. Cohen
We introduce a new in-context learning paradigm to measure Large Language Models' (LLMs) ability to learn novel words during inference.
1 code implementation • EMNLP 2021 • Julian Martin Eisenschlos, Maharshi Gor, Thomas Müller, William W. Cohen
However, more than 20% of relational tables on the web have 20 or more rows (Cafarella et al., 2008), and these large tables present a challenge for current Transformer models, which are typically limited to 512 tokens.
Ranked #2 on Question Answering on HybridQA
no code implementations • 29 Jun 2021 • Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, Daniel Gillick, Jacob Eisenstein, William W. Cohen
We introduce a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time and highlight problems with LMs at either end of the spectrum -- those trained on specific slices of temporal data, as well as those trained on a wide range of temporal data.
1 code implementation • Findings (ACL) 2021 • Syrine Krichene, Thomas Müller, Julian Martin Eisenschlos
To improve efficiency while maintaining a high accuracy, we propose a new architecture, DoT, a double transformer model, that decomposes the problem into two sub-tasks: A shallow pruning transformer that selects the top-K tokens, followed by a deep task-specific transformer that takes as input those K tokens.
1 code implementation • NAACL 2021 • Julian Martin Eisenschlos, Bhuwan Dhingra, Jannis Bulian, Benjamin Börschinger, Jordan Boyd-Graber
We release FoolMeTwice (FM2 for short), a large dataset of challenging entailment pairs collected through a fun multi-player game.
no code implementations • SEMEVAL 2021 • Thomas Müller, Julian Martin Eisenschlos, Syrine Krichene
We adopt the binary TAPAS model of Eisenschlos et al. (2020) to this task.
1 code implementation • NAACL 2021 • Jonathan Herzig, Thomas Müller, Syrine Krichene, Julian Martin Eisenschlos
Recent advances in open-domain QA have led to strong models based on dense retrieval, but only focused on retrieving textual passages.
1 code implementation • Findings of the Association for Computational Linguistics 2020 • Julian Martin Eisenschlos, Syrine Krichene, Thomas Müller
To be able to use long examples as input of BERT models, we evaluate table pruning techniques as a pre-processing step to drastically improve the training and prediction efficiency at a moderate drop in accuracy.
Ranked #10 on Table-based Fact Verification on TabFact
1 code implementation • 29 Jun 2020 • Sebastian Prillo, Julian Martin Eisenschlos
While sorting is an important procedure in computer science, the argsort operator - which takes as input a vector and returns its sorting permutation - has a discrete image and thus zero gradients almost everywhere.
7 code implementations • ACL 2020 • Jonathan Herzig, Paweł Krzysztof Nowak, Thomas Müller, Francesco Piccinno, Julian Martin Eisenschlos
In this paper, we present TAPAS, an approach to question answering over tables without generating logical forms.
Ranked #1 on Semantic Parsing on SQA (Accuracy metric)
4 code implementations • IJCNLP 2019 • Julian Martin Eisenschlos, Sebastian Ruder, Piotr Czapla, Marcin Kardas, Sylvain Gugger, Jeremy Howard
Pretrained language models are promising particularly for low-resource languages as they only require unlabelled data.
Ranked #1 on Zero-shot Cross-Lingual Document Classification on Cross-Lingual Sentiment (CLS)- English to German - DVD
Cross-Lingual Document Classification Document Classification +2