Improved Induction of Narrative Chains via Cross-Document Relations

1 code implementation • *SEM (NAACL) 2022 • Andrew Blair-Stanek, Benjamin Van Durme

The standard approach for inducing narrative chains considers statistics gathered per individual document.

BLT: Can Large Language Models Handle Basic Legal Text?

1 code implementation • 16 Nov 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

We find that the best publicly available LLMs like GPT-4, Claude, and {PaLM 2} currently perform poorly at basic legal text handling.

OpenAI Cribbed Our Tax Example, But Can GPT-4 Really Do Tax?

no code implementations • 15 Sep 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

The authors explain where OpenAI got the tax law example in its livestream demonstration of GPT-4, why GPT-4 got the wrong answer, and how it fails to reliably calculate taxes.

InteractiveIE: Towards Assessing the Strength of Human-AI Collaboration in Improving the Performance of Information Extraction

no code implementations • 24 May 2023 • Ishani Mondal, Michelle Yuan, Anandhavelu N, Aparna Garimella, Francis Ferraro, Andrew Blair-Stanek, Benjamin Van Durme, Jordan Boyd-Graber

Learning template based information extraction from documents is a crucial yet difficult task.

Question Answering Question Generation +1

Can GPT-3 Perform Statutory Reasoning?

1 code implementation • 13 Feb 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

Statutory reasoning is the task of reasoning with facts and statutes, which are rules written in natural language by a legislature.

A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering

1 code implementation • 11 May 2020 • Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme

Legislation can be viewed as a body of prescriptive rules expressed in natural language.

Natural Language Understanding Question Answering +1