1 code implementation • *SEM (NAACL) 2022 • Andrew Blair-Stanek, Benjamin Van Durme
The standard approach for inducing narrative chains considers statistics gathered per individual document.
no code implementations • 28 Jan 2025 • Andrew Blair-Stanek, Benjamin Van Durme
An LLM is stable if it reaches the same conclusion when asked the identical question multiple times.
1 code implementation • 16 Sep 2024 • Abe Bohan Hou, William Jurayj, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme
Large Language Models (LLMs) show promise as a writing aid for professionals performing legal analyses.
1 code implementation • 24 Jun 2024 • Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme
This dataset CLERC (Case Law Evaluation Retrieval Corpus), is constructed for training and evaluating models on their ability to (1) find corresponding citations for a given piece of legal analysis and to (2) compile the text of these citations (as well as previous context) into a cogent analysis that supports a reasoning goal.
1 code implementation • 16 Nov 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme
We find that the best publicly available LLMs like GPT-4 and Claude currently perform poorly on basic legal text handling.
no code implementations • 15 Sep 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme
The authors explain where OpenAI got the tax law example in its livestream demonstration of GPT-4, why GPT-4 got the wrong answer, and how it fails to reliably calculate taxes.
no code implementations • 24 May 2023 • Ishani Mondal, Michelle Yuan, Anandhavelu N, Aparna Garimella, Francis Ferraro, Andrew Blair-Stanek, Benjamin Van Durme, Jordan Boyd-Graber
Learning template based information extraction from documents is a crucial yet difficult task.
1 code implementation • 13 Feb 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme
Statutory reasoning is the task of reasoning with facts and statutes, which are rules written in natural language by a legislature.
1 code implementation • 11 May 2020 • Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme
Legislation can be viewed as a body of prescriptive rules expressed in natural language.