Search Results for author: Andrew Blair-Stanek

Found 9 papers, 6 papers with code

Improved Induction of Narrative Chains via Cross-Document Relations

1 code implementation *SEM (NAACL) 2022 Andrew Blair-Stanek, Benjamin Van Durme

The standard approach for inducing narrative chains considers statistics gathered per individual document.

LLMs Provide Unstable Answers to Legal Questions

no code implementations28 Jan 2025 Andrew Blair-Stanek, Benjamin Van Durme

An LLM is stable if it reaches the same conclusion when asked the identical question multiple times.

CLERC: A Dataset for Legal Case Retrieval and Retrieval-Augmented Analysis Generation

1 code implementation24 Jun 2024 Abe Bohan Hou, Orion Weller, Guanghui Qin, Eugene Yang, Dawn Lawrie, Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme

This dataset CLERC (Case Law Evaluation Retrieval Corpus), is constructed for training and evaluating models on their ability to (1) find corresponding citations for a given piece of legal analysis and to (2) compile the text of these citations (as well as previous context) into a cogent analysis that supports a reasoning goal.

Information Retrieval RAG +2

BLT: Can Large Language Models Handle Basic Legal Text?

1 code implementation16 Nov 2023 Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

We find that the best publicly available LLMs like GPT-4 and Claude currently perform poorly on basic legal text handling.

OpenAI Cribbed Our Tax Example, But Can GPT-4 Really Do Tax?

no code implementations15 Sep 2023 Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

The authors explain where OpenAI got the tax law example in its livestream demonstration of GPT-4, why GPT-4 got the wrong answer, and how it fails to reliably calculate taxes.

Can GPT-3 Perform Statutory Reasoning?

1 code implementation13 Feb 2023 Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme

Statutory reasoning is the task of reasoning with facts and statutes, which are rules written in natural language by a legislature.

Cannot find the paper you are looking for? You can Submit a new open access paper.