1 code implementation • NeurIPS 2023 • Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessica Wu, Joe Nudell, Joel Niklaus, John Nay, Jonathan H. Choi, Kevin Tobia, Margaret Hagan, Megan Ma, Michael Livermore, Nikon Rasumov-Rahe, Nils Holzenberger, Noam Kolt, Peter Henderson, Sean Rehaag, Sharad Goel, Shang Gao, Spencer Williams, Sunny Gandhi, Tom Zur, Varun Iyer, Zehua Li
The advent of large language models (LLMs) and their adoption by the legal community has given rise to the question: what types of legal reasoning can LLMs perform?
1 code implementation • 11 May 2020 • Nils Holzenberger, Andrew Blair-Stanek, Benjamin Van Durme
Legislation can be viewed as a body of prescriptive rules expressed in natural language.
1 code implementation • LREC (LAW) 2022 • Noah Weber, Anton Belyy, Nils Holzenberger, Rachel Rudinger, Benjamin Van Durme
Event schemas are structured knowledge sources defining typical real-world scenarios (e. g., going to an airport).
1 code implementation • 13 Feb 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme
Statutory reasoning is the task of reasoning with facts and statutes, which are rules written in natural language by a legislature.
1 code implementation • 16 Nov 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme
We find that the best publicly available LLMs like GPT-4, Claude, and {PaLM 2} currently perform poorly at basic legal text handling.
1 code implementation • ACL 2021 • Nils Holzenberger, Benjamin Van Durme
Statutory reasoning is the task of determining whether a legal statute, stated in natural language, applies to the text description of a case.
no code implementations • 21 Nov 2018 • Nils Holzenberger, Shruti Palaskar, Pranava Madhyastha, Florian Metze, Raman Arora
This shows it is possible to learn reliable representations across disparate, unaligned and noisy modalities, and encourages using the proposed approach on larger datasets.
no code implementations • CONLL 2019 • J. Edward Hu, Abhinav Singh, Nils Holzenberger, Matt Post, Benjamin Van Durme
Producing diverse paraphrases of a sentence is a challenging task.
no code implementations • 30 Dec 2019 • Nils Holzenberger, Raman Arora
Canonical correlation analysis (CCA) is a popular technique for learning representations that are maximally correlated across multiple views in data.
no code implementations • 25 May 2022 • Nils Holzenberger, Yunmo Chen, Benjamin Van Durme
Information Extraction (IE) researchers are mapping tasks to Question Answering (QA) in order to leverage existing large QA resources, and thereby improve data efficiency.
no code implementations • 15 Sep 2023 • Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme
The authors explain where OpenAI got the tax law example in its livestream demonstration of GPT-4, why GPT-4 got the wrong answer, and how it fails to reliably calculate taxes.
no code implementations • 12 Jan 2024 • Xinrui Zou, Ming Zhang, Nathaniel Weir, Benjamin Van Durme, Nils Holzenberger
We re-frame statutory reasoning as an analogy task, where each instance of the analogy task involves a combination of two instances of statutory reasoning.