no code implementations • LREC 2022 • Shanshan Xu, Katja Markert
We find that the model’s performance on CANLI does not correspond to its internal representation of CPH, which is the crucial linguistic ability central to the CANLI dataset.
no code implementations • 11 Feb 2024 • Shanshan Xu, T. Y. S. S Santosh, Oana Ichim, Barbara Plank, Matthias Grabmair
We observe limited alignment with the judge vote distribution.
no code implementations • 18 Oct 2023 • Shanshan Xu, T. Y. S. S Santosh, Oana Ichim, Isabella Risini, Barbara Plank, Matthias Grabmair
Overall, our case study reveals hitherto underappreciated complexities in creating benchmark datasets in legal NLP that revolve around identifying aspects of a case's facts supposedly relevant to its outcome.
1 code implementation • 17 Oct 2023 • Shanshan Xu, Leon Staufer, T. Y. S. S Santosh, Oana Ichim, Corina Heri, Matthias Grabmair
Our results demonstrate the challenging nature of the task with lower prediction performance and limited agreement between models and experts.
no code implementations • 28 Nov 2022 • Shanshan Xu, Irina Broda, Rashid Haddad, Marco Negrini, Matthias Grabmair
Recent work has demonstrated that natural language processing techniques can support consumer protection by automatically detecting unfair clauses in the Terms of Service (ToS) Agreement.
1 code implementation • 25 Oct 2022 • T. Y. S. S Santosh, Shanshan Xu, Oana Ichim, Matthias Grabmair
This work demonstrates that Legal Judgement Prediction systems without expert-informed adjustments can be vulnerable to shallow, distracting surface signals that arise from corpus construction, case distribution, and confounding factors.
no code implementations • 22 Oct 2022 • Abhishek Agarwal, Shanshan Xu, Matthias Grabmair
Summarizing legal decisions requires the expertise of law practitioners, which is both time- and cost-intensive.