Search Results for author: Jaromir Savelka

Found 22 papers, 3 papers with code

ECHR: Legal Corpus for Argument Mining

no code implementations COLING (ArgMining) 2020 Prakash Poudyal, Jaromir Savelka, Aagje Ieven, Marie Francine Moens, Teresa Goncalves, Paulo Quaresma

The corpus is annotated in terms of three types of clauses useful in argument mining: premise, conclusion, and non-argument parts of the text.

Argument Mining Relation

A Comparative Study of AI-Generated (GPT-4) and Human-crafted MCQs in Programming Education

no code implementations5 Dec 2023 Jacob Doughty, Zipiao Wan, Anishka Bompelli, Jubahed Qayum, Taozhi Wang, Juran Zhang, Yujia Zheng, Aidan Doyle, Pragnya Sridhar, Arav Agarwal, Christopher Bogart, Eric Keylor, Can Kultur, Jaromir Savelka, Majd Sakr

While there is a growing body of research in computing education on utilizing large language models (LLMs) in generation and engagement with coding exercises, the use of LLMs for generating programming MCQs has not been extensively explored.

Multiple-choice

From Text to Structure: Using Large Language Models to Support the Development of Legal Expert Systems

1 code implementation1 Nov 2023 Samyar Janatian, Hannes Westermann, Jinzhe Tan, Jaromir Savelka, Karim Benyekhlef

We use LLMs to create pathways from legislation, according to the JusticeBot methodology for legal decision support systems, evaluate the pathways and compare them to manually created pathways.

Efficient Classification of Student Help Requests in Programming Courses Using Large Language Models

no code implementations31 Oct 2023 Jaromir Savelka, Paul Denny, Mark Liffiton, Brad Sheese

This study evaluates the performance of the GPT-3. 5 and GPT-4 models for classifying help requests from students in an introductory programming class.

Using Large Language Models to Support Thematic Analysis in Empirical Legal Studies

no code implementations28 Oct 2023 Jakub Drápal, Hannes Westermann, Jaromir Savelka

We propose a novel framework facilitating effective collaboration of a legal expert with a large language model (LLM) for generating initial codes (phase 2 of thematic analysis), searching for themes (phase 3), and classifying the data in terms of the themes (to kick-start phase 4).

Language Modelling Large Language Model +1

The Robots are Here: Navigating the Generative AI Revolution in Computing Education

no code implementations1 Oct 2023 James Prather, Paul Denny, Juho Leinonen, Brett A. Becker, Ibrahim Albluwi, Michelle Craig, Hieke Keuning, Natalie Kiesler, Tobias Kohn, Andrew Luxton-Reilly, Stephen MacNeil, Andrew Peterson, Raymond Pettit, Brent N. Reeves, Jaromir Savelka

Second, we report the findings of a survey of computing students and instructors from across 20 countries, capturing prevailing attitudes towards LLMs and their use in computing education contexts.

Ethics

LLMediator: GPT-4 Assisted Online Dispute Resolution

no code implementations27 Jul 2023 Hannes Westermann, Jaromir Savelka, Karim Benyekhlef

In this article, we introduce LLMediator, an experimental platform designed to enhance online dispute resolution (ODR) by utilizing capabilities of state-of-the-art large language models (LLMs) such as GPT-4.

Harnessing LLMs in Curricular Design: Using GPT-4 to Support Authoring of Learning Objectives

no code implementations30 Jun 2023 Pragnya Sridhar, Aidan Doyle, Arav Agarwal, Christopher Bogart, Jaromir Savelka, Majd Sakr

We evaluated 127 LOs that were automatically generated based on a carefully crafted prompt (detailed guidelines on high-quality LOs authoring) submitted to GPT-4 for conceptual modules and projects of an AI Practitioner course.

Can GPT-4 Support Analysis of Textual Data in Tasks Requiring Highly Specialized Domain Expertise?

no code implementations24 Jun 2023 Jaromir Savelka, Kevin D. Ashley, Morgan A Gray, Hannes Westermann, Huihui Xu

We observed that, with a relatively minor decrease in performance, GPT-4 can perform batch predictions leading to significant cost reductions.

Thrilled by Your Progress! Large Language Models (GPT-4) No Longer Struggle to Pass Assessments in Higher Education Programming Courses

no code implementations15 Jun 2023 Jaromir Savelka, Arav Agarwal, Marshall An, Chris Bogart, Majd Sakr

Additionally, we analyze the assessments that were not handled well by GPT-4 to understand the current limitations of the model, as well as its capabilities to leverage feedback provided by an auto-grader.

Multiple-choice

Explaining Legal Concepts with Augmented Large Language Models (GPT-4)

no code implementations15 Jun 2023 Jaromir Savelka, Kevin D. Ashley, Morgan A. Gray, Hannes Westermann, Huihui Xu

We compare the performance of a baseline setup, where GPT-4 is directly asked to explain a legal term, to an augmented approach, where a legal information retrieval module is used to provide relevant context to the model, in the form of sentences from case law.

Hallucination Information Retrieval +1

Unlocking Practical Applications in Legal Domain: Evaluation of GPT for Zero-Shot Semantic Annotation of Legal Texts

no code implementations8 May 2023 Jaromir Savelka

We evaluated the capability of a state-of-the-art generative pre-trained transformer (GPT) model to perform semantic annotation of short text snippets (one to few sentences) coming from legal documents of various types.

Sentence Zero-Shot Learning

Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?

no code implementations16 Mar 2023 Jaromir Savelka, Arav Agarwal, Christopher Bogart, YiFan Song, Majd Sakr

We evaluated the capability of generative pre-trained transformers (GPT), to pass assessments in introductory and intermediate Python programming courses at the postsecondary level.

Multiple-choice

Large Language Models (GPT) Struggle to Answer Multiple-Choice Questions about Code

no code implementations9 Mar 2023 Jaromir Savelka, Arav Agarwal, Christopher Bogart, Majd Sakr

While questions requiring to fill-in a blank in the code or completing a natural language statement about the snippet are handled rather successfully, MCQs that require analysis and/or reasoning about the code (e. g., what is true/false about the snippet, or what is its output) appear to be the most challenging.

Multiple-choice

Toward an Intelligent Tutoring System for Argument Mining in Legal Texts

no code implementations24 Oct 2022 Hannes Westermann, Jaromir Savelka, Vern R. Walker, Kevin D. Ashley, Karim Benyekhlef

We propose an adaptive environment (CABINET) to support caselaw analysis (identifying key argument elements) based on a novel cognitive computing framework that carefully matches various machine learning (ML) capabilities to the proficiency of a user.

Argument Mining

Data-Centric Machine Learning in the Legal Domain

no code implementations17 Jan 2022 Hannes Westermann, Jaromir Savelka, Vern R. Walker, Kevin D. Ashley, Karim Benyekhlef

The results also indicate that enhancements to a data set could be considered, alongside the advancement of the ML models, as an additional path for increasing classification performance on various tasks in AI & Law.

BIG-bench Machine Learning

Sentence Embeddings and High-speed Similarity Search for Fast Computer Assisted Annotation of Legal Documents

no code implementations21 Dec 2021 Hannes Westermann, Jaromir Savelka, Vern R. Walker, Kevin D. Ashley, Karim Benyekhlef

We use this observation in allowing annotators to quickly view and annotate sentences that are semantically similar to a given sentence, across an entire corpus of documents.

Sentence Sentence Embeddings

Lex Rosetta: Transfer of Predictive Models Across Languages, Jurisdictions, and Legal Domains

1 code implementation15 Dec 2021 Jaromir Savelka, Hannes Westermann, Karim Benyekhlef, Charlotte S. Alexander, Jayla C. Grant, David Restrepo Amariles, Rajaa El Hamdani, Sébastien Meeùs, Michał Araszkiewicz, Kevin D. Ashley, Alexandra Ashley, Karl Branting, Mattia Falduti, Matthias Grabmair, Jakub Harašta, Tereza Novotná, Elizabeth Tippett, Shiwanni Johnson

In this paper, we examine the use of multi-lingual sentence embeddings to transfer predictive models for functional segmentation of adjudicatory decisions across jurisdictions, legal systems (common and civil law), languages, and domains (i. e. contexts).

Segmentation Sentence +1

Cross-Domain Generalization and Knowledge Transfer in Transformers Trained on Legal Data

no code implementations15 Dec 2021 Jaromir Savelka, Hannes Westermann, Karim Benyekhlef

We analyze the ability of pre-trained language models to transfer knowledge among datasets annotated with different type systems and to generalize beyond the domain and dataset they were trained on.

Domain Generalization Sentence +1

Cannot find the paper you are looking for? You can Submit a new open access paper.