Japanese-English Sentence Translation Exercises Dataset for Automatic Grading

no code implementations • 6 Mar 2024 • Naoki Miura, Hiroaki Funayama, Seiya Kikuchi, Yuichiroh Matsubayashi, Yuya Iwase, Kentaro Inui

Using this dataset, we demonstrate the performance of baselines including finetuned BERT and GPT models with few-shot in-context learning.

Few-Shot Learning In-Context Learning +2

Assessing Step-by-Step Reasoning against Lexical Negation: A Case Study on Syllogism

1 code implementation • 23 Oct 2023 • Mengyu Ye, Tatsuki Kuribayashi, Jun Suzuki, Goro Kobayashi, Hiroaki Funayama

Large language models (LLMs) take advantage of step-by-step reasoning instructions, e. g., chain-of-thought (CoT) prompting.

Logical Reasoning Negation

Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring

no code implementations • 16 Jun 2022 • Hiroaki Funayama, Tasuku Sato, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Kentaro Inui

Towards guaranteeing high-quality predictions, we present the first study of exploring the use of human-in-the-loop framework for minimizing the grading cost while guaranteeing the grading quality by allowing a SAS model to share the grading task with a human grader.

Preventing Critical Scoring Errors in Short Answer Scoring with Confidence Estimation

no code implementations • ACL 2020 • Hiroaki Funayama, Shota Sasaki, Yuichiroh Matsubayashi, Tomoya Mizumoto, Jun Suzuki, Masato Mita, Kentaro Inui

We introduce a new task formulation of SAS that matches the actual usage.