no code implementations • 2 May 2024 • Jionghao Lin, Zifei Han, Danielle R. Thomas, Ashish Gurung, Shivang Gupta, Vincent Aleven, Kenneth R. Koedinger
Our findings indicate that: 1) using a few-shot approach, the GPT-4 model effectively identifies correct/incorrect trainees' responses from three training lessons with an average F1 score of 0. 84 and an AUC score of 0. 85; and 2) using the few-shot approach, the GPT-4 model adeptly rephrases incorrect trainees' responses into desired responses, achieving performance comparable to that of human experts.
no code implementations • 1 May 2024 • Jionghao Lin, Eason Chen, Zeifei Han, Ashish Gurung, Danielle R. Thomas, Wei Tan, Ngoc Dang Nguyen, Kenneth R. Koedinger
To quantify the quality of highlighted praise components identified by GPT models, we introduced a Modified Intersection over Union (M-IoU) score.
no code implementations • 4 Mar 2024 • Liang Zhang, Jionghao Lin, Conrad Borchers, John Sabatini, John Hollander, Meng Cao, Xiangen Hu
This research is motivated by the potential of LLMs to predict learning performance based on its inherent reasoning and computational capabilities.
no code implementations • 4 Feb 2024 • Zifei, Han, Jionghao Lin, Ashish Gurung, Danielle R. Thomas, Eason Chen, Conrad Borchers, Shivang Gupta, Kenneth R. Koedinger
The results indicate that the RAG prompt demonstrated more accurate performance (assessed by the level of hallucination and correctness in the generated assessment texts) and lower financial costs than the other strategies evaluated.
no code implementations • 29 Jan 2024 • Liang Zhang, Jionghao Lin, Conrad Borchers, Meng Cao, Xiangen Hu
Learning performance data (e. g., quiz scores and attempts) is significant for understanding learner engagement and knowledge mastery level.
no code implementations • 6 Jan 2024 • Sanjit Kakarla, Danielle Thomas, Jionghao Lin, Shivang Gupta, Kenneth R. Koedinger
By analyzing 50 real-life tutoring dialogues, we find both GPT-3. 5-Turbo and GPT-4 demonstrate proficiency in assessing the criteria related to reacting to students making errors.
no code implementations • 21 Aug 2023 • Chen Cao, Zijian Ding, Gyeong-Geon Lee, Jiajun Jiao, Jionghao Lin, Xiaoming Zhai
Our study demonstrates the potential of applying large language models to educational practice on STEM subjects.
no code implementations • 8 Aug 2023 • Cassie Chen Cao, Zijian Ding, Jionghao Lin, Frank Hopfgartner
This study investigates the use of Artificial Intelligence (AI)-powered, multi-role chatbots as a means to enhance learning experiences and foster engagement in computer science education.
no code implementations • 5 Jul 2023 • Dollaya Hirunyasiri, Danielle R. Thomas, Jionghao Lin, Kenneth R. Koedinger, Vincent Aleven
We found that both zero-shot and few-shot chain of thought approaches yield comparable results.
no code implementations • 27 Jun 2023 • Jionghao Lin, Danielle R. Thomas, Feifei Han, Shivang Gupta, Wei Tan, Ngoc Dang Nguyen, Kenneth R. Koedinger
Research demonstrates learners engaging in the process of producing explanations to support their reasoning, can have a positive impact on learning.
no code implementations • 15 Apr 2023 • Jionghao Lin, Wei Tan, Ngoc Dang Nguyen, David Lang, Lan Du, Wray Buntine, Richard Beare, Guanliang Chen, Dragan Gasevic
We note that many prior studies on classifying educational DAs employ cross entropy (CE) loss to optimize DA classifiers on low-resource data with imbalanced DA distribution.
no code implementations • 12 Apr 2023 • Wei Tan, Jionghao Lin, David Lang, Guanliang Chen, Dragan Gasevic, Lan Du, Wray Buntine
Then, the study investigates how the AL methods can select informative samples to support DA classifiers in the AL sampling process.