no code implementations • 28 Apr 2024 • Atharva Naik, Jessica Ruhan Yin, Anusha Kamath, Qianou Ma, Sherry Tongshuang Wu, Charles Murray, Christopher Bogart, Majd Sakr, Carolyn P. Rose
An advantage of Large Language Models (LLMs) is their contextualization capability - providing different responses based on student inputs like solution strategy or prior discussion, to potentially better engage students than standard feedback.
no code implementations • 14 Apr 2024 • Arav Agarwal, Karthik Mittal, Aidan Doyle, Pragnya Sridhar, Zipiao Wan, Jacob Arthur Doughty, Jaromir Savelka, Majd Sakr
We conduct a preliminary study of the effect of GPT's temperature parameter on the diversity of GPT4-generated questions.
no code implementations • 30 Jan 2024 • Dachi Chen, Weitian Ding, Chen Liang, Chang Xu, Junwei Zhang, Majd Sakr
Training an effective Machine learning (ML) model is an iterative process that requires effort in multiple dimensions.
no code implementations • 5 Dec 2023 • Jacob Doughty, Zipiao Wan, Anishka Bompelli, Jubahed Qayum, Taozhi Wang, Juran Zhang, Yujia Zheng, Aidan Doyle, Pragnya Sridhar, Arav Agarwal, Christopher Bogart, Eric Keylor, Can Kultur, Jaromir Savelka, Majd Sakr
While there is a growing body of research in computing education on utilizing large language models (LLMs) in generation and engagement with coding exercises, the use of LLMs for generating programming MCQs has not been extensively explored.
no code implementations • 30 Jun 2023 • Pragnya Sridhar, Aidan Doyle, Arav Agarwal, Christopher Bogart, Jaromir Savelka, Majd Sakr
We evaluated 127 LOs that were automatically generated based on a carefully crafted prompt (detailed guidelines on high-quality LOs authoring) submitted to GPT-4 for conceptual modules and projects of an AI Practitioner course.
no code implementations • 15 Jun 2023 • Jaromir Savelka, Arav Agarwal, Marshall An, Chris Bogart, Majd Sakr
Additionally, we analyze the assessments that were not handled well by GPT-4 to understand the current limitations of the model, as well as its capabilities to leverage feedback provided by an auto-grader.
no code implementations • 16 Mar 2023 • Jaromir Savelka, Arav Agarwal, Christopher Bogart, YiFan Song, Majd Sakr
We evaluated the capability of generative pre-trained transformers (GPT), to pass assessments in introductory and intermediate Python programming courses at the postsecondary level.
no code implementations • 9 Mar 2023 • Jaromir Savelka, Arav Agarwal, Christopher Bogart, Majd Sakr
While questions requiring to fill-in a blank in the code or completing a natural language statement about the snippet are handled rather successfully, MCQs that require analysis and/or reasoning about the code (e. g., what is true/false about the snippet, or what is its output) appear to be the most challenging.
no code implementations • COLING 2018 • Halim-Antoine Boukaram, Nizar Habash, Micheline Ziadee, Majd Sakr
Automatic syntactic parsing for question constructions is a challenging task due to the paucity of training examples in most treebanks.