Search Results for author: Zhiyuan Pan

Found 1 papers, 0 papers with code

Evaluating Large Language Models with Runtime Behavior of Program Execution

no code implementations25 Mar 2024 Junkai Chen, Zhiyuan Pan, Xing Hu, Zhenhao Li, Ge Li, Xin Xia

Typically, they focus on predicting the input and output of a program, ignoring the evaluation of the intermediate behavior during program execution, as well as the logical consistency (e. g., the model should not give the correct output if the prediction of execution path is wrong) when performing the reasoning.

Cannot find the paper you are looking for? You can Submit a new open access paper.