Testing GPT-4 with Wolfram Alpha and Code Interpreter plug-ins on math and science problems
This report describes a test of the large language model GPT-4 with the Wolfram Alpha and the Code Interpreter plug-ins on 105 original problems in science and math, at the high school and college levels, carried out in June-August 2023. Our tests suggest that the plug-ins significantly enhance GPT's ability to solve these problems. Having said that, there are still often "interface" failures; that is, GPT often has trouble formulating problems in a way that elicits useful answers from the plug-ins. Fixing these interface failures seems like a central challenge in making GPT a reliable tool for college-level calculation problems.
PDF Abstract
MATH
SciBench
Results from the Paper
Submit
results from this paper
to get state-of-the-art GitHub badges and help the
community compare results to other papers.
Methods
Absolute Position Encodings •
Adam •
Attention Dropout •
BPE •
Cosine Annealing •
Dense Connections •
Discriminative Fine-Tuning •
Dropout •
GELU •
GPT •
GPT-4 •
Label Smoothing •
Layer Normalization •
Linear Layer •
Linear Warmup With Cosine Annealing •
Multi-Head Attention •
Position-Wise Feed-Forward Layer •
Residual Connection •
Scaled Dot-Product Attention •
Softmax •
Transformer •
Weight Decay
