TASK
DATASET
MODEL
METRIC NAME
METRIC VALUE
GLOBAL RANK
EXTRA DATA
REMOVE
Arithmetic Reasoning
GSM8K
Minerva 62B (8-shot)
Accuracy
52.4
# 113
Arithmetic Reasoning
GSM8K
Minerva 62B (8-shot)
Parameters (Billion)
62
# 74
Arithmetic Reasoning
GSM8K
Minerva 540B (CoT)
Accuracy
78.5
# 66
Arithmetic Reasoning
GSM8K
Minerva 540B (CoT)
Parameters (Billion)
540
# 106
Arithmetic Reasoning
GSM8K
PaLM 540B (8-shot)
Accuracy
56.5
# 107
Arithmetic Reasoning
GSM8K
PaLM 540B (8-shot)
Parameters (Billion)
540
# 106
Arithmetic Reasoning
GSM8K
Minerva 8B (maj5@100)
Accuracy
56.8
# 105
Arithmetic Reasoning
GSM8K
Minerva 8B (maj5@100)
Parameters (Billion)
8
# 44
Arithmetic Reasoning
GSM8K
Minerva 62B (maj5@100)
Accuracy
89
# 23
Arithmetic Reasoning
GSM8K
Minerva 62B (maj5@100)
Parameters (Billion)
62
# 74
Arithmetic Reasoning
GSM8K
Minerva 62B (maj1@100)
Accuracy
68.5
# 91
Arithmetic Reasoning
GSM8K
Minerva 62B (maj1@100)
Parameters (Billion)
62
# 74
Arithmetic Reasoning
GSM8K
PaLM 62B (8-shot)
Accuracy
33.0
# 126
Arithmetic Reasoning
GSM8K
PaLM 62B (8-shot)
Parameters (Billion)
62
# 74
Arithmetic Reasoning
GSM8K
Minerva 8B (8-shot)
Accuracy
16.2
# 138
Arithmetic Reasoning
GSM8K
Minerva 8B (8-shot)
Parameters (Billion)
8
# 44
Arithmetic Reasoning
GSM8K
Minerva 8B-maj1@k (8-shot)
Accuracy
28.4
# 129
Arithmetic Reasoning
GSM8K
Minerva 8B-maj1@k (8-shot)
Parameters (Billion)
8
# 44
Arithmetic Reasoning
GSM8K
PaLM 8B (8-shot)
Accuracy
4.1
# 144
Arithmetic Reasoning
GSM8K
PaLM 8B (8-shot)
Parameters (Billion)
8
# 44
Math Word Problem Solving
MATH
davinci-002 175B
Accuracy
19.1
# 73
Math Word Problem Solving
MATH
davinci-002 175B
Parameters (Billions)
175
# 5
Math Word Problem Solving
MATH
Minerva 540B (maj1@k, k=64)
Accuracy
50.3
# 26
Math Word Problem Solving
MATH
Minerva 8B (maj1@k, k=64)
Accuracy
25.4
# 65
Math Word Problem Solving
MATH
Minerva 8B (maj1@k, k=64)
Parameters (Billions)
8
# 49
Math Word Problem Solving
MATH
Minerva 540B
Accuracy
33.6
# 51
Math Word Problem Solving
MATH
Minerva 540B
Parameters (Billions)
540
# 1
Math Word Problem Solving
MATH
Minerva 62B (maj1@k, k=64)
Accuracy
43.4
# 44
Math Word Problem Solving
MATH
Minerva 62B (maj1@k, k=64)
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
Minerva 62B (maj5@256)
Accuracy
64.9
# 5
Math Word Problem Solving
MATH
Minerva 62B (maj5@256)
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
Minerva 8B (maj5@256)
Accuracy
47.6
# 33
Math Word Problem Solving
MATH
Minerva 8B (maj5@256)
Parameters (Billions)
8
# 49
Math Word Problem Solving
MATH
PaLM 540B
Accuracy
8.8
# 85
Math Word Problem Solving
MATH
PaLM 540B
Parameters (Billions)
540
# 1
Math Word Problem Solving
MATH
PaLM 8B (fine-tuned)
Accuracy
5.6
# 93
Math Word Problem Solving
MATH
PaLM 8B (fine-tuned)
Parameters (Billions)
8
# 49
Math Word Problem Solving
MATH
Minerva 62B (4-shot)
Accuracy
27.6
# 62
Math Word Problem Solving
MATH
Minerva 62B (4-shot)
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
PaLM 8B
Accuracy
1.5
# 103
Math Word Problem Solving
MATH
PaLM 8B
Parameters (Billions)
8
# 49
Math Word Problem Solving
MATH
PaLM 62B
Accuracy
4.4
# 98
Math Word Problem Solving
MATH
PaLM 62B
Parameters (Billions)
62
# 22
Math Word Problem Solving
MATH
Minerva 8B
Accuracy
14.1
# 77
Math Word Problem Solving
MATH
Minerva 8B
Parameters (Billions)
8
# 49