Galactica: A Large Language Model for Science

Browse State-of-the-Art
Datasets
Methods
More
Newsletter RC2022
About Trends Portals Libraries

Sign In

Subscribe to the PwC Newsletter

Stay informed on the latest trending ML papers with code, research developments, libraries, methods, and datasets.

Read previous issues

Join the community

You need to log in to edit.
You can create a new account if you don't have one.

TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Knowledge Probing	AminoProbe	GAL 120B (zero-shot)	Accuracy	21	# 1
Knowledge Probing	AminoProbe	GAL 125M (zero-shot)	Accuracy	12	# 7
Knowledge Probing	AminoProbe	GAL 1.3B (zero-shot)	Accuracy	16	# 4
Knowledge Probing	AminoProbe	GPT-3 (text-davinci-002) (zero-shot)	Accuracy	14	# 5
Knowledge Probing	AminoProbe	BLOOM (zero-shot)	Accuracy	14	# 5
Knowledge Probing	AminoProbe	GAL 30B (zero-shot)	Accuracy	21	# 1
Knowledge Probing	AminoProbe	OPT (zero-shot)	Accuracy	12	# 7
Knowledge Probing	AminoProbe	GAL 6.7B (zero-shot)	Accuracy	17	# 3
Common Sense Reasoning	ARC (Challenge)	GPT-3 (zero-shot)	Accuracy	51.4	# 31
Common Sense Reasoning	ARC (Challenge)	GAL 120B (zero-shot)	Accuracy	67.9	# 18
Common Sense Reasoning	ARC (Challenge)	BLOOM (few-shot, k=5)	Accuracy	32.9	# 48
Common Sense Reasoning	ARC (Challenge)	OPT (few-shot, k=5)	Accuracy	31.1	# 50
Common Sense Reasoning	ARC (Easy)	BLOOM (5-shot)	Accuracy	40.7	# 43
Common Sense Reasoning	ARC (Easy)	GPT-3 (zero-shot)	Accuracy	68.8	# 37
Common Sense Reasoning	ARC (Easy)	GAL 120B (0-shot)	Accuracy	83.8	# 9
Common Sense Reasoning	ARC (Easy)	OPT (5-shot)	Accuracy	37.4	# 45
Molecular Property Prediction	BACE	GAL 1.3B	ROC-AUC	57.6	# 17
Molecular Property Prediction	BACE	GAL 120B	ROC-AUC	61.7	# 15
Molecular Property Prediction	BACE	GAL 6.7B	ROC-AUC	58.4	# 16
Molecular Property Prediction	BACE	GAL 30B	ROC-AUC	72.7	# 14
Molecular Property Prediction	BACE	GAL 125M	ROC-AUC	56.1	# 18
Molecular Property Prediction	BBBP	GAL 125M	ROC-AUC	39.3	# 27
Molecular Property Prediction	BBBP	GAL 30B	ROC-AUC	59.6	# 25
Molecular Property Prediction	BBBP	GAL 6.7B	ROC-AUC	53.5	# 26
Molecular Property Prediction	BBBP	GAL 1.3B	ROC-AUC	60.4	# 24
Molecular Property Prediction	BBBP	Uni-Mol	ROC-AUC	72.9	# 13
Molecular Property Prediction	BBBP	GAL 120B	ROC-AUC	66.1	# 23
Word Sense Disambiguation	BIG-bench (Anachronisms)	OPT 175B	Accuracy	49.1	# 3
Word Sense Disambiguation	BIG-bench (Anachronisms)	GAL 120B (few-shot, k=5)	Accuracy	48.7	# 4
Word Sense Disambiguation	BIG-bench (Anachronisms)	GAL 30B (few-shot, k=5)	Accuracy	47.0	# 5
Word Sense Disambiguation	BIG-bench (Anachronisms)	BLOOM 176B	Accuracy	1.3	# 6
Question Answering	BioASQ	OPT (zero-shot)	Accuracy	81.4	# 7
Question Answering	BioASQ	BLOOM (zero-shot)	Accuracy	91.4	# 3
Question Answering	BioASQ	GAL 120B (zero-shot)	Accuracy	94.3	# 2
Knowledge Probing	BioLAMA	BLOOM (zero-shot)	Accuracy	9.7	# 1
Knowledge Probing	BioLAMA	GAL 6.7B	Accuracy	7.9	# 4
Knowledge Probing	BioLAMA	GAL 30B	Accuracy	6.9	# 7
Knowledge Probing	BioLAMA	OPT (zero-shot)	Accuracy	7.1	# 6
Knowledge Probing	BioLAMA	GAL 120B	Accuracy	8.0	# 3
Knowledge Probing	BioLAMA	GAL 1.3B	Accuracy	7.2	# 5

Edit Social Preview