Multiple Choice Question Answering (MCQA)

24 papers with code • 28 benchmarks • 7 datasets

A multiple-choice question (MCQ) is composed of two parts: a stem that identifies the question or problem, and a set of alternatives or possible answers that contain a key that is the best answer to the question, and a number of distractors that are plausible but incorrect answers to the question.

In a k-way MCQA task, a model is provided with a question q, a set of candidate options O = {O1, . . . , Ok}, and a supporting context for each option C = {C1, . . . , Ck}. The model needs to predict the correct answer option that is best supported by the given contexts.

Most implemented papers

Llama 2: Open Foundation and Fine-Tuned Chat Models

facebookresearch/llama 18 Jul 2023

In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters.

PaLM: Scaling Language Modeling with Pathways

lucidrains/CoCa-pytorch Google Research 2022

To further our understanding of the impact of scale on few-shot learning, we trained a 540-billion parameter, densely activated, Transformer language model, which we call Pathways Language Model PaLM.

From Recognition to Cognition: Visual Commonsense Reasoning

rowanz/r2c CVPR 2019

While this task is easy for humans, it is tremendously difficult for today's vision systems, requiring higher-order cognition and commonsense reasoning about the world.

Learning to Attend On Essential Terms: An Enhanced Retriever-Reader Model for Open-domain Question Answering

nijianmo/arc-etrr-code NAACL 2019

In this paper we propose a retriever-reader model that learns to attend on essential terms during the question answering process.

MMM: Multi-stage Multi-task Learning for Multi-choice Reading Comprehension

jind11/MMM-MCQA 1 Oct 2019

Machine Reading Comprehension (MRC) for question answering (QA), which aims to answer a question given the relevant context passages, is an important way to test the ability of intelligence systems to understand human language.

QuALITY: Question Answering with Long Input Texts, Yes!

nyu-mll/quality NAACL 2022

To enable building and testing models on long-document comprehension, we introduce QuALITY, a multiple-choice QA dataset with context passages in English that have an average length of about 5, 000 tokens, much longer than typical current models can process.

Variational Open-Domain Question Answering

VodLM/vod 23 Sep 2022

Retrieval-augmented models have proven to be effective in natural language processing tasks, yet there remains a lack of research on their optimization using variational inference.

MEDITRON-70B: Scaling Medical Pretraining for Large Language Models

epfllm/meditron 27 Nov 2023

Large language models (LLMs) can potentially democratize access to medical knowledge.

Counterfactual Variable Control for Robust and Interpretable Question Answering

PluviophileYU/CVC-QA 12 Oct 2020

We then conduct two novel CVC inference methods (on trained models) to capture the effect of comprehensive reasoning as the final prediction.

IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages

AI4Bharat/indic-bert Findings of the Association for Computational Linguistics 2020

These resources include: (a) large-scale sentence-level monolingual corpora, (b) pre-trained word embeddings, (c) pre-trained language models, and (d) multiple NLU evaluation datasets (IndicGLUE benchmark).