The MNumGLUESub dataset is a multi-task arithmetic reasoning benchmark. It is part of the NumGLUE collection, which focuses on evaluating mathematical reasoning abilities in various languages. Let's break down the details:

1. NumGLUE: This broader benchmark suite includes several tasks related to mathematical reasoning. One of its subsets is MNumGLUESub.

2. MNumGLUESub:

   • Tasks Selection: To create this dataset, specific tasks from the larger NumGLUE collection were chosen. Specifically, tasks 1, 4, and 8 were selected.
   • Mathematical World Problems: These tasks involve solving mathematical world problems. The questions in MNumGLUESub cover various mathematical concepts and require reasoning skills.
   • Translation: The questions from MNumGLUESub were translated into nine different languages using ChatGPT. These languages align with the languages used in the broader GSM8KInstruct dataset.

3. Purpose: MNumGLUESub serves as an evaluation dataset for assessing the mathematical reasoning capabilities of models across different languages. It helps researchers analyze how well models perform on arithmetic reasoning tasks in multilingual settings.

For further details, you can refer to the research paper on Advancing Multilingual Reasoning through Multilingual Alignment-as-Preference Optimization (MAPO), which introduces the MAPO framework for enhancing multilingual reasoning abilities ¹².

Source: Conversation with Bing, 3/17/2024 (1) MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment .... https://arxiv.org/html/2401.06838v2. (2) MAPO: Advancing Multilingual Reasoning through Multilingual Alignment .... https://arxiv.org/html/2401.06838v1. (3) [2401.06838] MAPO: Advancing Multilingual Reasoning through .... https://arxiv.org/abs/2401.06838.