Search Results for author:
Found 5 papers, 3 papers with code
MERA: A Comprehensive LLM Evaluation in Russian
To address these issues, we introduce an open Multimodal Evaluation of Russian-language Architectures (MERA), a new instruction benchmark for evaluating foundation models oriented towards the Russian language.
TAPE: Assessing Few-shot Russian Language Understanding
Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes.
Ranked #1 on
Ethics
on Ethics (per ethics)
Russian SuperGLUE 1.1: Revising the Lessons not Learned by Russian NLP models
In the last year, new neural architectures and multilingual pre-trained models have been released for Russian, which led to performance evaluation problems across a range of language understanding tasks.
Read and Reason with MuSeRC and RuCoS: Datasets for Machine Reading Comprehension for Russian
The paper introduces two Russian machine reading comprehension (MRC) datasets, called MuSeRC and RuCoS, which require reasoning over multiple sentences and commonsense knowledge to infer the answer.
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark
In this paper, we introduce an advanced Russian general language understanding evaluation benchmark -- RussianGLUE.
Ranked #1 on
Word Sense Disambiguation
on RUSSE
