Search Results for author:
Found 7 papers, 5 papers with code
MERA: A Comprehensive LLM Evaluation in Russian
To address these issues, we introduce an open Multimodal Evaluation of Russian-language Architectures (MERA), a new instruction benchmark for evaluating foundation models oriented towards the Russian language.
A Family of Pretrained Transformer Language Models for Russian
Transformer language models (LMs) are fundamental to NLP research methodologies and applications in various languages.
TAPE: Assessing Few-shot Russian Language Understanding
Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes.
Ranked #1 on
Ethics
on Ethics (per ethics)
mGPT: Few-Shot Learners Go Multilingual
Recent studies report that autoregressive language models can successfully solve many NLP tasks via zero- and few-shot learning paradigms, which opens up new possibilities for using the pre-trained language models.
Ranked #1 on
Natural Language Inference
on XWINO
Cross-Lingual Natural Language Inference
Cross-Lingual Paraphrase Identification
+5
Russian SuperGLUE 1.1: Revising the Lessons not Learned by Russian NLP models
In the last year, new neural architectures and multilingual pre-trained models have been released for Russian, which led to performance evaluation problems across a range of language understanding tasks.
MOROCCO: Model Resource Comparison Framework
The new generation of pre-trained NLP models push the SOTA to the new limits, but at the cost of computational resources, to the point that their use in real production environments is often prohibitively expensive.
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark
In this paper, we introduce an advanced Russian general language understanding evaluation benchmark -- RussianGLUE.
Ranked #1 on
Word Sense Disambiguation
on RUSSE
