Search Results for author:
Found 21 papers, 11 papers with code
A Study on Manual and Automatic Evaluation for Text Style Transfer: The Case of Detoxification
It is often difficult to reliably evaluate models which generate text.
Attention Understands Semantic Relations
Today, natural language processing heavily relies on pre-trained large language models.
A Family of Pretrained Transformer Language Models for Russian
Transformer language models (LMs) are fundamental to NLP research methodologies and applications in various languages.
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions.
Universal and Independent: Multilingual Probing Framework for Exhaustive Model Interpretation and Evaluation
Thus we propose a toolkit to systematize the multilingual flaws in multilingual models, providing a reproducible experimental setup for 104 languages and 80 morphosyntactic features.
TAPE: Assessing Few-shot Russian Language Understanding
Recent advances in zero-shot and few-shot learning have shown promise for a scope of research and practical purposes.
Ranked #1 on
Ethics
on Ethics (per ethics)
Vote'n'Rank: Revision of Benchmarking with Social Choice Theory
The development of state-of-the-art systems in different applied areas of machine learning (ML) is driven by benchmarks, which have shaped the paradigm of evaluating generalisation capabilities from multiple perspectives.
Is neural language acquisition similar to natural? A chronological probing study
The probing methodology allows one to obtain a partial representation of linguistic phenomena stored in the inner layers of the neural network, using external classifiers and statistical analysis.
Findings of the The RuATD Shared Task 2022 on Artificial Text Detection in Russian
The first task is framed as a binary classification problem.
WikiOmnia: generative QA corpus on the whole Russian Wikipedia
The General QA field has been developing the methodology referencing the Stanford Question answering dataset (SQuAD) as the significant benchmark.
mGPT: Few-Shot Learners Go Multilingual
Recent studies report that autoregressive language models can successfully solve many NLP tasks via zero- and few-shot learning paradigms, which opens up new possibilities for using the pre-trained language models.
Ranked #1 on
Natural Language Inference
on XWINO
Cross-Lingual Natural Language Inference
Cross-Lingual Paraphrase Identification
+5
Russian SuperGLUE 1.1: Revising the Lessons not Learned by Russian NLP models
In the last year, new neural architectures and multilingual pre-trained models have been released for Russian, which led to performance evaluation problems across a range of language understanding tasks.
How not to Lie with a Benchmark: Rearranging NLP Learderboards
Proper model ranking and comparison with a human level is an essential requirement for every benchmark to be a reliable measurement of the model quality.
MOROCCO: Model Resource Comparison Framework
The new generation of pre-trained NLP models push the SOTA to the new limits, but at the cost of computational resources, to the point that their use in real production environments is often prohibitively expensive.
Domain-Transferable Method for Named Entity Recognition Task
Named Entity Recognition (NER) is a fundamental task in the fields of natural language processing and information extraction.
RussianSuperGLUE: A Russian Language Understanding Evaluation Benchmark
In this paper, we introduce an advanced Russian general language understanding evaluation benchmark -- RussianGLUE.
Ranked #1 on
Word Sense Disambiguation
on RUSSE
DaNetQA: a yes/no Question Answering Dataset for the Russian Language
The task is to take both the question and a paragraph as input and come up with a yes/no answer, i. e. to produce a binary output.
Humans Keep It One Hundred: an Overview of AI Journey
Artificial General Intelligence (AGI) is showing growing performance in numerous applications - beating human performance in Chess and Go, using knowledge bases and text sources to answer questions (SQuAD) and even pass human examination (Aristo project).
AGRR 2019: Corpus for Gapping Resolution in Russian
This paper provides a comprehensive overview of the gapping dataset for Russian that consists of 7. 5k sentences with gapping (as well as 15k relevant negative sentences) and comprises data from various genres: news, fiction, social media and technical texts.
AGRR-2019: A Corpus for Gapping Resolution in Russian
This paper provides a comprehensive overview of the gapping dataset for Russian that consists of 7. 5k sentences with gapping (as well as 15k relevant negative sentences) and comprises data from various genres: news, fiction, social media and technical texts.
To the methodology of corpus construction for machine learning:“Taiga” syntax tree corpus and parser
The “Taiga” project unites the corpus and the syntactic parser, being created in a new field of the corpus linguistics: the material obtained primarily meets the needs of machine learning, rather than linguistic search.
