Browse > Natural Language Processing > Lexical Normalization

# Lexical Normalization Edit

3 papers with code · Natural Language Processing

Lexical normalization is the task of translating/transforming a non standard text to a standard register.

Example:

new pix comming tomoroe
new pictures coming tomorrow


Datasets usually consists of tweets, since these naturally contain a fair amount of these phenomena.

For lexical normalization, only replacements on the word-level are annotated. Some corpora include annotation for 1-N and N-1 replacements. However, word insertion/deletion and reordering is not part of the task.

# Leaderboards Add a Result

TREND DATASET BEST METHOD PAPER TITLE PAPER CODE COMPARE

# Adapting Sequence to Sequence models for Text Normalization in Social Media

12 Apr 2019Isminoula/TextNormSeq2Seq

Social media offer an abundant source of valuable raw data, however informal writing can quickly become a bottleneck for many natural language processing (NLP) tasks.

17

# MoNoise: Modeling Noise Using a Modular Normalization System

10 Oct 2017wesselreijngoud/masterthesis2019

We show that MoNoise beats the state-of-the-art on different normalization benchmarks for English and Dutch, which all define the task of normalization slightly different.

1

# A Multi-cascaded Deep Model for Bilingual SMS Classification

29 Nov 2019haroonshakeel/bilingual_sms_classification

Our model achieves high accuracy for classification on this dataset and outperforms the previous model for multilingual text classification, highlighting language independence of McM.

0