Lexical Analysis

7 papers with code • 0 benchmarks • 1 datasets

Lexical analysis is the process of converting a sequence of characters into a sequence of tokens (strings with an assigned and thus identified meaning). (Source: Adapted from Wikipedia)


Most implemented papers

Chinese Lexical Analysis with Deep Bi-GRU-CRF Network

baidu/lac 5 Jul 2018

Lexical analysis is believed to be a crucial step towards natural language understanding and has been widely studied.

cs60075_team2 at SemEval-2021 Task 1 : Lexical Complexity Prediction using Transformer-based Language Models pre-trained on various text corpora

abhi1nandy2/CS60075-Team-2-Task-1 4 Jun 2021

This paper describes the performance of the team cs60075_team2 at SemEval 2021 Task 1 - Lexical Complexity Prediction.

Monitoring Term Drift Based on Semantic Consistency in an Evolving Vector Field

peterwittek/concept_drifts 5 Feb 2015

Based on the Aristotelian concept of potentiality vs. actuality allowing for the study of energy and dynamics in language, we propose a field approach to lexical analysis.

Rethinking Text Attribute Transfer: A Lexical Analysis

FranxYao/pivot_analysis WS 2019

We apply this framework to existing datasets and models and show that: (1) the pivot words are strong features for the classification of sentence attributes; (2) to change the attribute of a sentence, many datasets only requires to change certain pivot words; (3) consequently, many transfer models only perform the lexical-level modification, while leaving higher-level sentence structures unchanged.

N-LTP: An Open-source Neural Language Technology Platform for Chinese


We introduce \texttt{N-LTP}, an open-source neural language technology platform supporting six fundamental Chinese NLP tasks: {lexical analysis} (Chinese word segmentation, part-of-speech tagging, and named entity recognition), {syntactic parsing} (dependency parsing), and {semantic parsing} (semantic dependency parsing and semantic role labeling).

American cultural regions mapped through the lexical analysis of social media

TLouf/words-use 16 Aug 2022

Through a hierarchical clustering of the data in this lower-dimensional space, this method yields clear cultural areas and the topics of discussion that define them.