Search Results for author:
Found 10 papers, 4 papers with code
“Is Whole Word Masking Always Better for Chinese BERT?”: Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
One Model, Multiple Modalities: A Sparsely Activated Approach for Text, Sound, Image, Video and Code
Moreover, our model supports self-supervised pretraining with the same sparsely activated way, resulting in better initialized parameters for different modalities.
Pretraining Chinese BERT for Detecting Word Insertion and Deletion Errors
We achieve this by introducing a special token \texttt{[null]}, the prediction of which stands for the non-existence of a word.
MarkBERT: Marking Word Boundaries Improves Chinese BERT
MarkBERT pushes the state-of-the-art of Chinese named entity recognition from 95. 4\% to 96. 5\% on the MSRA dataset and from 82. 8\% to 84. 2\% on the OntoNotes dataset, respectively.
Chinese Named Entity Recognition
named-entity-recognition
+5
Exploring and Adapting Chinese GPT to Pinyin Input Method
We find that a frozen GPT achieves state-of-the-art performance on perfect pinyin.
"Is Whole Word Masking Always Better for Chinese BERT?": Probing on Chinese Grammatical Error Correction
The meaning of a word in Chinese is different in that a word is a compositional unit consisting of multiple characters.
Pretraining without Wordpieces: Learning Over a Vocabulary of Millions of Words
(2) how to predict a word via cloze test without knowing the number of wordpieces in advance?
GraphCodeBERT: Pre-training Code Representations with Data Flow
Instead of taking syntactic-level structure of code like abstract syntax tree (AST), we use data flow in the pre-training stage, which is a semantic-level structure of code that encodes the relation of "where-the-value-comes-from" between variables.
Ranked #1 on
Type prediction
on ManyTypes4TypeScript
LogicalFactChecker: Leveraging Logical Operations for Fact Checking with Graph Module Network
The graph is used to obtain graph-enhanced contextual representations of words in Transformer-based architecture.
CodeBERT: A Pre-Trained Model for Programming and Natural Languages
Results show that CodeBERT achieves state-of-the-art performance on both natural language code search and code documentation generation tasks.
Ranked #1 on
Code Documentation Generation
on CodeSearchNet - Go
