Browse > Natural Language Processing > Constituency Parsing

Constituency Parsing

15 papers with code · Natural Language Processing

Consituency parsing aims to extract a constituency-based parse tree from a sentence that represents its syntactic structure according to a phrase structure grammar.

Example:

             Sentence (S)
                 |
   +-------------+------------+
   |                          |
 Noun (N)                Verb Phrase (VP)
   |                          |
 John                 +-------+--------+
                      |                |
                    Verb (V)         Noun (N)
                      |                |
                    sees              Bill

Recent approaches convert the parse tree into a sequence following a depth-first traversal in order to be able to apply sequence-to-sequence models to it. The linearized version of the above parse tree looks as follows: (S (N) (VP V N)).

State-of-the-art leaderboards

Greatest papers with code

Attention Is All You Need

NeurIPS 2017 facebookresearch/fairseq-py

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism.

CONSTITUENCY PARSING MACHINE TRANSLATION

Grammar as a Foreign Language

NeurIPS 2015 atpaino/deep-text-corrector

Syntactic constituency parsing is a fundamental problem in natural language processing and has been the subject of intensive research and engineering for decades. As a result, the most accurate parsers are domain specific, complex, and inefficient.

CONSTITUENCY PARSING

YellowFin and the Art of Momentum Tuning

ICLR 2018 JianGoForIt/YellowFin

Motivated by this trend, we ask: can simple adaptive methods based on SGD perform as well or better? We revisit the momentum SGD algorithm and show that hand-tuning a single learning rate and momentum makes it competitive with Adam.

CONSTITUENCY PARSING LANGUAGE MODELLING

Multilingual Constituency Parsing with Self-Attention and Pre-Training

31 Dec 2018nikitakit/self-attentive-parser

We extend our previous work on constituency parsing (Kitaev and Klein, 2018) by incorporating pre-training for ten additional languages, and compare the benefits of no pre-training, ELMo (Peters et al., 2018), and BERT (Devlin et al., 2018). Pre-training is effective across all languages evaluated, and BERT outperforms ELMo in large part due to the benefits of increased model capacity.

CONSTITUENCY PARSING

Constituency Parsing with a Self-Attentive Encoder

ACL 2018 nikitakit/self-attentive-parser

We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser. The use of attention makes explicit the manner in which information is propagated between different locations in the sentence, which we use to both analyze our model and propose potential improvements.

CONSTITUENCY PARSING

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

NeurIPS 2015 Chung-I/Variational-Recurrent-Autoencoder-Tensorflow

Recurrent Neural Networks can be trained to produce sequences of tokens given some input, as exemplified by recent results in machine translation and image captioning. The current approach to training them consists of maximizing the likelihood of each token in the sequence given the current (recurrent) state and the previous token.

CONSTITUENCY PARSING IMAGE CAPTIONING SPEECH RECOGNITION

Recurrent Neural Network Grammars

HLT 2016 clab/rnng

We introduce recurrent neural network grammars, probabilistic models of sentences with explicit phrase structure. We explain efficient inference procedures that allow application to both parsing and language modeling.

CONSTITUENCY PARSING LANGUAGE MODELLING

Span-Based Constituency Parsing with a Structure-Label System and Provably Optimal Dynamic Oracles

EMNLP 2016 jhcross/span-parser

Parsing accuracy using efficient greedy transition systems has improved dramatically in recent years thanks to neural networks. Despite striking results in dependency parsing, however, neural models have not surpassed state-of-the-art approaches in constituency parsing.

CONSTITUENCY PARSING DEPENDENCY PARSING

Grammar Induction with Neural Language Models: An Unusual Replication

WS 2018 nyu-mll/PRPN-Analysis

A substantial thread of recent work on latent tree learning has attempted to develop neural network models with parse-valued latent variables and train them on non-parsing tasks, in the hope of having them discover interpretable tree structure. In a recent paper, Shen et al. (2018) introduce such a model and report near-state-of-the-art results on the target task of language modeling, and the first strong latent tree learning result on constituency parsing.

CONSTITUENCY PARSING LANGUAGE MODELLING