Search Results for author:
Found 97 papers, 56 papers with code
Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks
On the other hand, symbolic rules such as regular expressions are interpretable, require no training, and often achieve decent accuracy; but rules cannot benefit from labeled data when available and hence underperform neural networks in rich-resource scenarios.
Improving Constituent Representation with Hypertree Neural Networks
Many natural language processing tasks involve text spans and thus high-quality span representations are needed to enhance neural approaches to these tasks.
SHARP: Search-Based Adversarial Attack for Structured Prediction
Adversarial attack of structured prediction models faces various challenges such as the difficulty of perturbing discrete words, the sentence quality issue, and the sensitivity of outputs to small perturbations.
Neuralizing Regular Expressions for Slot Filling
Neural models and symbolic rules such as regular expressions have their respective merits and weaknesses.
A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference
Recently, sharing key-value (KV) cache across layers has been found effective in efficient inference of large language models (LLMs).
Efficient Long-range Language Modeling with Self-supervised Causal Retrieval
For a given input sequence, we split it into chunks and use the current chunk to retrieve past chunks for subsequent text generation.
Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation
Named entity recognition (NER) models often struggle with noisy inputs, such as those with spelling mistakes or errors generated by Optical Character Recognition processes, and learning a robust NER model is challenging.
Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models
Syntactic Transformer language models aim to achieve better generalization through simultaneously modeling syntax trees and sentences.
Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers
Accommodating long sequences efficiently in autoregressive Transformers, especially within an extended context window, poses significant challenges due to the quadratic computational complexity and substantial KV memory requirements inherent in self-attention mechanisms.
Unsupervised Morphological Tree Tokenizer
Specifically, the deep model jointly encodes internal structures and representations of words with a mechanism named $\textit{MorphOverriding}$ to ensure the indecomposability of morphemes.
Layer-Condensed KV Cache for Efficient Inference of Large Language Models
In this paper, we propose a novel method that only computes and caches the KVs of a small number of layers, thus significantly saving memory consumption and improving inference throughput.
Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRL
Large Language Models (LLMs) play a crucial role in capturing structured semantics to enhance language understanding, improve interpretability, and reduce bias.
RoT: Enhancing Large Language Models with Reflection on Search Trees
It uses a strong LLM to summarize guidelines from previous tree search experiences to enhance the ability of a weak LLM.
Using Interpretation Methods for Model Enhancement
In this paper, we propose a framework of utilizing interpretation methods and gold rationales to enhance models.
Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts
With our method, the origin language models can cover several times longer contexts while keeping the computing requirements close to the baseline.
Generative Pretrained Structured Transformers: Unsupervised Syntactic Language Models at Scale
A syntactic language model (SLM) incrementally generates a sentence with its syntactic tree in a left-to-right manner.
Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation
Specifically, we design a conditional random field that models discrete latent representations of all words in a sentence as well as dependency arcs between them; and we use mean field variational inference for approximate inference.
Conic10K: A Challenging Math Problem Understanding and Reasoning Dataset
Since the dataset only involves a narrow range of knowledge, it is easy to separately analyse the knowledge a model possesses and the reasoning ability it has.
Joint Entity and Relation Extraction with Span Pruning and Hypergraph Neural Networks
Also, most of current ERE models do not take into account higher-order interactions between multiple entities and relations, while higher-order modeling could be beneficial. In this work, we propose HyperGraph neural network for ERE ($\hgnn{}$), which is built upon the PL-marker (a state-of-the-art marker-based pipleline model).
Simple Hardware-Efficient PCFGs with Independent Left and Right Productions
Scaling dense PCFGs to thousands of nonterminals via a low-rank parameterization of the rule probability tensor has been shown to be beneficial for unsupervised parsing.
AMR Parsing with Causal Hierarchical Attention and Pointers
Translation-based AMR parsers have recently gained popularity due to their simplicity and effectiveness.
Augmenting Transformers with Recursively Composed Multi-grained Representations
More interestingly, the hierarchical structures induced by ReCAT exhibit strong consistency with human-annotated syntactic trees, indicating good interpretability brought by the CIO layers.
Ranked #4 on
Semantic Role Labeling
on OntoNotes
Constituency Grammar Induction
Natural Language Inference
+2
Do PLMs Know and Understand Ontological Knowledge?
In this paper, we focus on probing whether PLMs store ontological knowledge and have a semantic understanding of the knowledge rather than rote memorization of the surface form.
SeqGPT: An Out-of-the-box Large Language Model for Open Domain Sequence Understanding
However, LLMs are sometimes too footloose for natural language understanding (NLU) tasks which always have restricted output and input format.
Improving Text Matching in E-Commerce Search with A Rationalizable, Intervenable and Fast Entity-Based Relevance Model
We also show that pretraining the QE module with auto-generated QE data from user logs can further improve the overall performance.
Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints
Neural QCFG is a grammar-based sequence-tosequence (seq2seq) model with strong inductive biases on hierarchical structures.
DAMO-NLP at SemEval-2023 Task 2: A Unified Retrieval-augmented System for Multilingual Named Entity Recognition
Also, we discover that the limited context length causes the retrieval knowledge to be invisible to the model.
Multilingual Named Entity Recognition
named-entity-recognition
+4
A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification
As the structured language model learns to predict constituency trees in a self-supervised manner, only raw texts and sentence-level labels are required as training data, which makes it essentially a general constituent-level self-interpretable classification model.
COMBO: A Complete Benchmark for Open KG Canonicalization
The subject and object noun phrases and the relation in open KG have severe redundancy and ambiguity and need to be canonicalized.
Recall, Expand and Multi-Candidate Cross-Encode: Fast and Accurate Ultra-Fine Entity Typing
We also found MCCE is very effective in fine-grained (130 types) and coarse-grained (9 types) entity typing.
Ranked #2 on
Entity Typing
on Open Entity
Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field
Prior works on joint Information Extraction (IE) typically model instance (e. g., event triggers, entities, roles, relations) interactions by representation enhancement, type dependencies scoring, or global decoding.
Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field
We use mean-field variational inference for efficient type inference on very large type sets and unfold it as a neural network module to enable end-to-end training.
Ranked #3 on
Entity Typing
on Open Entity
Named Entity and Relation Extraction with Multi-Modal Retrieval
MoRe contains a text retrieval module and an image-based retrieval module, which retrieve related knowledge of the input text and image in the knowledge corpus respectively.
Ranked #1 on
Multi-modal Named Entity Recognition
on SNAP (MNER)
Multi-modal Named Entity Recognition
Named Entity Recognition
+3
Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs
Recent research found it beneficial to use large state spaces for HMMs and PCFGs.
Modeling Label Correlations for Second-Order Semantic Dependency Parsing with Mean-Field Inference
Second-order semantic parsing with end-to-end mean-field inference has been shown good performance.
Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing
They treat nested entities as partially-observed constituency trees and propose the masked inside algorithm for partial marginalization.
DAMO-NLP at SemEval-2022 Task 11: A Knowledge-based System for Multilingual Named Entity Recognition
Our system wins 10 out of 13 tracks in the MultiCoNER shared task.
Multilingual Named Entity Recognition
Named Entity Recognition
+1
ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition
As text representations take the most important role in MNER, in this paper, we propose {\bf I}mage-{\bf t}ext {\bf A}lignments (ITA) to align image features into the textual space, so that the attention mechanism in transformer-based pretrained textual embeddings can be better utilized.
Ranked #1 on
Multi-modal Named Entity Recognition
on Twitter-17
Multi-modal Named Entity Recognition
named-entity-recognition
+1
Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks
Constituency parsing and nested named entity recognition (NER) are similar tasks since they both aim to predict a collection of nested and non-crossing spans.
Combining (second-order) graph-based and headed-span-based projective dependency parsing
Graph-based methods, which decompose the score of a dependency tree into scores of dependency arcs, are popular in dependency parsing for decades.
Headed-Span-Based Projective Dependency Parsing
In a projective dependency tree, the largest subtree rooted at each word covers a contiguous sequence (i. e., a span) in the surface order.
Adapting Unsupervised Syntactic Parsing Methodology for Discourse Dependency Parsing
In this paper, we propose a simple yet effective method to adapt unsupervised syntactic dependency parsing methodology for unsupervised discourse dependency parsing.
Risk Minimization for Zero-shot Sequence Labeling
In this paper, we propose a novel unified framework for zero-shot sequence labeling with minimum risk training and design a new decomposable risk function that models the relations between the predicted labels from the source models and the true labels.
Multi-View Cross-Lingual Structured Prediction with Minimum Supervision
In structured prediction problems, cross-lingual transfer learning is an efficient way to train quality models for low-resource languages, and further improvement can be obtained by learning from multiple source languages.
Enhanced Universal Dependency Parsing with Automated Concatenation of Embeddings
This paper describes the system used in submission from SHANGHAITECH team to the IWPT 2021 Shared Task.
Neural Bi-Lexicalized PCFG Induction
Neural lexicalized PCFGs (L-PCFGs) have been shown effective in grammar induction.
Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning
We find empirically that the contextual representations computed on the retrieval-based input view, constructed through the concatenation of a sentence and its external contexts, can achieve significantly improved performance compared to the original input view based only on the sentence.
Ranked #1 on
Named Entity Recognition (NER)
on WNUT 2017
PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols
In this work, we present a new parameterization form of PCFGs based on tensor decomposition, which has at most quadratic computational complexity in the symbol number and therefore allows us to use a much larger number of symbols.
Unsupervised Natural Language Parsing (Introductory Tutorial)
Unsupervised parsing learns a syntactic parser from training sentences without parse tree annotations.
Constrained Text Generation with Global Guidance -- Case Study on CommonGen
We focus on CommonGen, the task of generating text based on a set of concepts, as a representative task of constrained text generation.
Semi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding
By relaxing the tree constraint in both the encoder and the decoder during training, we make the learning of our model fully arc-factored and thus circumvent the challenges brought by the tree constraint.
Deep Inside-outside Recursive Autoencoder with All-span Objective
Deep inside-outside recursive autoencoder (DIORA) is a neural-based model designed for unsupervised constituency parsing.
An Investigation of Potential Function Designs for Neural CRF
The neural linear-chain CRF model is one of the most widely-used approach to sequence labeling.
Neural Latent Dependency Model for Sequence Labeling
One limitation of linear chain CRFs is their inability to model long-range dependencies between labels.
Learning Numeral Embedding
Numeral embeddings represented in this manner can be plugged into existing word embedding learning approaches such as skip-gram for training.
Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders
We consider the task of cross-lingual adaptation of dependency parsers without annotated target corpora and parallel corpora.
Second-Order Unsupervised Neural Dependency Parsing
Inspired by second-order supervised dependency parsing, we proposed a second-order extension of unsupervised neural dependency models that incorporate grandparent-child or sibling information.
Ranked #1 on
Dependency Grammar Induction
on WSJ10
Structural Knowledge Distillation: Tractably Distilling Information for Structured Predictor
The objective function of knowledge distillation is typically the cross-entropy between the teacher and the student's output distributions.
Second-Order Neural Dependency Parsing with Message Passing and End-to-End Training
In this paper, we propose second-order graph-based neural dependency parsing using message passing and end-to-end neural networks.
Ranked #1 on
Dependency Parsing
on Chinese Treebank
Automated Concatenation of Embeddings for Structured Prediction
Pretrained contextualized embeddings are powerful word representations for structured prediction tasks.
Ranked #1 on
Chunking
on Penn Treebank
Adversarial Attack and Defense of Structured Prediction Models
To address these problems, we propose a novel and unified framework that learns to attack a structured prediction model using a sequence-to-sequence model with feedbacks from multiple reference models of the same structured prediction task.
A Survey of Unsupervised Dependency Parsing
Syntactic dependency parsing is an important task in natural language processing.
AIN: Fast and Accurate Sequence Labeling with Approximate Inference Network
The linear-chain Conditional Random Field (CRF) model is one of the most widely-used neural sequence labeling approaches.
Ranked #3 on
Chunking
on CoNLL 2003 (German)
More Embeddings, Better Sequence Labelers?
Recent work proposes a family of contextual embeddings that significantly improves the accuracy of sequence labelers over non-contextual embeddings.
Ranked #2 on
Chunking
on CoNLL 2003 (German)
An Empirical Comparison of Unsupervised Constituency Parsing Methods
Unsupervised constituency parsing aims to learn a constituency parser from a training corpus without parse tree annotations.
Towards Holistic and Automatic Evaluation of Open-Domain Dialogue Generation
Open-domain dialogue generation has gained increasing attention in Natural Language Processing.
Enhanced Universal Dependency Parsing with Second-Order Inference and Mixture of Training Data
This paper presents the system used in our submission to the \textit{IWPT 2020 Shared Task}.
ShanghaiTech at MRP 2019: Sequence-to-Graph Transduction with Second-Order Edge Inference for Cross-Framework Meaning Representation Parsing
This paper presents the system used in our submission to the \textit{CoNLL 2019 shared task: Cross-Framework Meaning Representation Parsing}.
Structure-Level Knowledge Distillation For Multilingual Sequence Labeling
Multilingual sequence labeling is a task of predicting label sequences using a single unified model for multiple languages.
Learning Numeral Embeddings
Numeral embeddings represented in this manner can be plugged into existing word embedding learning approaches such as skip-gram for training.
A Regularization-based Framework for Bilingual Grammar Induction
Grammar induction aims to discover syntactic structures from unannotated sentences.
Multilingual Grammar Induction with Continuous Language Identification
The key to multilingual grammar induction is to couple grammar parameters of different languages together by exploiting the similarity between languages.
Bidirectional Transition-Based Dependency Parsing
Traditionally, a transitionbased dependency parser processes an input sentence and predicts a sequence of parsing actions in a left-to-right manner.
Enhancing Unsupervised Generative Dependency Parser with Contextual Information
In this paper, we propose a novel probabilistic model called discriminative neural dependency model with valence (D-NDMV) that generates a sentence and its parse from a continuous latent representation, which encodes global contextual information of the generated sentence.
Ranked #2 on
Dependency Grammar Induction
on WSJ10
Constituency Grammar Induction
Dependency Grammar Induction
+2
Latent Variable Sentiment Grammar
Neural models have been investigated for sentiment classification over constituent trees.
Second-Order Semantic Dependency Parsing with End-to-End Neural Networks
Semantic dependency parsing aims to identify semantic relationships between words in a sentence that form a graph.
Ranked #3 on
Semantic Dependency Parsing
on DM
Language Style Transfer from Sentences with Arbitrary Unknown Styles
Then, by recombining the content with the target style, we decode a sentence aligned in the target domain.
Gaussian Mixture Latent Vector Grammars
We introduce Latent Vector Grammars (LVeGs), a new framework that extends latent variable grammars such that each nonterminal symbol is associated with a continuous vector space representing the set of (infinitely many) subtypes of the nonterminal.
QA4IE: A Question Answering based Framework for Information Extraction
Information Extraction (IE) refers to automatically extracting structured relation tuples from unstructured texts.
Semi-supervised Structured Prediction with Neural CRF Autoencoder
In this paper we propose an end-to-end neural CRF autoencoder (NCRF-AE) model for semi-supervised learning of sequential structured prediction problems.
Maximum A Posteriori Inference in Sum-Product Networks
For the theoretical part, we reduce general MAP inference to its special case without evidence and hidden variables; we also show that it is NP-hard to approximate the MAP problem to $2^{n^\epsilon}$ for fixed $0 \leq \epsilon < 1$, where $n$ is the input size.
Structured Attentions for Visual Question Answering
In this paper, we demonstrate the importance of encoding such relations by showing the limited effective receptive field of ResNet on two datasets, and propose to model the visual attention as a multivariate distribution over a grid-structured Conditional Random Field on image regions.
CRF Autoencoder for Unsupervised Dependency Parsing
The encoder part of our model is discriminative and globally normalized which allows us to use rich features as well as universal linguistic priors.
Dependency Grammar Induction
Unsupervised Dependency Parsing
Dependency Grammar Induction with Neural Lexicalization and Big Training Data
We study the impact of big models (in terms of the degree of lexicalization) and big data (in terms of the training corpus size) on dependency grammar induction.
Combining Generative and Discriminative Approaches to Unsupervised Dependency Parsing via Dual Decomposition
Unsupervised dependency parsing aims to learn a dependency parser from unannotated sentences.
Latent Dependency Forest Models
Probabilistic modeling is one of the foundations of modern machine learning and artificial intelligence.
Stochastic And-Or Grammars: A Unified Framework and Logic Perspective
Stochastic And-Or grammars (AOG) extend traditional stochastic grammars of language to model other types of data such as images and events.
Mapping Energy Landscapes of Non-Convex Learning Problems
In many statistical learning problems, the target functions to be optimized are highly non-convex in various model spaces and thus are difficult to analyze.
Unsupervised Structure Learning of Stochastic And-Or Grammars
Stochastic And-Or grammars compactly represent both compositionality and reconfigurability and have been used to model different types of data such as images and events.
Joint Video and Text Parsing for Understanding Events and Answering Queries
We present a probabilistic generative model for joint parsing that captures the relations between the input video/text, their corresponding parse graphs and the joint parse graph.
