Search Results for author: Kewei Tu

Found 97 papers, 56 papers with code

Cold-Start and Interpretability: Turning Regular Expressions into Trainable Recurrent Neural Networks

no code implementations EMNLP 2020 Chengyue Jiang, Yinggong Zhao, Shanbo Chu, Libin Shen, Kewei Tu

On the other hand, symbolic rules such as regular expressions are interpretable, require no training, and often achieve decent accuracy; but rules cannot benefit from labeled data when available and hence underperform neural networks in rich-resource scenarios.

text-classification Text Classification

Improving Constituent Representation with Hypertree Neural Networks

no code implementations NAACL 2022 Hao Zhou, Gongshen Liu, Kewei Tu

Many natural language processing tasks involve text spans and thus high-quality span representations are needed to enhance neural approaches to these tasks.

Sentence

SHARP: Search-Based Adversarial Attack for Structured Prediction

no code implementations Findings (NAACL) 2022 Liwen Zhang, Zixia Jia, Wenjuan Han, Zilong Zheng, Kewei Tu

Adversarial attack of structured prediction models faces various challenges such as the difficulty of perturbing discrete words, the sentence quality issue, and the sensitivity of outputs to small perturbations.

Adversarial Attack Dependency Parsing +4

Neuralizing Regular Expressions for Slot Filling

no code implementations EMNLP 2021 Chengyue Jiang, Zijian Jin, Kewei Tu

Neural models and symbolic rules such as regular expressions have their respective merits and weaknesses.

slot-filling Slot Filling

A Systematic Study of Cross-Layer KV Sharing for Efficient LLM Inference

1 code implementation18 Oct 2024 You Wu, HaoYi Wu, Kewei Tu

Recently, sharing key-value (KV) cache across layers has been found effective in efficient inference of large language models (LLMs).

Language Modelling

Efficient Long-range Language Modeling with Self-supervised Causal Retrieval

no code implementations2 Oct 2024 Xiang Hu, Zhihao Teng, Wei Wu, Kewei Tu

For a given input sequence, we split it into chunks and use the current chunk to retrieve past chunks for subsequent text generation.

Language Modelling Retrieval +1

Learning Robust Named Entity Recognizers From Noisy Data With Retrieval Augmentation

no code implementations26 Jul 2024 Chaoyi Ai, Yong Jiang, Shen Huang, Pengjun Xie, Kewei Tu

Named entity recognition (NER) models often struggle with noisy inputs, such as those with spelling mistakes or errors generated by Optical Character Recognition processes, and learning a robust NER model is challenging.

named-entity-recognition Named Entity Recognition +5

Dependency Transformer Grammars: Integrating Dependency Structures into Transformer Language Models

1 code implementation24 Jul 2024 Yida Zhao, Chao Lou, Kewei Tu

Syntactic Transformer language models aim to achieve better generalization through simultaneously modeling syntax trees and sentences.

ARC Inductive Bias +1

Sparser is Faster and Less is More: Efficient Sparse Attention for Long-Range Transformers

no code implementations24 Jun 2024 Chao Lou, Zixia Jia, Zilong Zheng, Kewei Tu

Accommodating long sequences efficiently in autoregressive Transformers, especially within an extended context window, poses significant challenges due to the quadratic computational complexity and substantial KV memory requirements inherent in self-attention mechanisms.

Language Modelling

Unsupervised Morphological Tree Tokenizer

no code implementations21 Jun 2024 Qingyang Zhu, Xiang Hu, Pengyu Ji, Wei Wu, Kewei Tu

Specifically, the deep model jointly encodes internal structures and representations of words with a mechanism named $\textit{MorphOverriding}$ to ensure the indecomposability of morphemes.

Language Modelling

Layer-Condensed KV Cache for Efficient Inference of Large Language Models

1 code implementation17 May 2024 HaoYi Wu, Kewei Tu

In this paper, we propose a novel method that only computes and caches the KVs of a small number of layers, thus significantly saving memory consumption and improving inference throughput.

Language Modelling

Potential and Limitations of LLMs in Capturing Structured Semantics: A Case Study on SRL

no code implementations10 May 2024 Ning Cheng, Zhaohui Yan, ZiMing Wang, Zhijie Li, Jiaming Yu, Zilong Zheng, Kewei Tu, Jinan Xu, Wenjuan Han

Large Language Models (LLMs) play a crucial role in capturing structured semantics to enhance language understanding, improve interpretability, and reduce bias.

Semantic Role Labeling

RoT: Enhancing Large Language Models with Reflection on Search Trees

1 code implementation8 Apr 2024 Wenyang Hui, Kewei Tu

It uses a strong LLM to summarize guidelines from previous tree search experiences to enhance the ability of a weak LLM.

Using Interpretation Methods for Model Enhancement

1 code implementation2 Apr 2024 Zhuo Chen, Chengyue Jiang, Kewei Tu

In this paper, we propose a framework of utilizing interpretation methods and gold rationales to enhance models.

Improving Retrieval Augmented Open-Domain Question-Answering with Vectorized Contexts

1 code implementation2 Apr 2024 Zhuo Chen, Xinyu Wang, Yong Jiang, Pengjun Xie, Fei Huang, Kewei Tu

With our method, the origin language models can cover several times longer contexts while keeping the computing requirements close to the baseline.

In-Context Learning Language Modelling +2

Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation

1 code implementation26 Nov 2023 HaoYi Wu, Kewei Tu

Specifically, we design a conditional random field that models discrete latent representations of all words in a sentence as well as dependency arcs between them; and we use mean field variational inference for approximate inference.

Sentence Variational Inference

Conic10K: A Challenging Math Problem Understanding and Reasoning Dataset

1 code implementation9 Nov 2023 HaoYi Wu, Wenyang Hui, Yezeng Chen, Weiqi Wu, Kewei Tu, Yi Zhou

Since the dataset only involves a narrow range of knowledge, it is easy to separately analyse the knowledge a model possesses and the reasoning ability it has.

Math Natural Language Understanding

Joint Entity and Relation Extraction with Span Pruning and Hypergraph Neural Networks

1 code implementation26 Oct 2023 Zhaohui Yan, Songlin Yang, Wei Liu, Kewei Tu

Also, most of current ERE models do not take into account higher-order interactions between multiple entities and relations, while higher-order modeling could be beneficial. In this work, we propose HyperGraph neural network for ERE ($\hgnn{}$), which is built upon the PL-marker (a state-of-the-art marker-based pipleline model).

Joint Entity and Relation Extraction NER +1

Simple Hardware-Efficient PCFGs with Independent Left and Right Productions

1 code implementation23 Oct 2023 Wei Liu, Songlin Yang, Yoon Kim, Kewei Tu

Scaling dense PCFGs to thousands of nonterminals via a low-rank parameterization of the rule probability tensor has been shown to be beneficial for unsupervised parsing.

Constituency Grammar Induction Language Modelling

AMR Parsing with Causal Hierarchical Attention and Pointers

1 code implementation18 Oct 2023 Chao Lou, Kewei Tu

Translation-based AMR parsers have recently gained popularity due to their simplicity and effectiveness.

AMR Parsing Decoder +1

Augmenting Transformers with Recursively Composed Multi-grained Representations

1 code implementation28 Sep 2023 Xiang Hu, Qingyang Zhu, Kewei Tu, Wei Wu

More interestingly, the hierarchical structures induced by ReCAT exhibit strong consistency with human-annotated syntactic trees, indicating good interpretability brought by the CIO layers.

Constituency Grammar Induction Natural Language Inference +2

Do PLMs Know and Understand Ontological Knowledge?

1 code implementation12 Sep 2023 Weiqi Wu, Chengyue Jiang, Yong Jiang, Pengjun Xie, Kewei Tu

In this paper, we focus on probing whether PLMs store ontological knowledge and have a semantic understanding of the knowledge rather than rote memorization of the surface form.

Logical Reasoning Memorization +1

Improving Grammar-based Sequence-to-Sequence Modeling with Decomposition and Constraints

1 code implementation5 Jun 2023 Chao Lou, Kewei Tu

Neural QCFG is a grammar-based sequence-tosequence (seq2seq) model with strong inductive biases on hierarchical structures.

A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification

2 code implementations6 Mar 2023 Xiang Hu, Xinyu Kong, Kewei Tu

As the structured language model learns to predict constituency trees in a self-supervised manner, only raw texts and sentence-level labels are required as training data, which makes it essentially a general constituent-level self-interpretable classification model.

Language Modelling Sentence +2

COMBO: A Complete Benchmark for Open KG Canonicalization

1 code implementation8 Feb 2023 Chengyue Jiang, Yong Jiang, Weiqi Wu, Yuting Zheng, Pengjun Xie, Kewei Tu

The subject and object noun phrases and the relation in open KG have severe redundancy and ambiguity and need to be canonicalized.

Open Knowledge Graph Canonicalization Relation

Modeling Instance Interactions for Joint Information Extraction with Neural High-Order Conditional Random Field

1 code implementation17 Dec 2022 Zixia Jia, Zhaohui Yan, Wenjuan Han, Zilong Zheng, Kewei Tu

Prior works on joint Information Extraction (IE) typically model instance (e. g., event triggers, entities, roles, relations) interactions by representation enhancement, type dependencies scoring, or global decoding.

Decoder Variational Inference

Modeling Label Correlations for Ultra-Fine Entity Typing with Neural Pairwise Conditional Random Field

1 code implementation3 Dec 2022 Chengyue Jiang, Yong Jiang, Weiqi Wu, Pengjun Xie, Kewei Tu

We use mean-field variational inference for efficient type inference on very large type sets and unfold it as a neural network module to enable end-to-end training.

Entity Typing Sentence +2

Named Entity and Relation Extraction with Multi-Modal Retrieval

1 code implementation3 Dec 2022 Xinyu Wang, Jiong Cai, Yong Jiang, Pengjun Xie, Kewei Tu, Wei Lu

MoRe contains a text retrieval module and an image-based retrieval module, which retrieve related knowledge of the input text and image in the knowledge corpus respectively.

Multi-modal Named Entity Recognition Named Entity Recognition +3

Nested Named Entity Recognition as Latent Lexicalized Constituency Parsing

1 code implementation ACL 2022 Chao Lou, Songlin Yang, Kewei Tu

They treat nested entities as partially-observed constituency trees and propose the masked inside algorithm for partial marginalization.

Constituency Parsing Entity Typing +4

ITA: Image-Text Alignments for Multi-Modal Named Entity Recognition

1 code implementation NAACL 2022 Xinyu Wang, Min Gui, Yong Jiang, Zixia Jia, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

As text representations take the most important role in MNER, in this paper, we propose {\bf I}mage-{\bf t}ext {\bf A}lignments (ITA) to align image features into the textual space, so that the attention mechanism in transformer-based pretrained textual embeddings can be better utilized.

Multi-modal Named Entity Recognition named-entity-recognition +1

Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks

1 code implementation ACL 2022 Songlin Yang, Kewei Tu

Constituency parsing and nested named entity recognition (NER) are similar tasks since they both aim to predict a collection of nested and non-crossing spans.

Constituency Parsing named-entity-recognition +5

Combining (second-order) graph-based and headed-span-based projective dependency parsing

1 code implementation Findings (ACL) 2022 Songlin Yang, Kewei Tu

Graph-based methods, which decompose the score of a dependency tree into scores of dependency arcs, are popular in dependency parsing for decades.

ARC Dependency Parsing

Headed-Span-Based Projective Dependency Parsing

1 code implementation ACL 2022 Songlin Yang, Kewei Tu

In a projective dependency tree, the largest subtree rooted at each word covers a contiguous sequence (i. e., a span) in the surface order.

Constituency Parsing Dependency Parsing

Adapting Unsupervised Syntactic Parsing Methodology for Discourse Dependency Parsing

no code implementations ACL 2021 Liwen Zhang, Ge Wang, Wenjuan Han, Kewei Tu

In this paper, we propose a simple yet effective method to adapt unsupervised syntactic dependency parsing methodology for unsupervised discourse dependency parsing.

Dependency Parsing Discourse Parsing

Risk Minimization for Zero-shot Sequence Labeling

no code implementations ACL 2021 Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

In this paper, we propose a novel unified framework for zero-shot sequence labeling with minimum risk training and design a new decomposable risk function that models the relations between the predicted labels from the source models and the true labels.

Multi-View Cross-Lingual Structured Prediction with Minimum Supervision

no code implementations ACL 2021 Zechuan Hu, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

In structured prediction problems, cross-lingual transfer learning is an efficient way to train quality models for low-resource languages, and further improvement can be obtained by learning from multiple source languages.

Cross-Lingual Transfer Sentence +2

Improving Named Entity Recognition by External Context Retrieving and Cooperative Learning

3 code implementations ACL 2021 Xinyu Wang, Yong Jiang, Nguyen Bach, Tao Wang, Zhongqiang Huang, Fei Huang, Kewei Tu

We find empirically that the contextual representations computed on the retrieval-based input view, constructed through the concatenation of a sentence and its external contexts, can achieve significantly improved performance compared to the original input view based only on the sentence.

Chinese Named Entity Recognition Chunking +3

PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols

1 code implementation NAACL 2021 Songlin Yang, Yanpeng Zhao, Kewei Tu

In this work, we present a new parameterization form of PCFGs based on tensor decomposition, which has at most quadratic computational complexity in the symbol number and therefore allows us to use a much larger number of symbols.

Constituency Grammar Induction

Unsupervised Natural Language Parsing (Introductory Tutorial)

no code implementations EACL 2021 Kewei Tu, Yong Jiang, Wenjuan Han, Yanpeng Zhao

Unsupervised parsing learns a syntactic parser from training sentences without parse tree annotations.

Constrained Text Generation with Global Guidance -- Case Study on CommonGen

no code implementations12 Mar 2021 Yixian Liu, Liwen Zhang, Wenjuan Han, Yue Zhang, Kewei Tu

We focus on CommonGen, the task of generating text based on a set of concepts, as a representative task of constrained text generation.

Common Sense Reasoning reinforcement-learning +4

Semi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding

no code implementations COLING 2020 Ge Wang, Kewei Tu

By relaxing the tree constraint in both the encoder and the decoder during training, we make the learning of our model fully arc-factored and thus circumvent the challenges brought by the tree constraint.

ARC Decoder +1

Deep Inside-outside Recursive Autoencoder with All-span Objective

no code implementations COLING 2020 Ruyue Hong, Jiong Cai, Kewei Tu

Deep inside-outside recursive autoencoder (DIORA) is a neural-based model designed for unsupervised constituency parsing.

Constituency Parsing Sentence

Neural Latent Dependency Model for Sequence Labeling

no code implementations10 Nov 2020 Yang Zhou, Yong Jiang, Zechuan Hu, Kewei Tu

One limitation of linear chain CRFs is their inability to model long-range dependencies between labels.

Learning Numeral Embedding

1 code implementation Findings of the Association for Computational Linguistics 2020 Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, Kewei Tu

Numeral embeddings represented in this manner can be plugged into existing word embedding learning approaches such as skip-gram for training.

Word Similarity

Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders

1 code implementation Findings of the Association for Computational Linguistics 2020 Zhao Li, Kewei Tu

We consider the task of cross-lingual adaptation of dependency parsers without annotated target corpora and parallel corpora.

Second-Order Unsupervised Neural Dependency Parsing

1 code implementation COLING 2020 Songlin Yang, Yong Jiang, Wenjuan Han, Kewei Tu

Inspired by second-order supervised dependency parsing, we proposed a second-order extension of unsupervised neural dependency models that incorporate grandparent-child or sibling information.

Dependency Grammar Induction

Adversarial Attack and Defense of Structured Prediction Models

1 code implementation EMNLP 2020 Wenjuan Han, Liwen Zhang, Yong Jiang, Kewei Tu

To address these problems, we propose a novel and unified framework that learns to attack a structured prediction model using a sequence-to-sequence model with feedbacks from multiple reference models of the same structured prediction task.

Adversarial Attack Dependency Parsing +3

An Empirical Comparison of Unsupervised Constituency Parsing Methods

no code implementations ACL 2020 Jun Li, Yifan Cao, Jiong Cai, Yong Jiang, Kewei Tu

Unsupervised constituency parsing aims to learn a constituency parser from a training corpus without parse tree annotations.

Constituency Parsing

Semi-Supervised Semantic Dependency Parsing Using CRF Autoencoders

1 code implementation ACL 2020 Zixia Jia, Youmi Ma, Jiong Cai, Kewei Tu

Semantic dependency parsing, which aims to find rich bi-lexical relationships, allows words to have multiple dependency heads, resulting in graph-structured representations.

ARC Decoder +3

Learning Numeral Embeddings

no code implementations28 Dec 2019 Chengyue Jiang, Zhonglin Nian, Kaihao Guo, Shanbo Chu, Yinggong Zhao, Libin Shen, Kewei Tu

Numeral embeddings represented in this manner can be plugged into existing word embedding learning approaches such as skip-gram for training.

Word Similarity

A Regularization-based Framework for Bilingual Grammar Induction

no code implementations IJCNLP 2019 Yong Jiang, Wenjuan Han, Kewei Tu

Grammar induction aims to discover syntactic structures from unannotated sentences.

Multilingual Grammar Induction with Continuous Language Identification

no code implementations IJCNLP 2019 Wenjuan Han, Ge Wang, Yong Jiang, Kewei Tu

The key to multilingual grammar induction is to couple grammar parameters of different languages together by exploiting the similarity between languages.

Language Identification

Bidirectional Transition-Based Dependency Parsing

1 code implementation AAAI 2019 Yunzhe Yuan, Yong Jiang, Kewei Tu

Traditionally, a transitionbased dependency parser processes an input sentence and predicts a sequence of parsing actions in a left-to-right manner.

Sentence Transition-Based Dependency Parsing

Enhancing Unsupervised Generative Dependency Parser with Contextual Information

no code implementations ACL 2019 Wenjuan Han, Yong Jiang, Kewei Tu

In this paper, we propose a novel probabilistic model called discriminative neural dependency model with valence (D-NDMV) that generates a sentence and its parse from a continuous latent representation, which encodes global contextual information of the generated sentence.

Constituency Grammar Induction Dependency Grammar Induction +2

Language Style Transfer from Sentences with Arbitrary Unknown Styles

no code implementations13 Aug 2018 Yanpeng Zhao, Wei Bi, Deng Cai, Xiaojiang Liu, Kewei Tu, Shuming Shi

Then, by recombining the content with the target style, we decode a sentence aligned in the target domain.

Sentence Sentence ReWriting +1

Gaussian Mixture Latent Vector Grammars

1 code implementation ACL 2018 Yanpeng Zhao, Liwen Zhang, Kewei Tu

We introduce Latent Vector Grammars (LVeGs), a new framework that extends latent variable grammars such that each nonterminal symbol is associated with a continuous vector space representing the set of (infinitely many) subtypes of the nonterminal.

Constituency Parsing Part-Of-Speech Tagging

QA4IE: A Question Answering based Framework for Information Extraction

1 code implementation10 Apr 2018 Lin Qiu, Hao Zhou, Yanru Qu, Wei-Nan Zhang, Suoheng Li, Shu Rong, Dongyu Ru, Lihua Qian, Kewei Tu, Yong Yu

Information Extraction (IE) refers to automatically extracting structured relation tuples from unstructured texts.

Question Answering Relation +2

Semi-supervised Structured Prediction with Neural CRF Autoencoder

1 code implementation EMNLP 2017 Xiao Zhang, Yong Jiang, Hao Peng, Kewei Tu, Dan Goldwasser

In this paper we propose an end-to-end neural CRF autoencoder (NCRF-AE) model for semi-supervised learning of sequential structured prediction problems.

Decoder Part-Of-Speech Tagging +3

Maximum A Posteriori Inference in Sum-Product Networks

no code implementations16 Aug 2017 Jun Mei, Yong Jiang, Kewei Tu

For the theoretical part, we reduce general MAP inference to its special case without evidence and hidden variables; we also show that it is NP-hard to approximate the MAP problem to $2^{n^\epsilon}$ for fixed $0 \leq \epsilon < 1$, where $n$ is the input size.

Structured Attentions for Visual Question Answering

1 code implementation ICCV 2017 Chen Zhu, Yanpeng Zhao, Shuaiyi Huang, Kewei Tu, Yi Ma

In this paper, we demonstrate the importance of encoding such relations by showing the limited effective receptive field of ResNet on two datasets, and propose to model the visual attention as a multivariate distribution over a grid-structured Conditional Random Field on image regions.

Visual Question Answering

CRF Autoencoder for Unsupervised Dependency Parsing

1 code implementation EMNLP 2017 Jiong Cai, Yong Jiang, Kewei Tu

The encoder part of our model is discriminative and globally normalized which allows us to use rich features as well as universal linguistic priors.

Dependency Grammar Induction Unsupervised Dependency Parsing

Dependency Grammar Induction with Neural Lexicalization and Big Training Data

no code implementations EMNLP 2017 Wenjuan Han, Yong Jiang, Kewei Tu

We study the impact of big models (in terms of the degree of lexicalization) and big data (in terms of the training corpus size) on dependency grammar induction.

Dependency Grammar Induction

Latent Dependency Forest Models

no code implementations8 Sep 2016 Shanbo Chu, Yong Jiang, Kewei Tu

Probabilistic modeling is one of the foundations of modern machine learning and artificial intelligence.

Stochastic And-Or Grammars: A Unified Framework and Logic Perspective

no code implementations2 Jun 2015 Kewei Tu

Stochastic And-Or grammars (AOG) extend traditional stochastic grammars of language to model other types of data such as images and events.

Relational Reasoning

Mapping Energy Landscapes of Non-Convex Learning Problems

no code implementations2 Oct 2014 Maria Pavlovskaia, Kewei Tu, Song-Chun Zhu

In many statistical learning problems, the target functions to be optimized are highly non-convex in various model spaces and thus are difficult to analyze.

Clustering

Unsupervised Structure Learning of Stochastic And-Or Grammars

no code implementations NeurIPS 2013 Kewei Tu, Maria Pavlovskaia, Song-Chun Zhu

Stochastic And-Or grammars compactly represent both compositionality and reconfigurability and have been used to model different types of data such as images and events.

Joint Video and Text Parsing for Understanding Events and Answering Queries

no code implementations29 Aug 2013 Kewei Tu, Meng Meng, Mun Wai Lee, Tae Eun Choe, Song-Chun Zhu

We present a probabilistic generative model for joint parsing that captures the relations between the input video/text, their corresponding parse graphs and the joint parse graph.

Semantic Parsing

Cannot find the paper you are looking for? You can Submit a new open access paper.