Search Results for author: Patrick Ng

Found 24 papers, 10 papers with code

Towards a Holistic Evaluation of LLMs on Factual Knowledge Recall

no code implementations24 Apr 2024 Jiaqing Yuan, Lin Pan, Chung-Wei Hang, Jiang Guo, Jiarong Jiang, Bonan Min, Patrick Ng, Zhiguo Wang

By further decoupling model known and unknown knowledge, we find the degradation is attributed to exemplars that contradict a model's known knowledge, as well as the number of such exemplars.


Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks

no code implementations31 Jan 2024 Wenyue Hua, Jiang Guo, Mingwen Dong, Henghui Zhu, Patrick Ng, Zhiguo Wang

Our analysis over the chain-of-thought generation of edited models further uncover key reasons behind the inadequacy of existing knowledge editing methods from a reasoning standpoint, involving aspects on fact-wise editing, fact recall ability, and coherence in generation.

counterfactual knowledge editing

Few-Shot Data-to-Text Generation via Unified Representation and Multi-Source Learning

no code implementations10 Aug 2023 Alexander Hanbo Li, Mingyue Shang, Evangelia Spiliopoulou, Jie Ma, Patrick Ng, Zhiguo Wang, Bonan Min, William Wang, Kathleen McKeown, Vittorio Castelli, Dan Roth, Bing Xiang

We present a novel approach for structured data-to-text generation that addresses the limitations of existing methods that primarily focus on specific types of structured data.

Data-to-Text Generation

UNITE: A Unified Benchmark for Text-to-SQL Evaluation

1 code implementation25 May 2023 Wuwei Lan, Zhiguo Wang, Anuj Chauhan, Henghui Zhu, Alexander Li, Jiang Guo, Sheng Zhang, Chung-Wei Hang, Joseph Lilien, Yiqun Hu, Lin Pan, Mingwen Dong, Jun Wang, Jiarong Jiang, Stephen Ash, Vittorio Castelli, Patrick Ng, Bing Xiang

A practical text-to-SQL system should generalize well on a wide variety of natural language questions, unseen database schemas, and novel SQL query structures.


Importance of Synthesizing High-quality Data for Text-to-SQL Parsing

no code implementations17 Dec 2022 Yiyun Zhao, Jiarong Jiang, Yiqun Hu, Wuwei Lan, Henry Zhu, Anuj Chauhan, Alexander Li, Lin Pan, Jun Wang, Chung-Wei Hang, Sheng Zhang, Marvin Dong, Joe Lilien, Patrick Ng, Zhiguo Wang, Vittorio Castelli, Bing Xiang

In this paper, we first examined the existing synthesized datasets and discovered that state-of-the-art text-to-SQL algorithms did not further improve on popular benchmarks when trained with augmented synthetic data.

SQL Parsing SQL-to-Text +2

Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations

no code implementations17 Dec 2022 Jifan Chen, Yuhao Zhang, Lan Liu, Rui Dong, Xinchi Chen, Patrick Ng, William Yang Wang, Zhiheng Huang

There has been great progress in unifying various table-to-text tasks using a single encoder-decoder model trained via multi-task learning (Xie et al., 2022).

Decoder Multi-Task Learning

DecAF: Joint Decoding of Answers and Logical Forms for Question Answering over Knowledge Bases

1 code implementation30 Sep 2022 Donghan Yu, Sheng Zhang, Patrick Ng, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Yiqun Hu, William Wang, Zhiguo Wang, Bing Xiang

Question answering over knowledge bases (KBs) aims to answer natural language questions with factual information such as entities and relations in KBs.

Entity Linking Question Answering +2

Improving Text-to-SQL Semantic Parsing with Fine-grained Query Understanding

no code implementations28 Sep 2022 Jun Wang, Patrick Ng, Alexander Hanbo Li, Jiarong Jiang, Zhiguo Wang, Ramesh Nallapati, Bing Xiang, Sudipta Sengupta

When synthesizing a SQL query, there is no explicit semantic information of NLQ available to the parser which leads to undesirable generalization performance.

NER Semantic Parsing +1

REKnow: Enhanced Knowledge for Joint Entity and Relation Extraction

no code implementations10 Jun 2022 Sheng Zhang, Patrick Ng, Zhiguo Wang, Bing Xiang

Our generative model is a unified framework to sequentially generate relational triplets under various relation extraction settings and explicitly utilizes relevant knowledge from Knowledge Graph (KG) to resolve ambiguities.

Joint Entity and Relation Extraction Relation

Dual Reader-Parser on Hybrid Textual and Tabular Evidence for Open Domain Question Answering

1 code implementation ACL 2021 Alexander Hanbo Li, Patrick Ng, Peng Xu, Henghui Zhu, Zhiguo Wang, Bing Xiang

However, a large amount of world's knowledge is stored in structured databases, and need to be accessed using query languages such as SQL.

Open-Domain Question Answering

End-to-End Cross-Domain Text-to-SQL Semantic Parsing with Auxiliary Task

no code implementations17 Jun 2021 Peng Shi, Tao Yu, Patrick Ng, Zhiguo Wang

Furthermore, we propose two value filling methods to build the bridge from the existing zero-shot semantic parsers to real-world applications, considering most of the existing parsers ignore the values filling in the synthesized SQL.

Semantic Parsing Text-To-SQL

Retrieval, Re-ranking and Multi-task Learning for Knowledge-Base Question Answering

no code implementations EACL 2021 Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang

Experiments show that: (1) Our IR-based retrieval method is able to collect high-quality candidates efficiently, thus enables our method adapt to large-scale KBs easily; (2) the BERT model improves the accuracy across all three sub-tasks; and (3) benefiting from multi-task learning, the unified model obtains further improvements with only 1/3 of the original parameters.

Entity Linking Information Retrieval +4

Learning Contextual Representations for Semantic Parsing with Generation-Augmented Pre-Training

3 code implementations18 Dec 2020 Peng Shi, Patrick Ng, Zhiguo Wang, Henghui Zhu, Alexander Hanbo Li, Jun Wang, Cicero Nogueira dos santos, Bing Xiang

Most recently, there has been significant interest in learning contextual representations for various NLP tasks, by leveraging large scale text corpora to train large neural language models with self-supervised learning objectives, such as Masked Language Model (MLM).

Ranked #7 on Text-To-SQL on spider (Exact Match Accuracy (Dev) metric)

Language Modelling Self-Supervised Learning +2

Template-Based Question Generation from Retrieved Sentences for Improved Unsupervised Question Answering

1 code implementation ACL 2020 Alexander R. Fabbri, Patrick Ng, Zhiguo Wang, Ramesh Nallapati, Bing Xiang

Training a QA model on this data gives a relative improvement over a previous unsupervised model in F1 score on the SQuAD dataset by about 14%, and 20% when the answer is a named entity, achieving state-of-the-art performance on SQuAD for unsupervised QA.

Language Modelling Question Answering +3

Universal Text Representation from BERT: An Empirical Study

no code implementations17 Oct 2019 Xiaofei Ma, Zhiguo Wang, Patrick Ng, Ramesh Nallapati, Bing Xiang

We present a systematic investigation of layer-wise BERT activations for general-purpose text representations to understand what linguistic information they capture and how transferable they are across different tasks.

Learning-To-Rank Natural Language Inference +4

Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering

no code implementations IJCNLP 2019 Zhiguo Wang, Patrick Ng, Xiaofei Ma, Ramesh Nallapati, Bing Xiang

To tackle this issue, we propose a multi-passage BERT model to globally normalize answer scores across all passages of the same question, and this change enables our QA model find better answers by utilizing more passages.

Open-Domain Question Answering

dna2vec: Consistent vector representations of variable-length k-mers

2 code implementations23 Jan 2017 Patrick Ng

One of the ubiquitous representation of long DNA sequence is dividing it into shorter k-mer components.

Cannot find the paper you are looking for? You can Submit a new open access paper.