Search Results for author: PengFei Liu

Found 129 papers, 76 papers with code

Multi-task Learning with Gradient Communication

no code implementations ICLR 2019 Pengfei Liu, Xuanjing Huang

In this paper, we describe a general framework to systematically analyze current neural models for multi-task learning, in which we find that existing models expect to disentangle features into different spaces while features learned in practice are still entangled in shared space, leaving potential hazards for other training or unseen tasks.

Inductive Bias Multi-Task Learning

Are Factuality Checkers Reliable? Adversarial Meta-evaluation of Factuality in Summarization

1 code implementation Findings (EMNLP) 2021 Yiran Chen, PengFei Liu, Xipeng Qiu

In this paper, we present an adversarial meta-evaluation methodology that allows us to (i) diagnose the fine-grained strengths and weaknesses of 6 existing top-performing metrics over 24 diagnostic test datasets, (ii) search for directions for further improvement by data augmentation.

Data Augmentation

Understanding Reference Policies in Direct Preference Optimization

no code implementations18 Jul 2024 Yixin Liu, PengFei Liu, Arman Cohan

In this work, we explore an under-investigated aspect of DPO - its dependency on the reference model or policy.

Weak-to-Strong Reasoning

no code implementations18 Jul 2024 Yuqing Yang, Yan Ma, PengFei Liu

When large language models (LLMs) exceed human-level capabilities, it becomes increasingly challenging to provide full-scale and accurate supervisions for these models.

Halu-J: Critique-Based Hallucination Judge

no code implementations17 Jul 2024 Binjie Wang, Steffi Chern, Ethan Chern, PengFei Liu

To address these challenges, we introduce Halu-J, a critique-based hallucination judge with 7 billion parameters.

ANOLE: An Open, Autoregressive, Native Large Multimodal Models for Interleaved Image-Text Generation

1 code implementation8 Jul 2024 Ethan Chern, Jiadi Su, Yan Ma, PengFei Liu

Previous open-source large multimodal models (LMMs) have faced several limitations: (1) they often lack native integration, requiring adapters to align visual representations with pre-trained large language models (LLMs); (2) many are restricted to single-modal generation; (3) while some support multimodal generation, they rely on separate diffusion models for visual modeling and generation.

multimodal generation Text Generation

Progress or Regress? Self-Improvement Reversal in Post-training

no code implementations6 Jul 2024 Ting Wu, Xuefeng Li, PengFei Liu

Self-improvement through post-training methods such as iterative preference learning has been acclaimed for enhancing the problem-solving capabilities (e. g., mathematical reasoning) of Large Language Models (LLMs) without human intervention.

Diversity Mathematical Reasoning

FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in Large Language Models

1 code implementation1 Jul 2024 Yiyuan Li, Shichao Sun, PengFei Liu

Fuzzy reasoning is vital due to the frequent use of imprecise information in daily contexts.

Mathematical Reasoning

OlympicArena Medal Ranks: Who Is the Most Intelligent AI So Far?

1 code implementation24 Jun 2024 Zhen Huang, Zengzhi Wang, Shijie Xia, PengFei Liu

In this report, we pose the following question: Who is the most intelligent AI model to date, as measured by the OlympicArena (an Olympic-level, multi-discipline, multi-modal benchmark for superintelligent AI)?

BeHonest: Benchmarking Honesty in Large Language Models

1 code implementation19 Jun 2024 Steffi Chern, Zhulin Hu, Yuqing Yang, Ethan Chern, Yuan Guo, Jiahe Jin, Binjie Wang, PengFei Liu

Building on this foundation, we designed 10 scenarios to evaluate and analyze 9 popular LLMs on the market, including both closed-source and open-source models from different model families with varied model sizes.

Benchmarking Misinformation

OlympicArena: Benchmarking Multi-discipline Cognitive Reasoning for Superintelligent AI

1 code implementation18 Jun 2024 Zhen Huang, Zengzhi Wang, Shijie Xia, Xuefeng Li, Haoyang Zou, Ruijie Xu, Run-Ze Fan, Lyumanshan Ye, Ethan Chern, Yixin Ye, Yikai Zhang, Yuqing Yang, Ting Wu, Binjie Wang, Shichao Sun, Yang Xiao, Yiyuan Li, Fan Zhou, Steffi Chern, Yiwei Qin, Yan Ma, Jiadi Su, Yixiu Liu, Yuxiang Zheng, Shaoting Zhang, Dahua Lin, Yu Qiao, PengFei Liu

We delve into the models' cognitive reasoning abilities, their performance across different modalities, and their outcomes in process-level evaluations, which are vital for tasks requiring complex reasoning with lengthy solutions.

Benchmarking

MoPS: Modular Story Premise Synthesis for Open-Ended Automatic Story Generation

1 code implementation9 Jun 2024 Yan Ma, Yu Qiao, PengFei Liu

In supplementary materials, we provide the MoPS code suite, along with 7. 6k generated premises and 1k extended stories.

Diversity Sentence +1

Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization

no code implementations1 Jun 2024 Shichao Sun, Ruifeng Yuan, Ziqiang Cao, Wenjie Li, PengFei Liu

Two strategies are designed to perform this iterative process: Prompt Chaining and Stepwise Prompt.

Text Summarization

Benchmarking Benchmark Leakage in Large Language Models

1 code implementation29 Apr 2024 Ruijie Xu, Zengzhi Wang, Run-Ze Fan, PengFei Liu

By analyzing 31 LLMs under the context of mathematical reasoning, we reveal substantial instances of training even test set misuse, resulting in potentially unfair comparisons.

Benchmarking Mathematical Reasoning

RHanDS: Refining Malformed Hands for Generated Images with Decoupled Structure and Style Guidance

no code implementations22 Apr 2024 Chengrui Wang, PengFei Liu, Min Zhou, Ming Zeng, Xubin Li, Tiezheng Ge, Bo Zheng

The style guidance is a hand image, e. g., the malformed hand itself, and is employed to furnish the style reference for hand refining.

Evaluating Mathematical Reasoning Beyond Accuracy

1 code implementation8 Apr 2024 Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, PengFei Liu

To measure reasoning beyond final-answer accuracy, we introduce ReasonEval, a new methodology for evaluating the quality of reasoning steps.

Math Mathematical Reasoning

CodeBenchGen: Creating Scalable Execution-based Code Generation Benchmarks

1 code implementation31 Mar 2024 Yiqing Xie, Alex Xie, Divyanshu Sheth, PengFei Liu, Daniel Fried, Carolyn Rose

To demonstrate the complexity and solvability of examples in Exec-CSN, we present a human study demonstrating that 81. 3% of the examples can be solved by humans and 61% are rated as "requires effort to solve".

Code Generation Language Modelling +1

LLMCRIT: Teaching Large Language Models to Use Criteria

1 code implementation2 Mar 2024 Weizhe Yuan, PengFei Liu, Matthias Gallé

In particular, we present a model-in-the-loop framework that semi-automatically derives criteria from collected guidelines for different writing tasks and constructs in-context demonstrations for each criterion.

Reformatted Alignment

1 code implementation19 Feb 2024 Run-Ze Fan, Xuefeng Li, Haoyang Zou, Junlong Li, Shwai He, Ethan Chern, Jiewen Hu, PengFei Liu

This paper explores elevating the quality of existing instruction data to better align with human values, introducing a simple and effective approach named ReAlign, which reformats the responses of instruction data into a format that better aligns with pre-established criteria and the collated evidence.

GSM8K Hallucination +2

Dissecting Human and LLM Preferences

1 code implementation17 Feb 2024 Junlong Li, Fan Zhou, Shichao Sun, Yikai Zhang, Hai Zhao, PengFei Liu

As a relative quality comparison of model responses, human and Large Language Model (LLM) preferences serve as common alignment goals in model fine-tuning and criteria in evaluation.

Language Modelling Large Language Model

Impact of Domain Knowledge and Multi-Modality on Intelligent Molecular Property Prediction: A Systematic Survey

no code implementations11 Feb 2024 Taojie Kuang, PengFei Liu, Zhixiang Ren

We discover that integrating molecular information significantly improves molecular property prediction (MPP) for both regression and classification tasks.

Drug Discovery Molecular Property Prediction +2

Scientific Language Modeling: A Quantitative Review of Large Language Models in Molecular Science

1 code implementation6 Feb 2024 PengFei Liu, Jun Tao, Zhixiang Ren

Efficient molecular modeling and design are crucial for the discovery and exploration of novel molecules, and the incorporation of deep learning methods has revolutionized this field.

Language Modelling

Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate

1 code implementation30 Jan 2024 Steffi Chern, Ethan Chern, Graham Neubig, PengFei Liu

Despite the utility of Large Language Models (LLMs) across a wide range of tasks and scenarios, developing a method for reliably evaluating LLMs across varied contexts continues to be challenging.

Extending LLMs' Context Window with 100 Samples

1 code implementation13 Jan 2024 Yikai Zhang, Junlong Li, PengFei Liu

Large Language Models (LLMs) are known to have limited extrapolation ability beyond their pre-trained context window, constraining their application in downstream tasks with lengthy inputs.

Position

The Critique of Critique

1 code implementation9 Jan 2024 Shichao Sun, Junlong Li, Weizhe Yuan, Ruifeng Yuan, Wenjie Li, PengFei Liu

Critique, as a natural language description for assessing the quality of model-generated content, has played a vital role in the training, evaluation, and refinement of LLMs.

Question Answering

InFoBench: Evaluating Instruction Following Ability in Large Language Models

1 code implementation7 Jan 2024 Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, PengFei Liu, Dong Yu

This paper introduces the Decomposed Requirements Following Ratio (DRFR), a new metric for evaluating Large Language Models' (LLMs) ability to follow instructions.

Instruction Following

Generative AI for Math: Part I -- MathPile: A Billion-Token-Scale Pretraining Corpus for Math

1 code implementation28 Dec 2023 Zengzhi Wang, Rui Xia, PengFei Liu

Our meticulous data collection and processing efforts included a complex suite of preprocessing, prefiltering, language identification, cleaning, filtering, and deduplication, ensuring the high quality of our corpus.

Language Identification Math +1

How Far Are LLMs from Believable AI? A Benchmark for Evaluating the Believability of Human Behavior Simulation

1 code implementation28 Dec 2023 Yang Xiao, Yi Cheng, Jinlan Fu, Jiashuo Wang, Wenjie Li, PengFei Liu

In recent years, AI has demonstrated remarkable capabilities in simulating human behaviors, particularly those implemented with large language models (LLMs).

AI Agent Language Modelling

Align on the Fly: Adapting Chatbot Behavior to Established Norms

1 code implementation26 Dec 2023 Chunpu Xu, Steffi Chern, Ethan Chern, Ge Zhang, Zekun Wang, Ruibo Liu, Jing Li, Jie Fu, PengFei Liu

In this paper, we aim to align large language models with the ever-changing, complex, and diverse human values (e. g., social norms) across time and locations.

Chatbot

Alignment for Honesty

1 code implementation12 Dec 2023 Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, PengFei Liu

Recent research has made significant strides in applying alignment techniques to enhance the helpfulness and harmlessness of large language models (LLMs) in accordance with human intentions.

Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization

1 code implementation15 Nov 2023 Yixin Liu, Alexander R. Fabbri, Jiawen Chen, Yilun Zhao, Simeng Han, Shafiq Joty, PengFei Liu, Dragomir Radev, Chien-Sheng Wu, Arman Cohan

Our study reveals that instruction controllable text summarization remains a challenging task for LLMs, since (1) all LLMs evaluated still make factual and other types of errors in their summaries; (2) no LLM-based evaluation methods can achieve a strong alignment with human annotators when judging the quality of candidate summaries; (3) different LLMs show large performance gaps in summary generation and evaluation capabilities.

Benchmarking Instruction Following +1

LoBaSS: Gauging Learnability in Supervised Fine-tuning Data

no code implementations16 Oct 2023 Haotian Zhou, Tingkai Liu, Qianli Ma, Jianbo Yuan, PengFei Liu, Yang You, Hongxia Yang

In this paper, we introduce a new dimension in SFT data selection: learnability.

Generative Judge for Evaluating Alignment

1 code implementation9 Oct 2023 Junlong Li, Shichao Sun, Weizhe Yuan, Run-Ze Fan, Hai Zhao, PengFei Liu

The rapid development of Large Language Models (LLMs) has substantially expanded the range of tasks they can address.

Being Aware of Localization Accuracy By Generating Predicted-IoU-Guided Quality Scores

1 code implementation23 Sep 2023 PengFei Liu, Weibo Wang, Yuhan Guo, Jiubin Tan

Distinctly, for alleviating the inconsistency of classification score and localization quality during training and inference, under which some predictions with low classification scores but high LQE scores will impair the performance, instead of separately and independently setting, we embedded LQE branch into classification branch, producing a joint classification-localization-quality representation.

Classification

The Impact of Downgrading Protected Areas (PAD) on Biodiversity

no code implementations31 Aug 2023 Yufei Li, Lingling Hou, PengFei Liu

We quantitatively assess the impacts of Downgrading Protected Areas (PAD) on biodiversity in the U. S..

GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text

1 code implementation14 Aug 2023 PengFei Liu, Yiming Ren, Jun Tao, Zhixiang Ren

Large language models have made significant strides in natural language processing, enabling innovative applications in molecular science by processing textual representations of molecules.

Image Captioning Language Modelling +5

FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios

3 code implementations25 Jul 2023 I-Chun Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, PengFei Liu

With the above challenges in mind, in this paper, we propose FacTool, a task and domain agnostic framework for detecting factual errors of texts generated by large language models (e. g., ChatGPT).

Code Generation Fact Checking +1

DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions

1 code implementation26 May 2023 Vijay Viswanathan, Luyu Gao, Tongshuang Wu, PengFei Liu, Graham Neubig

Using this data, we compare various information retrieval algorithms on our test set and present a superior bi-encoder retriever for text-based dataset recommendation.

Information Retrieval Retrieval

On Learning to Summarize with Large Language Models as References

1 code implementation23 May 2023 Yixin Liu, Kejian Shi, Katherine S He, Longtian Ye, Alexander R. Fabbri, PengFei Liu, Dragomir Radev, Arman Cohan

Recent studies have found that summaries generated by large language models (LLMs) are favored by human annotators over the original reference summaries in commonly used summarization datasets.

Contrastive Learning Text Summarization

LIMA: Less Is More for Alignment

5 code implementations NeurIPS 2023 Chunting Zhou, PengFei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, Omer Levy

Large language models are trained in two stages: (1) unsupervised pretraining from raw text, to learn general-purpose representations, and (2) large scale instruction tuning and reinforcement learning, to better align to end tasks and user preferences.

Language Modelling reinforcement-learning

MusicFace: Music-driven Expressive Singing Face Synthesis

no code implementations24 Mar 2023 PengFei Liu, Wenjin Deng, Hengda Li, Jintai Wang, Yinglin Zheng, Yiwei Ding, Xiaohu Guo, Ming Zeng

In this paper, we present a method for this task with natural motions of the lip, facial expression, head pose, and eye states.

Face Generation

Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation

1 code implementation7 Mar 2023 Yixin Liu, Alexander R. Fabbri, Yilun Zhao, PengFei Liu, Shafiq Joty, Chien-Sheng Wu, Caiming Xiong, Dragomir Radev

Interpretability and efficiency are two important considerations for the adoption of neural automatic metrics.

GPTScore: Evaluate as You Desire

3 code implementations8 Feb 2023 Jinlan Fu, See-Kiong Ng, Zhengbao Jiang, PengFei Liu

Generative Artificial Intelligence (AI) has enabled the development of sophisticated models that are capable of producing high-caliber text, images, and other outputs through the utilization of large pre-trained models.

Text Generation

a cognitive frequency allocation strategy for multi-carrier radar against communication interference

no code implementations23 Dec 2022 Zhao Shan, Lei Wang, PengFei Liu, Tianyao Huang, Yimin Liu

To address this challenge, we use a novel iteratively selecting technique which breaks a difficult decision task into several easy tasks.

T5Score: Discriminative Fine-tuning of Generative Evaluation Metrics

1 code implementation12 Dec 2022 Yiwei Qin, Weizhe Yuan, Graham Neubig, PengFei Liu

Both have their advantages; discriminative metrics are able to directly optimize for the problem of distinguishing between good and bad outputs, while generative metrics can be trained using abundant raw text.

Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization

no code implementations12 Dec 2022 Yiwei Qin, Graham Neubig, PengFei Liu

Recently, a large number of tuning strategies have been proposed to adapt pre-trained language models to downstream tasks.

Text Summarization

PAL: Program-aided Language Models

3 code implementations18 Nov 2022 Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, PengFei Liu, Yiming Yang, Jamie Callan, Graham Neubig

Much of this success can be attributed to prompting methods such as "chain-of-thought'', which employ LLMs for both understanding the problem description by decomposing it into steps, as well as solving each step of the problem.

Arithmetic Reasoning GSM8K +2

Towards a Unified Multi-Dimensional Evaluator for Text Generation

2 code implementations13 Oct 2022 Ming Zhong, Yang Liu, Da Yin, Yuning Mao, Yizhu Jiao, PengFei Liu, Chenguang Zhu, Heng Ji, Jiawei Han

We re-frame NLG evaluation as a Boolean Question Answering (QA) task, and by guiding the model with different questions, we can use one evaluator to evaluate from multiple dimensions.

nlg evaluation Question Answering +4

Artificial Neural Networks for Finger Vein Recognition: A Survey

no code implementations29 Aug 2022 Yimin Yin, Renye Zhang, PengFei Liu, Wanxia Deng, Siliang He, Chen Li, Jinghua Zhang

To our best knowledge, this paper is the first comprehensive survey focusing on finger vein recognition based on artificial neural networks.

Feature Engineering Finger Vein Recognition

reStructured Pre-training

2 code implementations22 Jun 2022 Weizhe Yuan, PengFei Liu

In addition, we test our model in the 2022 College Entrance Examination English that happened a few days ago (2022. 06. 08), and it gets a total score of 134 (v. s.

Polyglot Prompt: Multilingual Multitask PrompTraining

1 code implementation29 Apr 2022 Jinlan Fu, See-Kiong Ng, PengFei Liu

This paper aims for a potential architectural improvement for multilingual learning and asks: Can different tasks from different languages be modeled in a monolithic framework, i. e. without any task/language-specific module?

named-entity-recognition Named Entity Recognition +7

BRIO: Bringing Order to Abstractive Summarization

3 code implementations ACL 2022 Yixin Liu, PengFei Liu, Dragomir Radev, Graham Neubig

Abstractive summarization models are commonly trained using maximum likelihood estimation, which assumes a deterministic (one-point) target distribution in which an ideal model will assign all the probability mass to the reference summary.

Abstractive Text Summarization

DataLab: A Platform for Data Analysis and Intervention

no code implementations ACL 2022 Yang Xiao, Jinlan Fu, Weizhe Yuan, Vijay Viswanathan, Zhoumianze Liu, Yixin Liu, Graham Neubig, PengFei Liu

Despite data's crucial role in machine learning, most existing tools and research tend to focus on systems on top of existing data rather than how to interpret and manipulate data.

The MSXF TTS System for ICASSP 2022 ADD Challenge

no code implementations27 Jan 2022 Chunyong Yang, PengFei Liu, Yanli Chen, Hongbin Wang, Min Liu

The end to end TTS system is VITS, and the pre-training self-supervised model is wav2vec 2. 0.

Group Gated Fusion on Attention-based Bidirectional Alignment for Multimodal Emotion Recognition

1 code implementation17 Jan 2022 PengFei Liu, Kun Li, Helen Meng

Emotion recognition is a challenging and actively-studied research area that plays a critical role in emotion-aware human-computer interaction systems.

Multimodal Emotion Recognition

Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing

1 code implementation28 Jul 2021 PengFei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig

This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub "prompt-based learning".

Language Modelling Zero-Shot Learning

BARTScore: Evaluating Generated Text as Text Generation

2 code implementations NeurIPS 2021 Weizhe Yuan, Graham Neubig, PengFei Liu

In this work, we conceptualize the evaluation of generated text as a text generation problem, modeled using pre-trained sequence-to-sequence models.

Decoder Informativeness +4

How well do you know your summarization datasets?

1 code implementation Findings (ACL) 2021 Priyam Tejaswin, Dhruv Naik, PengFei Liu

(2) The performance of models and reliability of metrics is dependent on sample complexity.

Diversity

CitationIE: Leveraging the Citation Graph for Scientific Information Extraction

1 code implementation ACL 2021 Vijay Viswanathan, Graham Neubig, PengFei Liu

Automatically extracting key information from scientific documents has the potential to help scientists work more efficiently and accelerate the pace of scientific progress.

SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization

2 code implementations ACL 2021 Yixin Liu, PengFei Liu

In this paper, we present a conceptually simple while empirically powerful framework for abstractive summarization, SimCLS, which can bridge the gap between the learning objective and evaluation metrics resulting from the currently dominated sequence-to-sequence learning framework by formulating text generation as a reference-free evaluation problem (i. e., quality estimation) assisted by contrastive learning.

Abstractive Text Summarization Contrastive Learning +1

SpanNER: Named Entity Re-/Recognition as Span Prediction

1 code implementation ACL 2021 Jinlan Fu, Xuanjing Huang, PengFei Liu

Recent years have seen the paradigm shift of Named Entity Recognition (NER) systems from sequence labeling to span prediction.

named-entity-recognition Named Entity Recognition +1

Out-of-Scope Domain and Intent Classification through Hierarchical Joint Modeling

1 code implementation30 Apr 2021 PengFei Liu, Kun Li, Helen Meng

User queries for a real-world dialog system may sometimes fall outside the scope of the system's capabilities, but appropriate system responses will enable smooth processing throughout the human-computer interaction.

Classification General Classification +3

Open Intent Discovery through Unsupervised Semantic Clustering and Dependency Parsing

1 code implementation25 Apr 2021 PengFei Liu, Youzhang Ning, King Keung Wu, Kun Li, Helen Meng

This paper presents an unsupervised two-stage approach to discover intents and generate meaningful intent labels automatically from a collection of unlabeled utterances in a domain.

Clustering Dependency Parsing +4

RefSum: Refactoring Neural Summarization

1 code implementation NAACL 2021 Yixin Liu, Zi-Yi Dou, PengFei Liu

Although some recent works show potential complementarity among different state-of-the-art systems, few works try to investigate this problem in text summarization.

Text Summarization

ExplainaBoard: An Explainable Leaderboard for NLP

1 code implementation ACL 2021 PengFei Liu, Jinlan Fu, Yang Xiao, Weizhe Yuan, Shuaicheng Chang, Junqi Dai, Yixin Liu, Zihuiwen Ye, Zi-Yi Dou, Graham Neubig

In this paper, we present a new conceptualization and implementation of NLP evaluation: the ExplainaBoard, which in addition to inheriting the functionality of the standard leaderboard, also allows researchers to (i) diagnose strengths and weaknesses of a single system (e. g.~what is the best-performing system bad at?)

Machine Translation

Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTa

1 code implementation NAACL 2021 Junqi Dai, Hang Yan, Tianxiang Sun, PengFei Liu, Xipeng Qiu

In this paper, we firstly compare the induced trees from PTMs and the dependency parsing trees on several popular models for the ABSA task, showing that the induced tree from fine-tuned RoBERTa (FT-RoBERTa) outperforms the parser-provided tree.

Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA) +1

Larger-Context Tagging: When and Why Does It Work?

no code implementations NAACL 2021 Jinlan Fu, Liangjing Feng, Qi Zhang, Xuanjing Huang, PengFei Liu

The development of neural networks and pretraining techniques has spawned many sentence-level tagging systems that achieved superior performance on typical benchmarks.

Attribute Sentence

Towards More Fine-grained and Reliable NLP Performance Prediction

1 code implementation EACL 2021 Zihuiwen Ye, PengFei Liu, Jinlan Fu, Graham Neubig

We perform an analysis of four types of NLP tasks, and both demonstrate the feasibility of fine-grained performance prediction and the necessity to perform reliability analysis for performance prediction methods in the future.

Can We Automate Scientific Reviewing?

1 code implementation30 Jan 2021 Weizhe Yuan, PengFei Liu, Graham Neubig

The rapid development of science and technology has been accompanied by an exponential growth in peer-reviewed scientific publications.

Review Generation

Accurate polymorphous description of the paramagnetic phases in MnBi$_{2}$Te$_{4}$

no code implementations7 Jan 2021 Yufei Zhao, Qiushi Yao, PengFei Liu, Jingzhi Han, Zhi Wang, Qihang Liu

Temperature-driven phase transition is a long-standing frontier in material science, among which the most common phenomenon is the transition from a low-temperature magnetic-ordered phase to a high-temperature paramagnetic phase.

Materials Science

Interpretable Multi-dataset Evaluation for Named Entity Recognition

2 code implementations EMNLP 2020 Jinlan Fu, PengFei Liu, Graham Neubig

With the proliferation of models for natural language processing tasks, it is even harder to understand the differences between models and their relative merits.

named-entity-recognition Named Entity Recognition +1

RethinkCWS: Is Chinese Word Segmentation a Solved Task?

1 code implementation EMNLP 2020 Jinlan Fu, PengFei Liu, Qi Zhang, Xuanjing Huang

The performance of the Chinese Word Segmentation (CWS) systems has gradually reached a plateau with the rapid development of deep neural networks, especially the successful use of large pre-trained models.

Chinese Word Segmentation

GSum: A General Framework for Guided Neural Abstractive Summarization

1 code implementation NAACL 2021 Zi-Yi Dou, PengFei Liu, Hiroaki Hayashi, Zhengbao Jiang, Graham Neubig

Neural abstractive summarization models are flexible and can produce coherent summaries, but they are sometimes unfaithful and can be difficult to control.

Abstractive Text Summarization

Re-evaluating Evaluation in Text Summarization

1 code implementation EMNLP 2020 Manik Bhandari, Pranav Gour, Atabak Ashfaq, PengFei Liu, Graham Neubig

Automated evaluation metrics as a stand-in for manual evaluation are an essential part of the development of text-generation tasks such as text summarization.

Text Generation Text Summarization

CDEvalSumm: An Empirical Study of Cross-Dataset Evaluation for Neural Summarization Systems

2 code implementations Findings of the Association for Computational Linguistics 2020 Yiran Chen, PengFei Liu, Ming Zhong, Zi-Yi Dou, Danqing Wang, Xipeng Qiu, Xuanjing Huang

In this paper, we perform an in-depth analysis of characteristics of different datasets and investigate the performance of different summarization models under a cross-dataset setting, in which a summarizer trained on one corpus will be evaluated on a range of out-of-domain corpora.

Text Summarization

Heterogeneous Graph Neural Networks for Extractive Document Summarization

1 code implementation ACL 2020 Danqing Wang, PengFei Liu, Yining Zheng, Xipeng Qiu, Xuanjing Huang

An intuitive way is to put them in the graph-based neural network, which has a more complex structure for capturing inter-sentence relationships.

Document Summarization Extractive Document Summarization +3

Robust Covariance Estimation for High-dimensional Compositional Data with Application to Microbial Communities Analysis

1 code implementation20 Apr 2020 Yong He, PengFei Liu, Xinsheng Zhang, Wang Zhou

We construct a Median-of-Means (MOM) estimator for the centered log-ratio covariance matrix and propose a thresholding procedure that is adaptive to the variability of individual entries.

Methodology

Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study

1 code implementation12 Jan 2020 Jinlan Fu, PengFei Liu, Qi Zhang, Xuanjing Huang

While neural network-based models have achieved impressive performance on a large body of NLP tasks, the generalization behavior of different models remains poorly understood: Does this excellent performance imply a perfect generalization model, or are there still some limitations?

named-entity-recognition Named Entity Recognition +1

RTM3D: Real-time Monocular 3D Detection from Object Keypoints for Autonomous Driving

2 code implementations ECCV 2020 Peixuan Li, Huaici Zhao, PengFei Liu, Feidao Cao

Different from these approaches, our method predicts the nine perspective keypoints of a 3D bounding box in image space, and then utilize the geometric relationship of 3D and 2D perspectives to recover the dimension, location, and orientation in 3D space.

Autonomous Driving Vehicle Pose Estimation

Target-Guided Structured Attention Network for Target-Dependent Sentiment Analysis

no code implementations TACL 2020 Ji Zhang, Chengyao Chen, PengFei Liu, Chao He, Cane Wing-Ki Leung

Second, it shows a strong advantage in determining the sentiment of a target when the context sentence contains multiple semantic segments.

Sentence Sentiment Analysis +1

Learning Sparse Sharing Architectures for Multiple Tasks

1 code implementation12 Nov 2019 Tianxiang Sun, Yunfan Shao, Xiaonan Li, PengFei Liu, Hang Yan, Xipeng Qiu, Xuanjing Huang

Most existing deep multi-task learning models are based on parameter sharing, such as hard sharing, hierarchical sharing, and soft sharing.

Multi-Task Learning

A Closer Look at Data Bias in Neural Extractive Summarization Models

no code implementations WS 2019 Ming Zhong, Danqing Wang, PengFei Liu, Xipeng Qiu, Xuanjing Huang

In this paper, we take stock of the current state of summarization datasets and explore how different factors of datasets influence the generalization behaviour of neural extractive summarization models.

Extractive Summarization

A TWO-STAGE FRAMEWORK FOR MATHEMATICAL EXPRESSION RECOGNITION

no code implementations25 Sep 2019 Jin Zhang, Weipeng Ming, PengFei Liu

In the first stage, this method locates and recognizes the math symbols of input image by object detection algorithm.

Math object-detection +2

Towards Interpretable Evaluations: A Case Study of Named Entity Recognition

no code implementations25 Sep 2019 Jinlan Fu, PengFei Liu, Xuanjing Huang

With the proliferation of models for natural language processing (NLP) tasks, it is even harder to understand the differences between models and their relative merits.

named-entity-recognition Named Entity Recognition +1

Exploring Domain Shift in Extractive Text Summarization

no code implementations30 Aug 2019 Danqing Wang, PengFei Liu, Ming Zhong, Jie Fu, Xipeng Qiu, Xuanjing Huang

Although domain shift has been well explored in many NLP applications, it still has received little attention in the domain of extractive text summarization.

Extractive Text Summarization Meta-Learning

Zero-shot Text-to-SQL Learning with Auxiliary Task

1 code implementation29 Aug 2019 Shuaichen Chang, PengFei Liu, Yun Tang, Jing Huang, Xiaodong He, Bo-Wen Zhou

Recent years have seen great success in the use of neural seq2seq models on the text-to-SQL task.

Text-To-SQL

DropAttention: A Regularization Method for Fully-Connected Self-Attention Networks

no code implementations25 Jul 2019 Lin Zehui, PengFei Liu, Luyao Huang, Junkun Chen, Xipeng Qiu, Xuanjing Huang

Variants dropout methods have been designed for the fully-connected layer, convolutional layer and recurrent layer in neural networks, and shown to be effective to avoid overfitting.

TIGS: An Inference Algorithm for Text Infilling with Gradient Search

1 code implementation ACL 2019 Dayiheng Liu, Jie Fu, PengFei Liu, Jiancheng Lv

Text infilling is defined as a task for filling in the missing part of a sentence or paragraph, which is suitable for many real-world natural language generation scenarios.

Sentence Text Infilling

Cognitive Radar Using Reinforcement Learning in Automotive Applications

no code implementations24 Apr 2019 PengFei Liu, Yimin Liu, Tianyao Huang, Yuxiang Lu, Xiqin Wang

The concept of cognitive radar (CR) enables radar systems to achieve intelligent adaption to a changeable environment with feedback facility from receiver to transmitter.

reinforcement-learning Reinforcement Learning (RL)

Star-Transformer

2 code implementations NAACL 2019 Qipeng Guo, Xipeng Qiu, PengFei Liu, Yunfan Shao, xiangyang xue, Zheng Zhang

Although Transformer has achieved great successes on many NLP tasks, its heavy structure with fully-connected attention connections leads to dependencies on large training data.

Named Entity Recognition (NER) Natural Language Inference +2

Drug cell line interaction prediction

1 code implementation28 Dec 2018 Pengfei Liu

Understanding the phenotypic drug response on cancer cell lines plays a vital rule in anti-cancer drug discovery and re-purposing.

Drug Discovery

Multi-task Learning over Graph Structures

no code implementations26 Nov 2018 Pengfei Liu, Jie Fu, Yue Dong, Xipeng Qiu, Jackie Chi Kit Cheung

We present two architectures for multi-task learning with neural sequence models.

General Classification Multi-Task Learning +2

Contextualized Non-local Neural Networks for Sequence Learning

no code implementations21 Nov 2018 Pengfei Liu, Shuaichen Chang, Xuanjing Huang, Jian Tang, Jackie Chi Kit Cheung

Recently, a large number of neural mechanisms and models have been proposed for sequence learning, of which self-attention, as exemplified by the Transformer model, and graph neural networks (GNNs) have attracted much attention.

General Classification Sentence +2

Meta-Learning Multi-task Communication

no code implementations23 Oct 2018 Pengfei Liu, Xuanjing Huang

In this paper, we describe a general framework: Parameters Read-Write Networks (PRaWNs) to systematically analyze current neural models for multi-task learning, in which we find that existing models expect to disentangle features into different spaces while features learned in practice are still entangled in shared space, leaving potential hazards for other training or unseen tasks.

Inductive Bias Meta-Learning +1

Meta Multi-Task Learning for Sequence Modeling

no code implementations25 Feb 2018 Junkun Chen, Xipeng Qiu, Pengfei Liu, Xuanjing Huang

Specifically, we use a shared meta-network to capture the meta-knowledge of semantic composition and generate the parameters of the task-specific semantic composition models.

Multi-Task Learning Representation Learning +3

Idiom-Aware Compositional Distributed Semantics

no code implementations EMNLP 2017 Pengfei Liu, Kaiyu Qian, Xipeng Qiu, Xuanjing Huang

Idioms are peculiar linguistic constructions that impose great challenges for representing the semantics of language, especially in current prevailing end-to-end neural models, which assume that the semantics of a phrase or sentence can be literally composed from its constitutive words.

General Classification Machine Translation +4

Dynamic Compositional Neural Networks over Tree Structure

no code implementations11 May 2017 Pengfei Liu, Xipeng Qiu, Xuanjing Huang

Tree-structured neural networks have proven to be effective in learning semantic representations by exploiting syntactic information.

Learning Semantic Representations

Adversarial Multi-task Learning for Text Classification

no code implementations ACL 2017 Pengfei Liu, Xipeng Qiu, Xuanjing Huang

Neural network models have shown their promising opportunities for multi-task learning, which focus on learning the shared layers to extract the common and task-invariant features.

General Classification Multi-Task Learning +2

Deep Multi-Task Learning with Shared Memory

no code implementations23 Sep 2016 Pengfei Liu, Xipeng Qiu, Xuanjing Huang

Neural network based models have achieved impressive results on various specific tasks.

General Classification Multi-Task Learning +2

Syntax-based Attention Model for Natural Language Inference

no code implementations22 Jul 2016 PengFei Liu, Xipeng Qiu, Xuanjing Huang

Introducing attentional mechanism in neural network is a powerful concept, and has achieved impressive results in many natural language processing tasks.

Natural Language Inference Sentence

Modelling Interaction of Sentence Pair with coupled-LSTMs

no code implementations EMNLP 2016 Pengfei Liu, Xipeng Qiu, Xuanjing Huang

Recently, there is rising interest in modelling the interactions of two sentences with deep neural networks.

Sentence

Cannot find the paper you are looking for? You can Submit a new open access paper.