Search Results for author: Bowen Yu

Found 66 papers, 40 papers with code

Maximal Clique Based Non-Autoregressive Open Information Extraction

no code implementations EMNLP 2021 Bowen Yu, Yucheng Wang, Tingwen Liu, Hongsong Zhu, Limin Sun, Bin Wang

However, the popular OpenIE systems usually output facts sequentially in the way of predicting the next fact conditioned on the previous decoded ones, which enforce an unnecessary order on the facts and involve the error accumulation between autoregressive steps.

Open Information Extraction Sentence

Enhancing Chinese Pre-trained Language Model via Heterogeneous Linguistics Graph

3 code implementations ACL 2022 Yanzeng Li, Jiangxia Cao, Xin Cong, Zhenyu Zhang, Bowen Yu, Hongsong Zhu, Tingwen Liu

Chinese pre-trained language models usually exploit contextual character information to learn representations, while ignoring the linguistics knowledge, e. g., word and sentence information.

Language Modelling Sentence

Transferable Post-training via Inverse Value Learning

no code implementations28 Oct 2024 Xinyu Lu, Xueru Wen, Yaojie Lu, Bowen Yu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han, Yongbin Li

After training this network on a small base model using demonstrations, this network can be seamlessly integrated with other pre-trained models during inference, enables them to achieve similar capability enhancements.

Aligning Large Language Models via Self-Steering Optimization

1 code implementation22 Oct 2024 Hao Xiang, Bowen Yu, Hongyu Lin, Keming Lu, Yaojie Lu, Xianpei Han, Le Sun, Jingren Zhou, Junyang Lin

The key to automated alignment lies in providing learnable and accurate preference signals for preference learning without human annotation.

A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models

no code implementations17 Oct 2024 Qiaoyu Tang, Le Yu, Bowen Yu, Hongyu Lin, Keming Lu, Yaojie Lu, Xianpei Han, Le Sun

Post-training has emerged as a crucial paradigm for adapting large-scale pre-trained models to various tasks, whose effects are fully reflected by delta parameters (i. e., the disparity between post-trained and pre-trained parameters).

Quantization

Rethinking Data Selection at Scale: Random Selection is Almost All You Need

1 code implementation12 Oct 2024 Tingyu Xia, Bowen Yu, Kai Dang, An Yang, Yuan Wu, Yuan Tian, Yi Chang, Junyang Lin

Supervised fine-tuning (SFT) is crucial for aligning Large Language Models (LLMs) with human instructions.

Self-play with Execution Feedback: Improving Instruction-following Capabilities of Large Language Models

1 code implementation19 Jun 2024 Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, Jingren Zhou

AutoIF transforms the validation of instruction-following data quality into code verification, requiring LLMs to generate instructions, the corresponding code to check the correctness of the instruction responses, and unit test samples to verify the code's correctness.

Instruction Following

Towards Scalable Automated Alignment of LLMs: A Survey

1 code implementation3 Jun 2024 Boxi Cao, Keming Lu, Xinyu Lu, Jiawei Chen, Mengjie Ren, Hao Xiang, Peilin Liu, Yaojie Lu, Ben He, Xianpei Han, Le Sun, Hongyu Lin, Bowen Yu

Alignment is the most critical step in building large language models (LLMs) that meet human needs.

Survey

Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment

1 code implementation28 May 2024 Keming Lu, Bowen Yu, Fei Huang, Yang Fan, Runji Lin, Chang Zhou

Effectively aligning Large Language Models (LLMs) with human-centric values while preventing the degradation of abilities acquired through Pre-training and Supervised Fine-tuning (SFT) poses a central challenge in Reinforcement Learning from Human Feedback (RLHF).

Language Models can Evaluate Themselves via Probability Discrepancy

1 code implementation17 May 2024 Tingyu Xia, Bowen Yu, Yuan Wu, Yi Chang, Chang Zhou

In this paper, we initiate our discussion by demonstrating how Large Language Models (LLMs), when tasked with responding to queries, display a more even probability distribution in their answers if they are more adept, as opposed to their less skilled counterparts.

Text Generation

Scaling Data Diversity for Fine-Tuning Language Models in Human Alignment

1 code implementation17 Mar 2024 Feifan Song, Bowen Yu, Hao Lang, Haiyang Yu, Fei Huang, Houfeng Wang, Yongbin Li

Additionally, the concept of diversity for prompts can be more complex than responses that are typically quantified by single digits.

Data Augmentation Diversity

SoFA: Shielded On-the-fly Alignment via Priority Rule Following

1 code implementation27 Feb 2024 Xinyu Lu, Bowen Yu, Yaojie Lu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han, Yongbin Li

The alignment problem in Large Language Models (LLMs) involves adapting them to the broad spectrum of human values.

Diversity

Self-Retrieval: Building an Information Retrieval System with One Large Language Model

no code implementations23 Feb 2024 Qiaoyu Tang, Jiawei Chen, Bowen Yu, Yaojie Lu, Cheng Fu, Haiyang Yu, Hongyu Lin, Fei Huang, Ben He, Xianpei Han, Le Sun, Yongbin Li

The rise of large language models (LLMs) has transformed the role of information retrieval (IR) systems in the way to humans accessing information.

Information Retrieval Language Modelling +2

Large Language Models are Superpositions of All Characters: Attaining Arbitrary Role-play via Self-Alignment

1 code implementation23 Jan 2024 Keming Lu, Bowen Yu, Chang Zhou, Jingren Zhou

Nevertheless, we posit that LLMs inherently harbor role-play capabilities, owing to the extensive knowledge of characters and potential dialogues ingrained in their vast training corpora.

Instruction Following Reading Comprehension

Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch

3 code implementations6 Nov 2023 Le Yu, Bowen Yu, Haiyang Yu, Fei Huang, Yongbin Li

We experiment with encoder- and decoder-based LMs, showing that: (1) SFT delta parameter value ranges are typically small (within 0. 002) with extreme redundancy, and DARE can effortlessly eliminate 90% or even 99% of them; (2) DARE can merge multiple task-specific LMs into one LM with diverse capabilities.

Decoder GSM8K +1

Diversify Question Generation with Retrieval-Augmented Style Transfer

1 code implementation23 Oct 2023 Qi Gou, Zehua Xia, Bowen Yu, Haiyang Yu, Fei Huang, Yongbin Li, Nguyen Cam-Tu

Given a textual passage and an answer, humans are able to ask questions with various expressions, but this ability is still challenging for most question generation (QG) systems.

Diversity Question Answering +5

Improving Question Generation with Multi-level Content Planning

1 code implementation20 Oct 2023 Zehua Xia, Qi Gou, Bowen Yu, Haiyang Yu, Fei Huang, Yongbin Li, Cam-Tu Nguyen

Previous studies have suggested that key phrase selection is essential for question generation (QG), yet it is still challenging to connect such disjointed phrases into meaningful questions, particularly for long context.

Answer Generation Question Generation +2

Quantifying and mitigating the impact of label errors on model disparity metrics

no code implementations4 Oct 2023 Julius Adebayo, Melissa Hall, Bowen Yu, Bobbie Chern

We empirically assess the proposed approach on a variety of datasets and find significant improvement, compared to alternative approaches, in identifying training inputs that improve a model's disparity metric.

A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment

1 code implementation10 Aug 2023 Yingxiu Zhao, Bowen Yu, Binyuan Hui, Haiyang Yu, Fei Huang, Yongbin Li, Nevin L. Zhang

Training large language models (LLMs) with open-domain instruction data has yielded remarkable success in aligning to end tasks and human preferences.

Wider and Deeper LLM Networks are Fairer LLM Evaluators

1 code implementation3 Aug 2023 Xinghua Zhang, Bowen Yu, Haiyang Yu, Yangyu Lv, Tingwen Liu, Fei Huang, Hongbo Xu, Yongbin Li

Each perspective corresponds to the role of a specific LLM neuron in the first layer.

Preference Ranking Optimization for Human Alignment

1 code implementation30 Jun 2023 Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, Houfeng Wang

In this manner, PRO effectively transforms human alignment into aligning the probability ranking of n responses generated by LLM with the preference ranking of humans towards these responses.

Unified Language Representation for Question Answering over Text, Tables, and Images

no code implementations29 Jun 2023 Bowen Yu, Cheng Fu, Haiyang Yu, Fei Huang, Yongbin Li

When trying to answer complex questions, people often rely on multiple sources of information, such as visual, textual, and tabular data.

Question Answering Retrieval

Causal Document-Grounded Dialogue Pre-training

1 code implementation18 May 2023 Yingxiu Zhao, Bowen Yu, Haiyang Yu, Bowen Li, Jinyang Li, Chao Wang, Fei Huang, Yongbin Li, Nevin L. Zhang

To tackle this issue, we are the first to present a causally-complete dataset construction strategy for building million-level DocGD pre-training corpora.

Domain Incremental Lifelong Learning in an Open World

1 code implementation11 May 2023 Yi Dai, Hao Lang, Yinhe Zheng, Bowen Yu, Fei Huang, Yongbin Li

Specifically, we dedicate task-level prompts to capture task-specific knowledge to retain high LL performances and maintain instance-level prompts to learn knowledge shared across input samples to improve the model's generalization performance.

Language Modelling

Towards Generalized Open Information Extraction

no code implementations29 Nov 2022 Bowen Yu, Zhenyu Zhang, Jingyang Li, Haiyang Yu, Tingwen Liu, Jian Sun, Yongbin Li, Bin Wang

Open Information Extraction (OpenIE) facilitates the open-domain discovery of textual facts.

Open Information Extraction

Semi-Supervised Lifelong Language Learning

1 code implementation23 Nov 2022 Yingxiu Zhao, Yinhe Zheng, Bowen Yu, Zhiliang Tian, Dongkyu Lee, Jian Sun, Haiyang Yu, Yongbin Li, Nevin L. Zhang

In this paper, we explore a novel setting, semi-supervised lifelong language learning (SSLL), where a model learns sequentially arriving language tasks with both labeled and unlabeled data.

Transfer Learning

Layout-Aware Information Extraction for Document-Grounded Dialogue: Dataset, Method and Demonstration

no code implementations14 Jul 2022 Zhenyu Zhang, Bowen Yu, Haiyang Yu, Tingwen Liu, Cheng Fu, Jingyang Li, Chengguang Tang, Jian Sun, Yongbin Li

In this paper, we propose a Layout-aware document-level Information Extraction dataset, LIE, to facilitate the study of extracting both structural and semantic knowledge from visually rich documents (VRDs), so as to generate accurate responses in dialogue systems.

Language Modelling

Relation-Guided Few-Shot Relational Triple Extraction

1 code implementation SIGIR 2022 Xin Cong, Jiawei Sheng, Shiyao Cui, Bowen Yu, Tingwen Liu, Bin Wang

To instantiate this strategy, we further propose a model, RelATE, which builds a dual-level attention to aggregate relationrelevant information to detect the relation occurrence and utilizes the annotated samples of the detected relations to extract the corresponding head/tail entities.

Relation RTE +1

Profiling and Evolution of Intellectual Property

no code implementations20 Apr 2022 Bowen Yu, Yingxia Shao, Ang Li

In recent years, with the rapid growth of Internet data, the number and types of scientific and technological resources are also rapidly expanding.

Retrieval

Web Page Content Extraction Based on Multi-feature Fusion

no code implementations21 Mar 2022 Bowen Yu, Junping Du, Yingxia Shao

With the rapid growth of the number and types of web resources, there are still problems to be solved when using a single strategy to extract the text information of different pages.

Document-Level Event Extraction via Human-Like Reading Process

no code implementations7 Feb 2022 Shiyao Cui, Xin Cong, Bowen Yu, Tingwen Liu, Yucheng Wang, Jinqiao Shi

Meanwhile, rough reading is explored in a multi-round manner to discover undetected events, thus the multi-events problem is handled.

Document-level Event Extraction Event Extraction

Improving Distantly-Supervised Named Entity Recognition with Self-Collaborative Denoising Learning

1 code implementation EMNLP 2021 Xinghua Zhang, Bowen Yu, Tingwen Liu, Zhenyu Zhang, Jiawei Sheng, Mengge Xue, Hongbo Xu

Distantly supervised named entity recognition (DS-NER) efficiently reduces labor costs but meanwhile intrinsically suffers from the label noise due to the strong assumption of distant supervision.

Denoising named-entity-recognition +2

Label Enhanced Event Detection with Heterogeneous Graph Attention Networks

no code implementations3 Dec 2020 Shiyao Cui, Bowen Yu, Xin Cong, Tingwen Liu, Quangang Li, Jinqiao Shi

A heterogeneous graph attention networks is then introduced to propagate relational message and enrich information interaction.

Event Detection Graph Attention +1

Document-level Relation Extraction with Dual-tier Heterogeneous Graph

no code implementations COLING 2020 Zhenyu Zhang, Bowen Yu, Xiaobo Shu, Tingwen Liu, Hengzhu Tang, Wang Yubin, Li Guo

Document-level relation extraction (RE) poses new challenges over its sentence-level counterpart since it requires an adequate comprehension of the whole document and the multi-hop reasoning ability across multiple sentences to reach the final result.

Decision Making Document-level Relation Extraction +2

Porous Lattice Transformer Encoder for Chinese NER

no code implementations COLING 2020 Xue Mengge, Bowen Yu, Tingwen Liu, Yue Zhang, Erli Meng, Bin Wang

Incorporating lexicons into character-level Chinese NER by lattices is proven effective to exploitrich word boundary information.

NER

TPLinker: Single-stage Joint Extraction of Entities and Relations Through Token Pair Linking

1 code implementation COLING 2020 Yucheng Wang, Bowen Yu, Yueyang Zhang, Tingwen Liu, Hongsong Zhu, Limin Sun

To mitigate the issue, we propose in this paper a one-stage joint extraction model, namely, TPLinker, which is capable of discovering overlapping relations sharing one or both entities while immune from the exposure bias.

Relation Relation Extraction

Inductive Unsupervised Domain Adaptation for Few-Shot Classification via Clustering

1 code implementation23 Jun 2020 Xin Cong, Bowen Yu, Tingwen Liu, Shiyao Cui, Hengzhu Tang, Bin Wang

We first build a representation extractor to derive features for unlabeled data from the target domain (no test data is necessary) and then group them with a cluster miner.

Classification Clustering +2

Enhancing Pre-trained Chinese Character Representation with Word-aligned Attention

1 code implementation ACL 2020 Yanzeng Li, Bowen Yu, Mengge Xue, Tingwen Liu

Most Chinese pre-trained models take character as the basic unit and learn representation according to character's external contexts, ignoring the semantics expressed in the word, which is the smallest meaningful utterance in Chinese.

Joint Extraction of Entities and Relations Based on a Novel Decomposition Strategy

1 code implementation10 Sep 2019 Bowen Yu, Zhen-Yu Zhang, Xiaobo Shu, Yubin Wang, Tingwen Liu, Bin Wang, Sujian Li

Joint extraction of entities and relations aims to detect entity pairs along with their relations using a single model.

Relation Extraction

FlowSense: A Natural Language Interface for Visual Data Exploration within a Dataflow System

1 code implementation2 Aug 2019 Bowen Yu, Claudio T. Silva

Dataflow visualization systems enable flexible visual data exploration by allowing the user to construct a dataflow diagram that composes query and visualization modules to specify system functionality.

Machine Translation

Visus: An Interactive System for Automatic Machine Learning Model Building and Curation

no code implementations5 Jul 2019 Aécio Santos, Sonia Castelo, Cristian Felix, Jorge Piazentin Ono, Bowen Yu, Sungsoo Hong, Cláudio T. Silva, Enrico Bertini, Juliana Freire

In this paper, we present Visus, a system designed to support the model building process and curation of ML data processing pipelines generated by AutoML systems.

AutoML BIG-bench Machine Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.