Search Results for author: Zewen Chi

Found 27 papers, 17 papers with code

FlashBack:Efficient Retrieval-Augmented Language Modeling for Long Context Inference

no code implementations7 May 2024 Runheng Liu, Xingchen Xiao, Heyan Huang, Zewen Chi, Zhijing Wu

And the inference speed of FlashBack is up to $4\times$ faster than the prepending counterpart on a 7B LLM (Llama 2) in the runtime test.

Language Modelling Retrieval

Measuring Cross-Lingual Transferability of Multilingual Transformers on Sentence Classification

no code implementations15 May 2023 Zewen Chi, Heyan Huang, Xian-Ling Mao

Recent studies have exhibited remarkable capabilities of pre-trained multilingual Transformers, especially cross-lingual transferability.

Cross-Lingual Transfer Sentence +1

Optimizing Prompts for Text-to-Image Generation

3 code implementations NeurIPS 2023 Yaru Hao, Zewen Chi, Li Dong, Furu Wei

Instead of laborious human engineering, we propose prompt adaptation, a general framework that automatically adapts original user input to model-preferred prompts.

Language Modelling Prompt Engineering +2

Beyond English-Centric Bitexts for Better Multilingual Language Representation Learning

no code implementations26 Oct 2022 Barun Patra, Saksham Singhal, Shaohan Huang, Zewen Chi, Li Dong, Furu Wei, Vishrav Chaudhary, Xia Song

In this paper, we elaborate upon recipes for building multilingual representation models that are not only competitive with existing state-of-the-art models but are also more parameter efficient, thereby promoting better adoption in resource-constrained scenarios and practical applications.

Representation Learning

ET5: A Novel End-to-end Framework for Conversational Machine Reading Comprehension

1 code implementation COLING 2022 Xiao Zhang, Heyan Huang, Zewen Chi, Xian-Ling Mao

Conversational machine reading comprehension (CMRC) aims to assist computers to understand an natural language text and thereafter engage in a multi-turn conversation to answer questions related to the text.

Decision Making Machine Reading Comprehension

Unsupervised Question Answering via Answer Diversifying

1 code implementation COLING 2022 Yuxiang Nie, Heyan Huang, Zewen Chi, Xian-Ling Mao

Previous works usually make use of heuristic rules as well as pre-trained models to construct data and train QA models.

Data Augmentation Denoising +5

Language Models are General-Purpose Interfaces

1 code implementation13 Jun 2022 Yaru Hao, Haoyu Song, Li Dong, Shaohan Huang, Zewen Chi, Wenhui Wang, Shuming Ma, Furu Wei

Experimental results across various language-only and vision-language benchmarks show that our model outperforms or is competitive with specialized models on finetuning, zero-shot generalization, and few-shot learning.

Causal Language Modeling Few-Shot Learning +6

Cross-Lingual Phrase Retrieval

1 code implementation ACL 2022 Heqi Zheng, Xiao Zhang, Zewen Chi, Heyan Huang, Tan Yan, Tian Lan, Wei Wei, Xian-Ling Mao

In this paper, we propose XPR, a cross-lingual phrase retriever that extracts phrase representations from unlabeled example sentences.

Retrieval Sentence

Unifying Cross-lingual Summarization and Machine Translation with Compression Rate

1 code implementation15 Oct 2021 Yu Bai, Heyan Huang, Kai Fan, Yang Gao, Yiming Zhu, Jiaao Zhan, Zewen Chi, Boxing Chen

Through introducing compression rate, the information ratio between the source and the target text, we regard the MT task as a special CLS task with a compression rate of 100%.

Data Augmentation Machine Translation +1

Cross-Lingual Language Model Meta-Pretraining

no code implementations23 Sep 2021 Zewen Chi, Heyan Huang, Luyang Liu, Yu Bai, Xian-Ling Mao

The success of pretrained cross-lingual language models relies on two essential abilities, i. e., generalization ability for learning downstream tasks in a source language, and cross-lingual transferability for transferring the task knowledge to other languages.

Cross-Lingual Transfer Language Modelling

MT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs

1 code implementation EMNLP 2021 Zewen Chi, Li Dong, Shuming Ma, Shaohan Huang Xian-Ling Mao, Heyan Huang, Furu Wei

Multilingual T5 (mT5) pretrains a sequence-to-sequence model on massive monolingual texts, which has shown promising results on many cross-lingual tasks.

Abstractive Text Summarization Machine Translation +7

A Robust and Domain-Adaptive Approach for Low-Resource Named Entity Recognition

1 code implementation2 Jan 2021 Houjin Yu, Xian-Ling Mao, Zewen Chi, Wei Wei, Heyan Huang

Recently, it has attracted much attention to build reliable named entity recognition (NER) systems using limited annotated data.

Ranked #3 on Named Entity Recognition (NER) on SciERC (using extra training data)

Low Resource Named Entity Recognition named-entity-recognition +2

InfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training

4 code implementations NAACL 2021 Zewen Chi, Li Dong, Furu Wei, Nan Yang, Saksham Singhal, Wenhui Wang, Xia Song, Xian-Ling Mao, He-Yan Huang, Ming Zhou

In this work, we present an information-theoretic framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts.

Contrastive Learning Cross-Lingual Transfer +2

Generating Informative Dialogue Responses with Keywords-Guided Networks

no code implementations3 Jul 2020 Heng-Da Xu, Xian-Ling Mao, Zewen Chi, Jing-Jing Zhu, Fanshu Sun, He-Yan Huang

Specifically, KW-Seq2Seq first uses a keywords decoder to predict some topic keywords, and then generates the final response under the guidance of them.

Decoder

Cross-Lingual Natural Language Generation via Pre-Training

1 code implementation23 Sep 2019 Zewen Chi, Li Dong, Furu Wei, Wenhui Wang, Xian-Ling Mao, He-Yan Huang

In this work we focus on transferring supervision signals of natural language generation (NLG) tasks between multiple languages.

Abstractive Text Summarization Decoder +6

Complicated Table Structure Recognition

1 code implementation13 Aug 2019 Zewen Chi, He-Yan Huang, Heng-Da Xu, Houjin Yu, Wanxuan Yin, Xian-Ling Mao

It also attracts lots of attention to recognize the table structures in PDF files.

Graph Neural Network

Zewen at SemEval-2018 Task 1: An Ensemble Model for Affect Prediction in Tweets

no code implementations SEMEVAL 2018 Zewen Chi, He-Yan Huang, Jiangui Chen, Hao Wu, Ran Wei

This paper presents a method for Affect in Tweets, which is the task to automatically determine the intensity of emotions and intensity of sentiment of tweets.

Sentence Classification Sentiment Analysis

Cannot find the paper you are looking for? You can Submit a new open access paper.