Search Results for author: Juntao Li

Found 36 papers, 18 papers with code

Plan-CVAE: A Planning-based Conditional Variational Autoencoder for Story Generation

no code implementations CCL 2020 Lin Wang, Juntao Li, Rui Yan, Dongyan Zhao

Story generation is a challenging task of automatically creating natural languages to describe a sequence of events, which requires outputting text with not only a consistent topic but also novel wordings.

Story Generation

OpenBA: An Open-sourced 15B Bilingual Asymmetric seq2seq Model Pre-trained from Scratch

1 code implementation19 Sep 2023 Juntao Li, Zecheng Tang, Yuyang Ding, Pinzheng Wang, Pei Guo, Wangjie You, Dan Qiao, Wenliang Chen, Guohong Fu, Qiaoming Zhu, Guodong Zhou, Min Zhang

This report provides the main details to pre-train an analogous model, including pre-training data processing, Bilingual Flan data collection, the empirical observations that inspire our model architecture design, training objectives of different stages, and other enhancement techniques.

LayoutNUWA: Revealing the Hidden Layout Expertise of Large Language Models

1 code implementation18 Sep 2023 Zecheng Tang, Chenfei Wu, Juntao Li, Nan Duan

Graphic layout generation, a growing research field, plays a significant role in user engagement and information perception.

Code Completion Code Generation

Harnessing the Power of David against Goliath: Exploring Instruction Data Generation without Using Closed-Source Models

no code implementations24 Aug 2023 Yue Wang, Xinrui Wang, Juntao Li, Jinxiong Chang, Qishen Zhang, Zhongyi Liu, Guannan Zhang, Min Zhang

Instruction tuning is instrumental in enabling Large Language Models~(LLMs) to follow user instructions to complete various open-domain tasks.

GameEval: Evaluating LLMs on Conversational Games

1 code implementation19 Aug 2023 Dan Qiao, Chenfei Wu, Yaobo Liang, Juntao Li, Nan Duan

In this paper, we propose GameEval, a novel approach to evaluating LLMs through goal-driven conversational games, overcoming the limitations of previous methods.

Question Answering

Detoxify Language Model Step-by-Step

1 code implementation16 Aug 2023 Zecheng Tang, Keyan Zhou, Pinzheng Wang, Yuyang Ding, Juntao Li, Minzhang

Detoxification for LLMs is challenging since it requires models to avoid generating harmful content while maintaining the generation capability.

Language Modelling

Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!

1 code implementation8 May 2023 Zecheng Tang, Pinzheng Wang, Keyan Zhou, Juntao Li, Ziqiang Cao, Min Zhang

Diffusion models have been successfully adapted to text generation tasks by mapping the discrete text into the continuous space.

Text Generation

Test-Time Adaptation with Perturbation Consistency Learning

no code implementations25 Apr 2023 Yi Su, Yixin Ji, Juntao Li, Hai Ye, Min Zhang

Accordingly, in this paper, we propose perturbation consistency learning (PCL), a simple test-time adaptation method to promote the model to make stable predictions for samples with distribution shifts.

Adversarial Robustness Pseudo Label

Gated Mechanism Enhanced Multi-Task Learning for Dialog Routing

no code implementations COLING 2022 Ziming Huang, Zhuoxuan Jiang, Ke Wang, Juntao Li, Shanshan Feng, Xian-Ling Mao

Although most existing methods can fulfil this requirement, they can only model single-source dialog data and cannot effectively capture the underlying knowledge of relations among data and subtasks.

Multi-Task Learning

RenewNAT: Renewing Potential Translation for Non-Autoregressive Transformer

no code implementations14 Mar 2023 Pei Guo, Yisheng Xiao, Juntao Li, Min Zhang

Non-autoregressive neural machine translation (NAT) models are proposed to accelerate the inference process while maintaining relatively high performance.

Machine Translation Translation

AMOM: Adaptive Masking over Masking for Conditional Masked Language Model

1 code implementation13 Mar 2023 Yisheng Xiao, Ruiyang Xu, Lijun Wu, Juntao Li, Tao Qin, Yan-Tie Liu, Min Zhang

Experiments on \textbf{3} different tasks (neural machine translation, summarization, and code generation) with \textbf{15} datasets in total confirm that our proposed simple method achieves significant performance improvement over the strong CMLM model.

Code Generation Language Modelling +2

SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training

1 code implementation COLING 2022 Dan Qiao, Chenchen Dai, Yuyang Ding, Juntao Li, Qiang Chen, Wenliang Chen, Min Zhang

The conventional success of textual classification relies on annotated data, and the new paradigm of pre-trained language models (PLMs) still requires a few labeled data for downstream tasks.

text-classification Text Classification

Chinese grammatical error correction based on knowledge distillation

2 code implementations31 Jul 2022 Peng Xia, Yuechi Zhou, Ziyan Zhang, Zecheng Tang, Juntao Li

In view of the poor robustness of existing Chinese grammatical error correction models on attack test sets and large model parameters, this paper uses the method of knowledge distillation to compress model parameters and improve the anti-attack ability of the model.

Grammatical Error Correction Knowledge Distillation

A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond

1 code implementation20 Apr 2022 Yisheng Xiao, Lijun Wu, Junliang Guo, Juntao Li, Min Zhang, Tao Qin, Tie-Yan Liu

While NAR generation can significantly accelerate inference speed for machine translation, the speedup comes at the cost of sacrificed translation accuracy compared to its counterpart, autoregressive (AR) generation.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +11

Image-text Retrieval: A Survey on Recent Research and Development

no code implementations28 Mar 2022 Min Cao, Shiping Li, Juntao Li, Liqiang Nie, Min Zhang

On top of this, the efficiency-focused study on the ITR system is introduced as the third perspective.

Retrieval Text Retrieval

CT4Rec: Simple yet Effective Consistency Training for Sequential Recommendation

2 code implementations13 Dec 2021 Chong Liu, Xiaoyang Liu, Rongqin Zheng, Lixin Zhang, Xiaobo Liang, Juntao Li, Lijun Wu, Min Zhang, Leyu Lin

State-of-the-art sequential recommendation models proposed very recently combine contrastive learning techniques for obtaining high-quality user representations.

Click-Through Rate Prediction Contrastive Learning +2

Building an Efficient and Effective Retrieval-based Dialogue System via Mutual Learning

no code implementations1 Oct 2021 Chongyang Tao, Jiazhan Feng, Chang Liu, Juntao Li, Xiubo Geng, Daxin Jiang

For this task, the adoption of pre-trained language models (such as BERT) has led to remarkable progress in a number of benchmarks.

Re-Ranking Retrieval

Are BERT Families Zero-Shot Learners? A Study on Their Potential and Limitations

no code implementations29 Sep 2021 Yue Wang, Lijun Wu, Xiaobo Liang, Juntao Li, Min Zhang

Starting from the resurgence of deep learning, language models (LMs) have never been so popular.

DM-CT: Consistency Training with Data and Model Perturbation

no code implementations29 Sep 2021 Xiaobo Liang, Runze Mao, Lijun Wu, Juntao Li, Weiqing Liu, Qing Li, Min Zhang

The common approach of consistency training is performed on the data-level, which typically utilizes the data augmentation strategy (or adversarial training) to make the predictions from the augmented input and the original input to be consistent, so that the model is more robust and attains better generalization ability.

Data Augmentation Image Classification +2

Learning to Organize a Bag of Words into Sentences with Neural Networks: An Empirical Study

no code implementations NAACL 2021 Chongyang Tao, Shen Gao, Juntao Li, Yansong Feng, Dongyan Zhao, Rui Yan

Sequential information, a. k. a., orders, is assumed to be essential for processing a sequence with recurrent neural network or convolutional neural network based encoders.

Dialogue History Matters! Personalized Response Selectionin Multi-turn Retrieval-based Chatbots

no code implementations17 Mar 2021 Juntao Li, Chang Liu, Chongyang Tao, Zhangming Chan, Dongyan Zhao, Min Zhang, Rui Yan

To fill the gap between these up-to-date methods and the real-world applications, we incorporate user-specific dialogue history into the response selection and propose a personalized hybrid matching network (PHMN).

Representation Learning Retrieval

Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model

1 code implementation23 Nov 2020 Juntao Li, Ruidan He, Hai Ye, Hwee Tou Ng, Lidong Bing, Rui Yan

Experimental results show that our proposed method achieves significant performance improvements over the state-of-the-art pretrained cross-lingual language model in the CLCD setting.

Language Modelling Mutual Information Estimation +1

Feature Adaptation of Pre-Trained Language Models across Languages and Domains with Robust Self-Training

2 code implementations EMNLP 2020 Hai Ye, Qingyu Tan, Ruidan He, Juntao Li, Hwee Tou Ng, Lidong Bing

To improve the robustness of self-training, in this paper we present class-aware feature self-distillation (CFd) to learn discriminative features from PrLMs, in which PrLM features are self-distilled into a feature adaptation module and the features from the same class are more tightly clustered.

Text Classification Unsupervised Domain Adaptation

Solution Path Algorithm for Twin Multi-class Support Vector Machine

1 code implementation30 May 2020 Liuyuan Chen, Kanglei Zhou, Junchang Jing, Haiju Fan, Juntao Li

Next, Lagrangian multipliers are proved to be 1 as the regularization parameter approaches infinity, thus, a simple yet effective initialization algorithm is devised.

Binary Classification Classification +2

Cross-Lingual Low-Resource Set-to-Description Retrieval for Global E-Commerce

1 code implementation17 May 2020 Juntao Li, Chang Liu, Jian Wang, Lidong Bing, Hongsong Li, Xiaozhong Liu, Dongyan Zhao, Rui Yan

We manually collect a new and high-quality paired dataset, where each pair contains an unordered product attribute set in the source language and an informative product description in the target language.

Cross-Lingual Information Retrieval Retrieval

Stick to the Facts: Learning towards a Fidelity-oriented E-Commerce Product Description Generation

no code implementations IJCNLP 2019 Zhangming Chan, Xiuying Chen, Yongliang Wang, Juntao Li, Zhiqiang Zhang, Kun Gai, Dongyan Zhao, Rui Yan

Different from other text generation tasks, in product description generation, it is of vital importance to generate faithful descriptions that stick to the product attribute information.

Text Generation

Are Training Samples Correlated? Learning to Generate Dialogue Responses with Multiple References

no code implementations ACL 2019 Lisong Qiu, Juntao Li, Wei Bi, Dongyan Zhao, Rui Yan

Due to its potential applications, open-domain dialogue generation has become popular and achieved remarkable progress in recent years, but sometimes suffers from generic responses.

Dialogue Generation

Cannot find the paper you are looking for? You can Submit a new open access paper.