Search Results for author: Shengqiang Zhang

Found 4 papers, 2 papers with code

Attention Temperature Matters in Abstractive Summarization Distillation

1 code implementation ACL 2022 Shengqiang Zhang, Xingxing Zhang, Hangbo Bao, Furu Wei

In this paper, we find simply manipulating attention temperatures in Transformers can make pseudo labels easier to learn for student models.

Abstractive Text Summarization

Pre-trained Language Model based Ranking in Baidu Search

no code implementations24 May 2021 Lixin Zou, Shengqiang Zhang, Hengyi Cai, Dehong Ma, Suqi Cheng, Daiting Shi, Zhifan Zhu, Weiyue Su, Shuaiqiang Wang, Zhicong Cheng, Dawei Yin

However, it is nontrivial to directly apply these PLM-based rankers to the large-scale web search system due to the following challenging issues:(1) the prohibitively expensive computations of massive neural PLMs, especially for long texts in the web-document, prohibit their deployments in an online ranking system that demands extremely low latency;(2) the discrepancy between existing ranking-agnostic pre-training objectives and the ad-hoc retrieval scenarios that demand comprehensive relevance modeling is another main barrier for improving the online ranking system;(3) a real-world search engine typically involves a committee of ranking components, and thus the compatibility of the individually fine-tuned ranking model is critical for a cooperative ranking system.

Language Modelling Retrieval

Cannot find the paper you are looking for? You can Submit a new open access paper.