no code implementations • CCL 2020 • Zhengwei Lv, Lei Yang, Zhizhong Shi, Xiao Liang, Tao Lei, Duoxing Liu
阅读理解问答系统是利用语义理解等自然语言处理技术, 根据输入问题, 对非结构化文档数据进行分析, 生成一个答案, 具有很高的研究和应用价值。在垂直领域应用过程中, 阅读理解问答数据标注成本高且用户问题表达复杂多样, 使得阅读理解问答系统准确率低、鲁棒性差。针对这一问题, 本文提出一种面向垂直领域的阅读理解问答数据的增强方法, 该方法基于真实用户问题, 构造阅读理解训练数据, 一方面降低标注成本, 另一方面增加训练数据多样性, 提升模型的准确率和鲁棒性。本文用汽车领域数据对该方法进行实验验证, 其结果表明该方法对垂直领域阅读理解模型的准确率和鲁棒性均能有效提升。
no code implementations • 17 Mar 2023 • Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but also from applying feedforward and projection layers to every token.
Ranked #1 on
Long-range modeling
on SCROLLS
no code implementations • 4 Dec 2022 • Shu Liu, Enquan Huang, Yan Xu, Kexuan Wang, Xiaoyan Kui, Tao Lei, Hongying Meng
To make the best use of the dataset, the manual ratings, attractiveness score, and standard deviation are aggregated explicitly to construct a dual label distribution, including the attractiveness distribution and the rating distribution.
no code implementations • 2 Nov 2022 • Yujie Qian, Jinhyuk Lee, Sai Meher Karthik Duddu, Zhuyun Dai, Siddhartha Brahma, Iftekhar Naim, Tao Lei, Vincent Y. Zhao
With sparsified unary saliences, we are able to prune a large number of query and document token vectors and improve the efficiency of multi-vector retrieval.
1 code implementation • 25 May 2022 • Zexuan Zhong, Tao Lei, Danqi Chen
Recent work has improved language models (LMs) remarkably by equipping them with a non-parametric memory component.
no code implementations • 23 May 2022 • Tao Lei, Ran Tian, Jasmijn Bastings, Ankur P. Parikh
In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into the Transformer.
no code implementations • 18 Feb 2022 • Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew Dai, Zhifeng Chen, Quoc Le, James Laudon
Prior work allocates a fixed number of experts to each token using a top-k function regardless of the relative importance of different tokens.
no code implementations • 11 Oct 2021 • Jing Pan, Tao Lei, Kwangyoun Kim, Kyu Han, Shinji Watanabe
The Transformer architecture has been well adopted as a dominant architecture in most sequence transduction tasks including automatic speech recognition (ASR), since its attention mechanism excels in capturing long-range dependencies.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
no code implementations • 17 Aug 2021 • Xianyuan Liu, Shuo Zhou, Tao Lei, Haiping Lu
Finally, we propose a Channel-Temporal Attention Network (CTAN) to integrate these blocks into existing architectures.
no code implementations • 22 Jun 2021 • Xianyuan Liu, Raivo Koot, Shuo Zhou, Tao Lei, Haiping Lu
Under the team name xy9, our submission achieved 5th place in terms of top-1 accuracy for verb class and all top-5 accuracies.
no code implementations • NAACL 2021 • Darsh Shah, Lili Yu, Tao Lei, Regina Barzilay
We present a method for generating comparative summaries that highlight similarities and contradictions in input documents.
2 code implementations • 8 Apr 2021 • Darsh J Shah, Lili Yu, Tao Lei, Regina Barzilay
We present a method for generating comparative summaries that highlights similarities and contradictions in input documents.
1 code implementation • 22 Mar 2021 • Darsh J Shah, Lili Yu, Tao Lei, Regina Barzilay
We introduce \emph{Nutri-bullets}, a multi-document summarization task for health and nutrition.
1 code implementation • EMNLP 2021 • Tao Lei
In this work, we present SRU++, a highly-efficient architecture that combines fast recurrence and attention for sequence modeling.
Ranked #3 on
Language Modelling
on enwik8
2 code implementations • 28 Sep 2020 • Risheng Wang, Tao Lei, Ruixia Cui, Bingtao Zhang, Hongy-ing Meng, Asoke K. Nandi
Firstly, compared to traditional surveys that directly divide literatures of deep learning on medical image segmentation into many groups and introduce literatures in detail for each group, we classify currently popular literatures according to a multi-level structure from coarse to fine.
1 code implementation • EMNLP 2020 • Alexander Lin, Jeremy Wohlwend, Howard Chen, Tao Lei
The performance of autoregressive models on natural language generation tasks has dramatically improved due to the adoption of deep, self-attentive architectures.
Ranked #20 on
Machine Translation
on IWSLT2014 German-English
1 code implementation • ACL 2020 • Kyle Swanson, Lili Yu, Tao Lei
Selecting input features of top relevance has become a popular method for building self-explaining models.
no code implementations • 21 May 2020 • Jing Pan, Joshua Shapiro, Jeremy Wohlwend, Kyu J. Han, Tao Lei, Tao Ma
In this paper we present state-of-the-art (SOTA) performance on the LibriSpeech corpus with two novel neural network architectures, a multistream CNN for acoustic modeling and a self-attentive simple recurrent unit (SRU) for language modeling.
Ranked #6 on
Speech Recognition
on LibriSpeech test-clean
1 code implementation • ACL 2020 • Lili Yu, Howard Chen, Sida Wang, Tao Lei, Yoav Artzi
We study the potential for interaction in natural language classification.
1 code implementation • WS 2019 • Jeremy Wohlwend, Ethan R. Elenberg, Samuel Altschul, Shawn Henry, Tao Lei
However, in many real-world applications the label set is frequently changing.
2 code implementations • EMNLP 2020 • Ziheng Wang, Jeremy Wohlwend, Tao Lei
Large language models have recently achieved state of the art performance across a wide variety of natural language tasks.
no code implementations • WS 2019 • Kyle Swanson, Lili Yu, Christopher Fox, Jeremy Wohlwend, Tao Lei
Response suggestion is an important task for building human-computer conversation systems.
no code implementations • SEMEVAL 2019 • Zhengwei Lv, Duoxing Liu, Haifeng Sun, Xiao Liang, Tao Lei, Zhizhong Shi, Feng Zhu, Lei Yang
In order to address this task, we propose a system based on the BERT model with meta information of questions.
1 code implementation • 8 Apr 2019 • Tao Lei, Xiaohong Jia, Tongliang Liu, Shigang Liu, Hongy-ing Meng, Asoke K. Nandi
However, MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed.
1 code implementation • EMNLP 2018 • Darsh J Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, Preslav Nakov
We address the problem of detecting duplicate questions in forums, which is an important step towards automating the process of answering new questions.
no code implementations • IEEE 2018 • Tao Lei, Xiaohong Jia, Yanning Zhang, Lifeng He, Hongy-ing Meng, Senior Member, and Asoke K. Nandi, Fellow, IEEE
However, the introduction oflocal spatial information often leads to a high computationalcomplexity, arising out of an iterative calculation of the distancebetween pixels within local spatial neighbors and clusteringcenters.
1 code implementation • ICLR 2018 • Tao Lei, Yu Zhang, Yoav Artzi
Common recurrent neural network architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
11 code implementations • EMNLP 2018 • Tao Lei, Yu Zhang, Sida I. Wang, Hui Dai, Yoav Artzi
Common recurrent neural architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.
Ranked #32 on
Question Answering
on SQuAD1.1 dev
12 code implementations • NeurIPS 2017 • Tianxiao Shen, Tao Lei, Regina Barzilay, Tommi Jaakkola
We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.
Ranked #7 on
Text Style Transfer
on Yelp Review Dataset (Small)
no code implementations • ICML 2017 • Tao Lei, Wengong Jin, Regina Barzilay, Tommi Jaakkola
The design of neural architectures for structured objects is typically guided by experimental insights rather than a formal process.
3 code implementations • EMNLP 2016 • Tao Lei, Regina Barzilay, Tommi Jaakkola
Our approach combines two modular components, generator and encoder, which are trained to operate well together.
1 code implementation • NAACL 2016 • Tao Lei, Hrishikesh Joshi, Regina Barzilay, Tommi Jaakkola, Katerina Tymoshenko, Alessandro Moschitti, Lluis Marquez
Question answering forums are rapidly growing in size with no effective automated ability to refer to and reuse answers already available for previous posted questions.
2 code implementations • EMNLP 2015 • Tao Lei, Regina Barzilay, Tommi Jaakkola
Moreover, we extend the n-gram convolution to non-consecutive words to recognize patterns with intervening words.
no code implementations • TACL 2014 • Yonatan Belinkov, Tao Lei, Regina Barzilay, Amir Globerson
In this paper, we show that word vector representations can yield significant PP attachment performance gains.