Search Results for author: Tao Lei

Found 50 papers, 23 papers with code

面向垂直领域的阅读理解数据增强方法(Method for reading comprehension data enhancement in vertical field)

no code implementations CCL 2020 Zhengwei Lv, Lei Yang, Zhizhong Shi, Xiao Liang, Tao Lei, Duoxing Liu

阅读理解问答系统是利用语义理解等自然语言处理技术, 根据输入问题, 对非结构化文档数据进行分析, 生成一个答案, 具有很高的研究和应用价值。在垂直领域应用过程中, 阅读理解问答数据标注成本高且用户问题表达复杂多样, 使得阅读理解问答系统准确率低、鲁棒性差。针对这一问题, 本文提出一种面向垂直领域的阅读理解问答数据的增强方法, 该方法基于真实用户问题, 构造阅读理解训练数据, 一方面降低标注成本, 另一方面增加训练数据多样性, 提升模型的准确率和鲁棒性。本文用汽车领域数据对该方法进行实验验证, 其结果表明该方法对垂直领域阅读理解模型的准确率和鲁棒性均能有效提升。

Reading Comprehension

Learning to Skip for Language Modeling

no code implementations26 Nov 2023 Dewen Zeng, Nan Du, Tao Wang, Yuanzhong Xu, Tao Lei, Zhifeng Chen, Claire Cui

Overparameterized large-scale language models have impressive generalization performance of in-context few-shot learning.

Few-Shot Learning Language Modelling

TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation

1 code implementation7 Jun 2023 Rui Sun, Tao Lei, Weichuan Zhang, Yong Wan, Yong Xia, Asoke K. Nandi

The hybrid architecture of convolution neural networks (CNN) and Transformer has been the most popular method for medical image segmentation.

Image Segmentation Medical Image Segmentation +2

Lightweight Structure-aware Transformer Network for VHR Remote Sensing Image Change Detection

no code implementations3 Jun 2023 Tao Lei, Yetong Xu, Hailong Ning, Zhiyong Lv, Chongdan Min, Yaochu Jin, Asoke K. Nandi

Popular Transformer networks have been successfully applied to remote sensing (RS) image change detection (CD) identifications and achieve better results than most convolutional neural networks (CNNs), but they still suffer from two main problems.

Change Detection

Rethinking the Role of Token Retrieval in Multi-Vector Retrieval

2 code implementations NeurIPS 2023 Jinhyuk Lee, Zhuyun Dai, Sai Meher Karthik Duddu, Tao Lei, Iftekhar Naim, Ming-Wei Chang, Vincent Y. Zhao

Multi-vector retrieval models such as ColBERT [Khattab and Zaharia, 2020] allow token-level interactions between queries and documents, and hence achieve state of the art on many information retrieval benchmarks.

Information Retrieval Retrieval

CoLT5: Faster Long-Range Transformers with Conditional Computation

no code implementations17 Mar 2023 Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai

Many natural language processing tasks benefit from long inputs, but processing long documents with Transformers is expensive -- not only due to quadratic attention complexity but also from applying feedforward and projection layers to every token.

Long-range modeling

Lightweight Facial Attractiveness Prediction Using Dual Label Distribution

1 code implementation4 Dec 2022 Shu Liu, Enquan Huang, Ziyu Zhou, Yan Xu, Xiaoyan Kui, Tao Lei, Hongying Meng

The data processing is simplified to a minimum for a lightweight design, and MobileNetV2 is selected as our backbone.

Multi-Vector Retrieval as Sparse Alignment

no code implementations2 Nov 2022 Yujie Qian, Jinhyuk Lee, Sai Meher Karthik Duddu, Zhuyun Dai, Siddhartha Brahma, Iftekhar Naim, Tao Lei, Vincent Y. Zhao

With sparsified unary saliences, we are able to prune a large number of query and document token vectors and improve the efficiency of multi-vector retrieval.

Argument Retrieval Information Retrieval +1

Training Language Models with Memory Augmentation

1 code implementation25 May 2022 Zexuan Zhong, Tao Lei, Danqi Chen

Recent work has improved language models (LMs) remarkably by equipping them with a non-parametric memory component.

Language Modelling Machine Translation

Simple Recurrence Improves Masked Language Models

no code implementations23 May 2022 Tao Lei, Ran Tian, Jasmijn Bastings, Ankur P. Parikh

In this work, we explore whether modeling recurrence into the Transformer architecture can both be beneficial and efficient, by building an extremely simple recurrent module into the Transformer.

Mixture-of-Experts with Expert Choice Routing

no code implementations18 Feb 2022 Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Zhao, Andrew Dai, Zhifeng Chen, Quoc Le, James Laudon

Prior work allocates a fixed number of experts to each token using a top-k function regardless of the relative importance of different tokens.

SRU++: Pioneering Fast Recurrence with Attention for Speech Recognition

no code implementations11 Oct 2021 Jing Pan, Tao Lei, Kwangyoun Kim, Kyu Han, Shinji Watanabe

The Transformer architecture has been well adopted as a dominant architecture in most sequence transduction tasks including automatic speech recognition (ASR), since its attention mechanism excels in capturing long-range dependencies.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Channel-Temporal Attention for First-Person Video Domain Adaptation

no code implementations17 Aug 2021 Xianyuan Liu, Shuo Zhou, Tao Lei, Haiping Lu

Finally, we propose a Channel-Temporal Attention Network (CTAN) to integrate these blocks into existing architectures.

Action Recognition Unsupervised Domain Adaptation

Nutri-bullets Hybrid: Consensual Multi-document Summarization

no code implementations NAACL 2021 Darsh Shah, Lili Yu, Tao Lei, Regina Barzilay

We present a method for generating comparative summaries that highlight similarities and contradictions in input documents.

Document Summarization Language Modelling +3

Nutribullets Hybrid: Multi-document Health Summarization

2 code implementations8 Apr 2021 Darsh J Shah, Lili Yu, Tao Lei, Regina Barzilay

We present a method for generating comparative summaries that highlights similarities and contradictions in input documents.

Language Modelling Nutrition +1

When Attention Meets Fast Recurrence: Training Language Models with Reduced Compute

1 code implementation EMNLP 2021 Tao Lei

In this work, we present SRU++, a highly-efficient architecture that combines fast recurrence and attention for sequence modeling.

Language Modelling Machine Translation

Medical Image Segmentation Using Deep Learning: A Survey

2 code implementations28 Sep 2020 Risheng Wang, Tao Lei, Ruixia Cui, Bingtao Zhang, Hongy-ing Meng, Asoke K. Nandi

Firstly, compared to traditional surveys that directly divide literatures of deep learning on medical image segmentation into many groups and introduce literatures in detail for each group, we classify currently popular literatures according to a multi-level structure from coarse to fine.

Data Augmentation Image Segmentation +6

Autoregressive Knowledge Distillation through Imitation Learning

2 code implementations EMNLP 2020 Alexander Lin, Jeremy Wohlwend, Howard Chen, Tao Lei

The performance of autoregressive models on natural language generation tasks has dramatically improved due to the adoption of deep, self-attentive architectures.

Imitation Learning Knowledge Distillation +3

Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport

1 code implementation ACL 2020 Kyle Swanson, Lili Yu, Tao Lei

Selecting input features of top relevance has become a popular method for building self-explaining models.

Text Matching

ASAPP-ASR: Multistream CNN and Self-Attentive SRU for SOTA Speech Recognition

no code implementations21 May 2020 Jing Pan, Joshua Shapiro, Jeremy Wohlwend, Kyu J. Han, Tao Lei, Tao Ma

In this paper we present state-of-the-art (SOTA) performance on the LibriSpeech corpus with two novel neural network architectures, a multistream CNN for acoustic modeling and a self-attentive simple recurrent unit (SRU) for language modeling.

Data Augmentation Diversity +3

Structured Pruning of Large Language Models

2 code implementations EMNLP 2020 Ziheng Wang, Jeremy Wohlwend, Tao Lei

Large language models have recently achieved state of the art performance across a wide variety of natural language tasks.

Language Modelling Model Compression +1

Adaptive Morphological Reconstruction for Seeded Image Segmentation

1 code implementation8 Apr 2019 Tao Lei, Xiaohong Jia, Tongliang Liu, Shigang Liu, Hongy-ing Meng, Asoke K. Nandi

However, MR might mistakenly filter meaningful seeds that are required for generating accurate segmentation and it is also sensitive to the scale because a single-scale structuring element is employed.

Image Segmentation Segmentation +1

Adversarial Domain Adaptation for Duplicate Question Detection

1 code implementation EMNLP 2018 Darsh J Shah, Tao Lei, Alessandro Moschitti, Salvatore Romeo, Preslav Nakov

We address the problem of detecting duplicate questions in forums, which is an important step towards automating the process of answering new questions.

Domain Adaptation Question Similarity

Significantly Fast and Robust Fuzzy C-MeansClustering Algorithm Based on MorphologicalReconstruction and Membership Filtering

no code implementations IEEE 2018 Tao Lei, Xiaohong Jia, Yanning Zhang, Lifeng He, Hongy-ing Meng, Senior Member, and Asoke K. Nandi, Fellow, IEEE

However, the introduction oflocal spatial information often leads to a high computationalcomplexity, arising out of an iterative calculation of the distancebetween pixels within local spatial neighbors and clusteringcenters.

Clustering Image Segmentation +1

Training RNNs as Fast as CNNs

1 code implementation ICLR 2018 Tao Lei, Yu Zhang, Yoav Artzi

Common recurrent neural network architectures scale poorly due to the intrinsic difficulty in parallelizing their state computations.

General Classification Language Modelling +4

Style Transfer from Non-Parallel Text by Cross-Alignment

12 code implementations NeurIPS 2017 Tianxiao Shen, Tao Lei, Regina Barzilay, Tommi Jaakkola

We demonstrate the effectiveness of this cross-alignment method on three tasks: sentiment modification, decipherment of word substitution ciphers, and recovery of word order.

Decipherment Machine Translation +3

Deriving Neural Architectures from Sequence and Graph Kernels

no code implementations ICML 2017 Tao Lei, Wengong Jin, Regina Barzilay, Tommi Jaakkola

The design of neural architectures for structured objects is typically guided by experimental insights rather than a formal process.

Graph Regression Language Modelling +1

Rationalizing Neural Predictions

3 code implementations EMNLP 2016 Tao Lei, Regina Barzilay, Tommi Jaakkola

Our approach combines two modular components, generator and encoder, which are trained to operate well together.

Retrieval Sentiment Analysis

Semi-supervised Question Retrieval with Gated Convolutions

1 code implementation NAACL 2016 Tao Lei, Hrishikesh Joshi, Regina Barzilay, Tommi Jaakkola, Katerina Tymoshenko, Alessandro Moschitti, Lluis Marquez

Question answering forums are rapidly growing in size with no effective automated ability to refer to and reuse answers already available for previous posted questions.

Decoder Question Answering +1

Cannot find the paper you are looking for? You can Submit a new open access paper.