Search Results for author:
Found 4 papers, 1 papers with code
ClusterFormer: Neural Clustering Attention for Efficient and Effective Transformer
Other sparse methods use clustering patterns to select words, but the clustering process is separate from the training process of the target task, which causes a decrease in effectiveness.
基于多层次预训练策略和多任务学习的端到端蒙汉语音翻译(End-to-end Mongolian-Chinese Speech Translation Based on Multi-level Pre-training Strategies and Multi-task Learning)
“端到端语音翻译将源语言语音直接翻译为目标语言文本, 它需要“源语言语音-目标语言文本”作为训练数据, 然而这类数据极其稀缺, 本文提出了一种多层次预训练策略和多任务学习相结合的训练方法, 首先分别对语音识别和机器翻译模型的各个模块进行多层次预训练, 接着将语音识别和机器翻译模型连接起来构成语音翻译模型, 然后使用迁移学习对预训练好的模型进行多步骤微调, 在此过程中又运用多任务学习的方法, 将语音识别作为语音翻译的一个辅助任务来组织训练, 充分利用了已经存在的各种不同形式的数据来训练端到端模型, 首次将端到端技术应用于资源受限条件下的蒙汉语音翻译, 构建了首个翻译质量较高、实际可用的端到端蒙汉语音翻译系统。”
TensorCoder: Dimension-Wise Attention via Tensor Representation for Natural Language Modeling
Transformer has been widely-used in many Natural Language Processing (NLP) tasks and the scaled dot-product attention between tokens is a core module of Transformer.
Adaptive Fusion for RGB-D Salient Object Detection
RGB-D salient object detection aims to identify the most visually distinctive objects in a pair of color and depth images.
