Search Results for author:
Found 43 papers, 13 papers with code
Tencent AI Lab Machine Translation Systems for WMT20 Chat Translation Task
This paper describes the Tencent AI Lab’s submission of the WMT 2020 shared task on chat translation in English-German.
Redistributing Low-Frequency Words: Making the Most of Monolingual Data in Non-Autoregressive Translation
In this work, we provide an appealing alternative for NAT – monolingual KD, which trains NAT student on external monolingual data with AT teacher trained on the original bilingual data.
The USYD-JD Speech Translation System for IWSLT2021
Our constrained system is based on a pipeline framework, i. e. ASR and NMT.
Comprehensive Graph Gradual Pruning for Sparse Training in Graph Neural Networks
However, the LTH-based methods suffer from two major drawbacks: 1) they require exhaustive and iterative training of dense models, resulting in an extremely large training computation cost, and 2) they only trim graph structures and model parameters but ignore the node feature dimension, where significant redundancy exists.
Dynamic Contrastive Distillation for Image-Text Retrieval
Although the vision-and-language pretraining (VLP) equipped cross-modal image-text retrieval (ITR) has achieved remarkable progress in the past two years, it suffers from a major drawback: the ever-increasing size of VLP models restricts its deployment to real-world search scenarios (where the high latency is unacceptable).
E2S2: Encoding-Enhanced Sequence-to-Sequence Pretraining for Language Understanding and Generation
Sequence-to-sequence (seq2seq) learning has become a popular trend for pretraining language models, due to its succinct and universal framework.
Parameter-Efficient and Student-Friendly Knowledge Distillation
In this paper, we present a parameter-efficient and student-friendly knowledge distillation method, namely PESF-KD, to achieve efficient and sufficient knowledge transfer by updating relatively few partial parameters.
Interpretable Proof Generation via Iterative Backward Reasoning
We present IBR, an Iterative Backward Reasoning model to solve the proof generation tasks on rule-based Question Answering (QA), where models are required to reason over a series of textual rules and facts to find out the related proof path and derive the final answer.
Bridging Cross-Lingual Gaps During Leveraging the Multilingual Sequence-to-Sequence Pretraining for Text Generation
For multilingual sequence-to-sequence pretrained language models (multilingual Seq2Seq PLMs), e. g. mBART, the self-supervised pretraining task is trained on a wide range of monolingual languages, e. g. 25 languages from commoncrawl, while the downstream cross-lingual tasks generally progress on a bilingual language subset, e. g. English-German, making there exists the cross-lingual data discrepancy, namely \textit{domain discrepancy}, and cross-lingual learning objective discrepancy, namely \textit{task discrepancy}, between the pretrain and finetune stages.
BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representation
Data augmentations (DA) are the cores to achieving robust sequence-to-sequence learning on various natural language processing (NLP) tasks.
A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysis
To mitigate this issue, we design a novel training framework, called Contrastive Cross-Channel Data Augmentation (C3DA).
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning
Instead, we propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG), which relieves the issue of direct model aggregation.
Where Does the Performance Improvement Come From? -- A Reproducibility Concern about Image-Text Retrieval
In contrast to previous works, we focus on the reproducibility of the approaches and the examination of the elements that lead to improved performance by pretrained and nonpretrained models in retrieving images and text.
Kernel Packet: An Exact and Scalable Algorithm for Gaussian Process Regression with Matérn Correlations
We develop an exact and scalable algorithm for one-dimensional Gaussian process regression with Mat\'ern correlations whose smoothness parameter $\nu$ is a half-integer.
Improving Neural Machine Translation by Denoising Training
We present a simple and effective pretraining strategy {D}en{o}ising {T}raining DoT for neural machine translation.
Knowledge Graph Augmented Network Towards Multiview Representation Learning for Aspect-based Sentiment Analysis
To this end, we propose a knowledge graph augmented network (KGAN), which aims to effectively incorporate external knowledge with explicitly syntactic and contextual information.
A Sparse Expansion For Deep Gaussian Processes
In this work, we propose an efficient scheme for accurate inference and prediction based on a range of Gaussian Processes, called the Tensor Markov Gaussian Processes (TMGP).
Unified Instance and Knowledge Alignment Pretraining for Aspect-based Sentiment Analysis
In practice, we formulate the model pretrained on the sampled instances into a knowledge guidance model and a learner model, respectively.
On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation
Pre-training (PT) and back-translation (BT) are two simple and powerful methods to utilize monolingual data for improving the model performance of neural machine translation (NMT).
FLBoost: On-the-Fly Fine-tuning Boosts Federated Learning via Data-free Distillation
On the contrary, we propose a new solution: on-the-fly fine-tuning the global model in server via data-free distillation to boost its performance, dubbed FLBoost to relieve the issue of direct model aggregation.
Improving Neural Machine Translation by Bidirectional Training
We present a simple and effective pretraining strategy -- bidirectional training (BiT) for neural machine translation.
The USYD-JD Speech Translation System for IWSLT 2021
Our constrained system is based on a pipeline framework, i. e. ASR and NMT.
High-Dimensional Simulation Optimization via Brownian Fields and Sparse Grids
High-dimensional simulation optimization is notoriously challenging.
On the Copying Behaviors of Pre-Training for Neural Machine Translation
In response to this problem, we propose a simple and effective method named copying penalty to control the copying behaviors in decoding.
Progressive Multi-Granularity Training for Non-Autoregressive Translation
Non-autoregressive translation (NAT) significantly accelerates the inference process via predicting the entire target sequence.
Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation
Results demonstrate that the proposed approach can significantly and universally improve translation quality by reducing translation errors on low-frequency words.
Self-Guided Curriculum Learning for Neural Machine Translation
In the field of machine learning, the well-trained model is assumed to be able to recover the training labels, i. e. the synthetic labels predicted by the model should be as close to the ground-truth labels as possible.
Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language Understanding
Spoken language understanding (SLU) system usually consists of various pipeline components, where each component heavily relies on the results of its upstream ones.
Towards Efficiently Diversifying Dialogue Generation via Embedding Augmentation
Dialogue generation models face the challenge of producing generic and repetitive responses.
MirrorAlign: A Super Lightweight Unsupervised Word Alignment Model via Cross-Lingual Contrastive Learning
Recently, the performance of the neural word alignment models has exceeded that of statistical models.
Unsupervised Word Alignment via Cross-Lingual Contrastive Learning
Recently, the performance of the neural word alignment models has exceeded that of statistical models.
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning
Encoder layer fusion (EncoderFusion) is a technique to fuse all the encoder layers (instead of the uppermost layer) for sequence-to-sequence (Seq2Seq) models, which has proven effective on various NLP tasks.
Understanding and Improving Lexical Choice in Non-Autoregressive Translation
To this end, we introduce an extra Kullback-Leibler divergence term derived by comparing the lexical choice of NAT model and that embedded in the raw data.
Context-Aware Cross-Attention for Non-Autoregressive Translation
Non-autoregressive translation (NAT) significantly accelerates the inference process by predicting the entire target sequence.
Sample and Computationally Efficient Stochastic Kriging in High Dimensions
However, its use is limited to cases where the design space is low-dimensional because, in general, the sample complexity (i. e., the number of design points required for stochastic kriging to produce an accurate prediction) grows exponentially in the dimensionality of the design space.
Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns
In response to this issue, we propose to expose explicit cross-lingual patterns, \textit{e. g.} word alignments and generation score, to our proposed zero-shot models.
SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling
Slot filling and intent detection are two main tasks in spoken language understanding (SLU) system.
High-Dimensional Non-Parametric Density Estimation in Mixed Smooth Sobolev Spaces
Density estimation plays a key role in many tasks in machine learning, statistical inference, and visualization.
Self-Attention with Cross-Lingual Position Representation
Position encoding (PE), an essential part of self-attention networks (SANs), is used to preserve the word order information for natural language processing tasks, generating fixed position indices for input sequences.
Generalization Guarantees for Sparse Kernel Approximation with Entropic Optimal Features
Despite their success, kernel methods suffer from a massive computational cost in practice.
Recurrent Graph Syntax Encoder for Neural Machine Translation
Syntax-incorporated machine translation models have been proven successful in improving the model's reasoning and meaning preservation ability.
The University of Sydney's Machine Translation System for WMT19
This paper describes the University of Sydney's submission of the WMT 2019 shared news translation task.
Ranked #1 on
Machine Translation
on WMT2019 Finnish-English
Efficient Learning of Optimal Markov Network Topology with k-Tree Modeling
The seminal work of Chow and Liu (1968) shows that approximation of a finite probabilistic system by Markov trees can achieve the minimum information loss with the topology of a maximum spanning tree.
