Search Results for author:
Found 197 papers, 97 papers with code
Global Ranking Using Continuous Conditional Random Fields
It can naturally represent the content information of objects as well as the relation information between objects, necessary for global ranking.
A New Probabilistic Model for Rank Aggregation
To avoid these limitations, in this paper, we propose a new model, which is defined with a coset-permutation distance, and models the generation of a permutation as a stagewise process.
Pure Price of Anarchy for Generalized Second Price Auction
Previous studies have shown that the pure Price Of Anarchy (POA) of GSP is 1. 25 when there are two ad slots and 1. 259 when three ad slots.
Computer Science and Game Theory
Estimation Bias in Multi-Armed Bandit Algorithms for Search Advertising
In search advertising, the search engine needs to select the most profitable advertisements to display, which can be formulated as an instance of online learning with partial feedback, also known as the stochastic multi-armed bandit (MAB) problem.
Agent Behavior Prediction and Its Generalization Analysis
Then we prove a generalization bound for the machine learning algorithms on the behavior data generated by the new Markov chain, which depends on both the Markovian parameters and the covering number of the function class compounded by the loss function for behavior prediction and the behavior prediction model.
Generalization Analysis for Game-Theoretic Machine Learning
For Internet applications like sponsored search, cautions need to be taken when using machine learning to optimize their mechanisms (e. g., auction) since self-interested agents in these applications may change their behaviors (and thus the data distribution) in response to the mechanisms.
Thompson Sampling for Budgeted Multi-armed Bandits
In this paper, we extend the Thompson sampling to Budgeted MAB, where there is random cost for pulling an arm and the total cost is constrained by a budget.
LightRNN: Memory and Computation-Efficient Recurrent Neural Networks
Based on the 2-Component shared embedding, we design a new RNN algorithm and evaluate it using the language modeling task on several benchmark datasets.
Dual Learning for Machine Translation
Based on the feedback signals generated during this process (e. g., the language-model likelihood of the output of a model, and the reconstruction error of the original sentence after the primal and dual translations), we can iteratively update the two models until convergence (e. g., using the policy gradient methods).
Randomized Mechanisms for Selling Reserved Instances in Cloud
We then extend our mechanism to the general case and achieve a competitive ratio $\frac{1}{42\log k\log T}$ for both social welfare and revenue, where $T$ is the ratio of the maximum request length to the minimum request length and $k$ is the ratio of the maximum request value density to the minimum request value density.
Adversarial Neural Machine Translation
The goal of the adversary is to differentiate the translation result generated by the NMT model from that by human.
Reinforcement Learning for Learning Rate Control
Stochastic gradient descent (SGD), which updates the model parameters by adding a local gradient times a learning rate at each step, is widely used in model training of machine learning algorithms such as neural networks.
Question Answering and Question Generation as Dual Tasks
On one side, the QA model judges whether the generated question of a QG model is relevant to the answer.
Dual Supervised Learning
Many supervised learning tasks are emerged in dual forms, e. g., English-to-French translation vs. French-to-English translation, speech recognition vs. text to speech, and image classification vs. image generation.
Decoding with Value Networks for Neural Machine Translation
Inspired by the success and methodology of AlphaGo, in this paper we propose using a prediction network to improve beam search, which takes the source sentence $x$, the currently available decoding output $y_1,\cdots, y_{t-1}$ and a candidate word $w$ at step $t$ as inputs and predicts the long-term value (e. g., BLEU score) of the partial target sentence if it is completed by the NMT model.
Deliberation Networks: Sequence Generation Beyond One-Pass Decoding
In this work, we introduce the deliberation process into the encoder-decoder framework and propose deliberation networks for sequence generation.
Achieving Human Parity on Automatic Chinese to English News Translation
Machine translation has made rapid advances in recent years.
Ranked #3 on
Machine Translation
on WMT 2017 English-Chinese
Conditional Image-to-Image Translation
In this paper, we study a new problem, conditional image-to-image translation, which is to translate an image from the source domain to the target domain conditioned on a given image in the target domain.
Learning to Teach
Teaching plays a very important role in our society, by spreading human knowledge and educating our next generations.
Efficient Sequence Learning with Group Recurrent Networks
Recurrent neural networks have achieved state-of-the-art results in many artificial intelligence tasks, such as language modeling, neural machine translation, speech recognition and so on.
Dense Information Flow for Neural Machine Translation
Recently, neural machine translation has achieved remarkable progress by introducing well-designed deep neural networks into its encoder-decoder framework.
Ranked #68 on
Machine Translation
on WMT2014 English-German
Towards Binary-Valued Gates for Robust LSTM Training
Long Short-Term Memory (LSTM) is one of the most widely used recurrent structures in sequence modeling.
Double Path Networks for Sequence to Sequence Learning
In this work we propose Double Path Networks for Sequence to Sequence learning (DPN-S2S), which leverage the advantages of both models by using double path information fusion.
Model-Level Dual Learning
Many artificial intelligence tasks appear in dual forms like English$\leftrightarrow$French translation and speech$\leftrightarrow$text transformation.
Neural Architecture Optimization
The performance predictor and the encoder enable us to perform gradient based optimization in the continuous space to find the embedding of a new architecture with potentially better accuracy.
A Study of Reinforcement Learning for Neural Machine Translation
Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation (NMT) system.
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter
Many previous works have discussed the relationship between error propagation and the \emph{accuracy drop} (i. e., the left part of the translated sentence is often better than its right part in left-to-right decoding models) problem.
FRAGE: Frequency-Agnostic Word Representation
Continuous word representation (aka word embedding) is a basic building block in many neural network-based models used in natural language processing tasks.
Ranked #3 on
Machine Translation
on IWSLT2015 German-English
Learning to Teach with Dynamic Loss Functions
Different from typical learning settings in which the loss function of a machine learning model is predefined and fixed, in our framework, the loss function of a machine learning model (we call it student) is defined by another machine learning model (we call it teacher).
Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation
Neural Machine Translation (NMT) has achieved remarkable progress with the quick evolvement of model structures.
Sentence-wise Smooth Regularization for Sequence to Sequence Learning
Maximum-likelihood estimation (MLE) is widely used in sequence to sequence tasks for model training.
Non-Autoregressive Neural Machine Translation with Enhanced Decoder Input
Non-autoregressive translation (NAT) models, which remove the dependence on previous target tokens from the inputs of the decoder, achieve significantly inference speedup but at the cost of inferior accuracy compared to autoregressive translation (AT) models.
Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation
After pre-training, this network is used to extract the domain-specific features of each image.
Non-Autoregressive Machine Translation with Auxiliary Regularization
However, the high efficiency has come at the cost of not capturing the sequential dependency on the target side of translation, which causes NAT to suffer from two kinds of translation errors: 1) repeated translations (due to indistinguishable adjacent decoder hidden states), and 2) incomplete translations (due to incomplete transfer of source side information via the decoder hidden states).
Multilingual Neural Machine Translation with Knowledge Distillation
Multilingual machine translation, which translates multiple languages with a single model, has attracted much attention due to its efficiency of offline training and online serving.
Competitive Bridge Bidding with Deep Neural Networks
Second, based on the analysis of the impact of other players' unknown cards on one's final rewards, we design two neural networks to deal with imperfect information, the first one inferring the cards of the partner and the second one taking the outputs of the first one as part of its input to select a bid.
Token-Level Ensemble Distillation for Grapheme-to-Phoneme Conversion
Recently, G2P conversion is viewed as a sequence to sequence task and modeled by RNN or CNN based encoder-decoder framework.
Ranked #1 on
Text-To-Speech Synthesis
on CMUDict 0.7b
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Multi-Agent Dual Learning
Dual learning has attracted much attention in machine learning, computer vision and natural language processing communities.
Ranked #1 on
Machine Translation
on WMT2016 English-German
Local Image-to-Image Translation via Pixel-wise Highway Adaptive Instance Normalization
Recently, image-to-image translation has seen a significant success.
Dual Learning: Theoretical Study and Algorithmic Extensions
Based on the theoretical discoveries, we extend dual learning by introducing more related mappings and propose highly symmetric frameworks, cycle dual learning and multipath dual learning, in both of which we can leverage the feedback signals from additional domains to improve the qualities of the mappings.
Hint-based Training for Non-Autoregressive Translation
To improve the accuracy of NART models, in this paper, we propose to leverage the hints from a well-trained ART model to train the NART model.
MASS: Masked Sequence to Sequence Pre-training for Language Generation
Pre-training and fine-tuning, e. g., BERT, have achieved great success in language understanding by transferring knowledge from rich-resource pre-training task to the low/zero-resource downstream tasks.
Almost Unsupervised Text to Speech and Automatic Speech Recognition
Text to speech (TTS) and automatic speech recognition (ASR) are two dual tasks in speech processing and both achieve impressive performance thanks to the recent advance in deep learning and large amount of aligned speech and text data.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
FastSpeech: Fast, Robust and Controllable Text to Speech
In this work, we propose a novel feed-forward network based on Transformer to generate mel-spectrogram in parallel for TTS.
Ranked #10 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
FastSpeech: Fast,Robustand Controllable Text-to-Speech
Compared with traditional concatenative and statistical parametric approaches, neural network based end-to-end models suffer from slow inference speed, and the synthesized speech is usually not robust (i. e., some words are skipped or repeated) and lack of controllability (voice speed or prosody control).
Soft Contextual Data Augmentation for Neural Machine Translation
While data augmentation is an important trick to boost the accuracy of deep learning methods in computer vision tasks, its study in natural language tasks is still very limited.
Image-to-Image Translation with Multi-Path Consistency Regularization
In this work, we introduce a new kind of loss, multi-path consistency loss, which evaluates the differences between direct translation $\mathcal{D}_s\to\mathcal{D}_t$ and indirect translation $\mathcal{D}_s\to\mathcal{D}_a\to\mathcal{D}_t$ with $\mathcal{D}_a$ as an auxiliary domain, to regularize training.
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View
In this paper, we provide a novel perspective towards understanding the architecture: we show that the Transformer can be mathematically interpreted as a numerical Ordinary Differential Equation (ODE) solver for a convection-diffusion equation in a multi-particle dynamic system.
Unsupervised Pivot Translation for Distant Languages
In this work, we introduce unsupervised pivot translation for distant languages, which translates a language to a distant language through multiple hops, and the unsupervised translation on each hop is relatively easier than the original direct translation.
What and Where to Translate: Local Mask-based Image-to-Image Translation
First, those methods extract style from an entire exemplar which includes noisy information, which impedes a translation model from properly extracting the intended style of the exemplar.
Depth Growing for Neural Machine Translation
While very deep neural networks have shown effectiveness for computer vision and text classification applications, how to increase the network depth of neural machine translation (NMT) models for better translation quality remains a challenging problem.
Ranked #11 on
Machine Translation
on WMT2014 English-French
Representation Degeneration Problem in Training Natural Language Generation Models
We study an interesting problem in training neural network-based models for natural language generation tasks, which we call the \emph{representation degeneration problem}.
Language Graph Distillation for Low-Resource Machine Translation
Neural machine translation on low-resource language is challenging due to the lack of bilingual sentence pairs.
Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation
Neural machine translation (NMT) typically adopts the encoder-decoder framework.
Multilingual Neural Machine Translation with Language Clustering
We study two methods for language clustering: (1) using prior knowledge, where we cluster languages according to language family, and (2) using language embedding, in which we represent each language by an embedding vector and cluster them in the embedding space.
Efficient Bidirectional Neural Machine Translation
In this paper, we propose an efficient method to generate a sequence in both left-to-right and right-to-left manners using a single encoder and decoder, combining the advantages of both generation directions.
Effective Search of Logical Forms for Weakly Supervised Knowledge-Based Question Answering
These two problems lead to a poorly-trained semantic parsing model.
Hint-Based Training for Non-Autoregressive Machine Translation
Due to the unparallelizable nature of the autoregressive factorization, AutoRegressive Translation (ART) models have to generate tokens sequentially during decoding and thus suffer from high inference latency.
Balanced One-shot Neural Architecture Optimization
One-shot NAS is proposed to reduce the expense but shows inferior performance against conventional NAS and is not adequately stable.
Independence-aware Advantage Estimation
Most of existing advantage function estimation methods in reinforcement learning suffer from the problem of high variance, which scales unfavorably with the time horizon.
Demonstration Actor Critic
One approach leverages demonstration data in a supervised manner, which is simple and direct, but can only provide supervision signal over those states seen in the demonstrations.
Multi-Task Learning for Conversational Question Answering over a Large-Scale Knowledge Base
We consider the problem of conversational question answering over a large-scale knowledge base.
Exploiting Monolingual Data at Scale for Neural Machine Translation
In this work, we study how to use both the source-side and target-side monolingual data for NMT, and propose an effective strategy leveraging both of them.
Ranked #1 on
Machine Translation
on WMT2016 English-German
(SacreBLEU metric, using extra
training data)
Machine Translation With Weakly Paired Documents
1) We provide a simple approach to mine implicitly bilingual sentence pairs from document pairs which can then be used as supervised training signals.
Fully Parameterized Quantile Function for Distributional Reinforcement Learning
The key challenge in practical distributional RL algorithms lies in how to parameterize estimated distributions so as to better approximate the true continuous distribution.
Ranked #3 on
Atari Games
on Atari 2600 Skiing
(using extra training data)
Distributional Reward Decomposition for Reinforcement Learning
Many reinforcement learning (RL) tasks have specific properties that can be leveraged to modify existing RL algorithms to adapt to those tasks and further improve performance, and a general class of such properties is the multiple reward channel.
Microsoft Research Asia's Systems for WMT19
We Microsoft Research Asia made submissions to 11 language directions in the WMT19 news translation tasks.
Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation
Non-autoregressive translation (NAT) models remove the dependence on previous target tokens and generate all target tokens in parallel, resulting in significant inference speedup but at the cost of inferior translation accuracy compared to autoregressive translation (AT) models.
Neural Machine Translation with Soft Prototype
Neural machine translation models usually use the encoder-decoder framework and generate translation from left to right (or right to left) without fully utilizing the target-side global information.
Normalization Helps Training of Quantized LSTM
The long-short-term memory (LSTM), though powerful, is memory and computa\x02tion expensive.
A Study of Multilingual Neural Machine Translation
Multilingual neural machine translation (NMT) has recently been investigated from different aspects (e. g., pivot translation, zero-shot translation, fine-tuning, or training from scratch) and in different settings (e. g., rich resource and low resource, one-to-many, and many-to-one translation).
Incorporating BERT into Neural Machine Translation
While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning.
Semi-Supervised Neural Architecture Search
On ImageNet, it achieves 23. 5% top-1 error rate (under 600M FLOPS constraint) using 4 GPU-days for search.
Ranked #81 on
Neural Architecture Search
on ImageNet
Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View.
In particular, how words in a sentence are abstracted into contexts by passing through the layers of the Transformer can be interpreted as approximating multiple particles' movement in the space using the Lie-Trotter splitting scheme and the Euler's method.
Suphx: Mastering Mahjong with Deep Reinforcement Learning
Artificial Intelligence (AI) has achieved great success in many domains, and game AI is widely regarded as its beachhead since the dawn of AI.
MPNet: Masked and Permuted Pre-training for Language Understanding
Since BERT neglects dependency among predicted tokens, XLNet introduces permuted language modeling (PLM) for pre-training to address this problem.
Ranked #16 on
Only Connect Walls Dataset Task 1 (Grouping)
on OCW
(using extra training data)
LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning
While pre-training and fine-tuning, e. g., BERT~\citep{devlin2018bert}, GPT-2~\citep{radford2019language}, have achieved great success in language understanding and generation tasks, the pre-trained models are usually too big for online deployment in terms of both memory cost and inference speed, which hinders them from practical online usage.
Dual Learning: Theoretical Study and an Algorithmic Extension
Dual learning has been successfully applied in many machine learning applications including machine translation, image-to-image transformation, etc.
MultiSpeech: Multi-Speaker Text to Speech with Transformer
Transformer-based text to speech (TTS) model (e. g., Transformer TTS~\cite{li2019neural}, FastSpeech~\cite{ren2019fastspeech}) has shown the advantages of training and inference efficiency over RNN-based model (e. g., Tacotron~\cite{shen2018natural}) due to its parallel computation in training and/or inference.
FastSpeech 2: Fast and High-Quality End-to-End Text to Speech
In this paper, we propose FastSpeech 2, which addresses the issues in FastSpeech and better solves the one-to-many mapping problem in TTS by 1) directly training the model with ground-truth target instead of the simplified output from teacher, and 2) introducing more variation information of speech (e. g., pitch, energy and more accurate duration) as conditional inputs.
Ranked #6 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
UWSpeech: Speech to Speech Translation for Unwritten Languages
Existing speech to speech translation systems heavily rely on the text of target language: they usually translate source language either to target text and then synthesize target speech from text, or directly to target speech with target text for auxiliary training.
Multi-branch Attentive Transformer
While the multi-branch architecture is one of the key ingredients to the success of computer vision tasks, it has not been well investigated in natural language processing, especially sequence learning tasks.
Ranked #4 on
Machine Translation
on WMT2014 English-German
(SacreBLEU metric)
SimulSpeech: End-to-End Simultaneous Speech to Text Translation
In this work, we develop SimulSpeech, an end-to-end simultaneous speech to text translation system which translates speech in source language to text in target language concurrently.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+7
DeepSinger: Singing Voice Synthesis with Data Mined From the Web
DeepSinger has several advantages over previous SVS systems: 1) to the best of our knowledge, it is the first SVS system that directly mines training data from music websites, 2) the lyrics-to-singing alignment model further avoids any human efforts for alignment labeling and greatly reduces labeling cost, 3) the singing model based on a feed-forward Transformer is simple and efficient, by removing the complicated acoustic feature modeling in parametric synthesis and leveraging a reference encoder to capture the timbre of a singer from noisy singing data, and 4) it can synthesize singing voices in multiple languages and multiple singers.
Accuracy Prediction with Non-neural Model for Neural Architecture Search
Considering that most architectures are represented as sequences of discrete symbols which are more like tabular data and preferred by non-neural predictors, in this paper, we study an alternative approach which uses non-neural model for accuracy prediction.
Ranked #81 on
Neural Architecture Search
on ImageNet
Learning to Reweight with Deep Interactions
Recently, the concept of teaching has been introduced into machine learning, in which a teacher model is used to guide the training of a student model (which will be used in real tasks) through data selection, loss function design, etc.
Temporally Correlated Task Scheduling for Sequence Learning
In many applications, a sequence learning task is usually associated with multiple temporally correlated auxiliary tasks, which are different in terms of how much input information to use or which future step to predict.
Learning to Match Distributions for Domain Adaptation
However, it remains challenging to determine which method is suitable for a given application since they are built with certain priors or bias.
Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation
SAT contains a hyperparameter k, and each k value defines a SAT task with different degrees of parallelism.
LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition
However, there are more than 6, 000 languages in the world and most languages are lack of speech training data, which poses significant challenges when building TTS and ASR systems for extremely low-resource languages.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
PopMAG: Pop Music Accompaniment Generation
To improve harmony, in this paper, we propose a novel MUlti-track MIDI representation (MuMIDI), which enables simultaneous multi-track generation in a single sequence and explicitly models the dependency of the notes from different tracks.
HiFiSinger: Towards High-Fidelity Neural Singing Voice Synthesis
To tackle the difficulty of singing modeling caused by high sampling rate (wider frequency band and longer waveform), we introduce multi-scale adversarial training in both the acoustic model and vocoder to improve singing modeling.
Knowledge-Aware Procedural Text Understanding with Multi-Stage Training
In this work, we focus on the task of procedural text understanding, which aims to comprehend such documents and track entities' states and locations during a process.
Towards Interpretable Reasoning over Paragraph Effects in Situation
We focus on the task of reasoning over paragraph effects in situation, which requires a model to understand the cause and effect described in a background paragraph, and apply the knowledge to a novel situation.
Masked Contrastive Representation Learning for Reinforcement Learning
During inference, the CNN encoder and the policy network are used to take actions, and the Transformer module is discarded.
Learning Causal Semantic Representation for Out-of-Distribution Prediction
Conventional supervised learning methods, especially deep ones, are found to be sensitive to out-of-distribution (OOD) examples, largely because the learned representation mixes the semantic factor with the variation factor due to their domain-specific correlation, while only the semantic factor causes the output.
Latent Causal Invariant Model
To avoid spurious correlation, we propose a Latent Causal Invariance Model (LaCIM) which pursues causal prediction.
RD$^2$: Reward Decomposition with Representation Decomposition
In this work, we propose a set of novel reward decomposition principles by constraining uniqueness and compactness of different state features/representations relevant to different sub-rewards.
SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint
Automatic song writing aims to compose a song (lyric and/or melody) by machine, which is an interesting topic in both academia and industry.
Denoising Text to Speech with Frame-Level Noise Modeling
In DenoiSpeech, we handle real-world noisy speech by modeling the fine-grained frame-level noise with a noise condition module, which is jointly trained with the TTS model.
ChemistryQA: A Complex Question Answering Dataset from Chemistry
We notice that some real-world QA tasks are more complex, which cannot be solved by end-to-end neural networks or translated to any kind of formal representations.
Task-Agnostic and Adaptive-Size BERT Compression
NAS-BERT trains a big supernet on a carefully designed search space containing various architectures and outputs multiple compressed models with adaptive sizes and latency.
Learning to Use Future Information in Simultaneous Translation
For wait-k inference, we observe that wait-m training with $m>k$ in simultaneous NMT (i. e., using more future information for training than inference) generally outperforms wait-k training.
Image-to-Image Translation: Methods and Applications
Image-to-image translation (I2I) aims to transfer images from a source domain to a target domain while preserving the content representations.
LightSpeech: Lightweight and Fast Text to Speech with Neural Architecture Search
Text to speech (TTS) has been broadly used to synthesize natural and intelligible speech in different scenarios.
Return-Based Contrastive Representation Learning for Reinforcement Learning
Recently, various auxiliary tasks have been proposed to accelerate representation learning and improve sample efficiency in deep reinforcement learning (RL).
MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition
In this paper, we propose MixSpeech, a simple yet effective data augmentation method based on mixup for automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
AdaSpeech: Adaptive Text to Speech for Custom Voice
2) To better trade off the adaptation parameters and voice quality, we introduce conditional layer normalization in the mel-spectrogram decoder of AdaSpeech, and fine-tune this part in addition to speaker embedding for adaptation.
Generalizing to Unseen Domains: A Survey on Domain Generalization
Domain generalization deals with a challenging setting where one or several different but related domain(s) are given, and the goal is to learn a model that can generalize to an unseen test domain.
Learning Invariant Representations across Domains and Tasks
Being expensive and time-consuming to collect massive COVID-19 image samples to train deep classification models, transfer learning is a promising approach by transferring knowledge from the abundant typical pneumonia datasets for COVID-19 image classification.
IOT: Instance-wise Layer Reordering for Transformer Structures
Based on this observation, in this work, we break the assumption of the fixed layer order in the Transformer and introduce instance-wise layer reordering into the model structure.
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
Therefore, in this paper, we integrate different dropout techniques into the training of Transformer models.
Ranked #4 on
Machine Translation
on IWSLT2014 English-German
Cross-domain Speech Recognition with Unsupervised Character-level Distribution Matching
End-to-end automatic speech recognition (ASR) can achieve promising performance with large-scale training data.
Ranked #1 on
Cross-environment ASR
on Libri-Adapt
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data
In adaptation, we use untranscribed speech data for speech reconstruction and only fine-tune the TTS decoder.
FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition
A straightforward solution to reduce latency, inspired by non-autoregressive (NAR) neural machine translation, is to use an NAR sequence generation model for ASR error correction, which, however, comes at the cost of significantly increased ASR error rate.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Exploiting Adapters for Cross-lingual Low-resource Speech Recognition
Based on our previous MetaAdapter that implicitly leverages adapters, we propose a novel algorithms called SimAdapter for explicitly learning knowledge from adapters.
Ranked #1 on
Cross-Lingual ASR
on Common Voice
Distance-Enhanced Graph Neural Network for Link Prediction
To overcome this difficulty, we propose an anchorbased distance: First, we randomly select K anchor vertices from the graph and then calculate the shortest distances of all vertices in the graph to them.
Ranked #1 on
Link Property Prediction
on ogbl-ddi
Learning Structures for Deep Neural Networks
We further establish connections between this principle and the theory of Bayesian optimal classification, and empirically verify that larger entropy of the outputs of a deep neural network indeed corresponds to a better classification accuracy.
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search
The technical challenge of NAS-BERT is that training a big supernet on the pre-training task is extremely costly.
MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training
Inspired by the success of pre-training models in natural language processing, in this paper, we develop MusicBERT, a large-scale pre-trained model for music understanding.
PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior
Denoising diffusion probabilistic models have been recently proposed to generate high-quality samples by estimating the gradient of the data density.
Dual-view Molecule Pre-training
After pre-training, we can use either the Transformer branch (this one is recommended according to empirical results), the GNN branch, or both for downstream tasks.
Ranked #1 on
Molecular Property Prediction
on HIV dataset
R-Drop: Regularized Dropout for Neural Networks
Dropout is a powerful and widely used technique to regularize the training of deep neural networks.
Ranked #4 on
Machine Translation
on WMT2014 English-French
A Survey on Neural Speech Synthesis
Text to speech (TTS), or speech synthesis, which aims to synthesize intelligible and natural speech given text, is a hot research topic in speech, language, and machine learning communities and has broad applications in the industry.
On the Generative Utility of Cyclic Conditionals
This is motivated by the observation that deep generative models, in addition to a likelihood model $p(x|z)$, often also use an inference model $q(z|x)$ for extracting representation, but they rely on a usually uninformative prior distribution $p(z)$ to define a joint distribution, which may render problems like posterior collapse and manifold mismatch.
Supervised Off-Policy Ranking
Inspired by the two observations, in this work, we study a new problem, supervised off-policy ranking (SOPR), which aims to rank a set of target policies based on supervised learning by leveraging off-policy data and policies with known performance.
DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling
In this paper, we develop DeepRapper, a Transformer-based rap generation system that can model both rhymes and rhythms.
AdaSpeech 3: Adaptive Text to Speech for Spontaneous Style
While recent text to speech (TTS) models perform very well in synthesizing reading-style (e. g., audiobook) speech, it is still challenging to synthesize spontaneous-style speech (e. g., podcast or conversation), mainly because of two reasons: 1) the lack of training data for spontaneous speech; 2) the difficulty in modeling the filled pauses (um and uh) and diverse rhythms in spontaneous speech.
A Survey on Low-Resource Neural Machine Translation
Neural approaches have achieved state-of-the-art accuracy on machine translation but suffer from the high cost of collecting large scale parallel data.
AdaRNN: Adaptive Learning and Forecasting of Time Series
This paper proposes Adaptive RNNs (AdaRNN) to tackle the TCS problem by building an adaptive model that generalizes well on the unseen test data.
Analyzing and Mitigating Interference in Neural Architecture Search
In this paper, we investigate the interference issue by sampling different child models and calculating the gradient similarity of shared operators, and observe: 1) the interference on a shared operator between two child models is positively correlated with the number of different operators; 2) the interference is smaller when the inputs and outputs of the shared operator are more similar.
PDAugment: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics Transcription
Considering that there is a large amount of ASR training data, a straightforward method is to leverage ASR data to enhance ALT training.
Automatic Lyrics Transcription
Automatic Speech Recognition
+3
TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method
In this paper, we develop TeleMelody, a two-stage lyric-to-melody generation system with music template (e. g., tonality, chord progression, rhythm pattern, and cadence) to bridge the gap between lyrics and melodies (i. e., the system consists of a lyric-to-template module and a template-to-melody module).
Discovering Drug-Target Interaction Knowledge from Biomedical Literature
We regard the DTI triplets as a sequence and use a Transformer-based model to directly generate them without using the detailed annotations of entities and relations.
Multi-Agent Reinforcement Learning with Shared Resource in Inventory Management
We consider inventory management (IM) problem for a single store with a large number of SKUs (stock keeping units) in this paper, where we need to make replenishment decisions for each SKU to balance its supply and demand.
Particle Based Stochastic Policy Optimization
Stochastic polic have been widely applied for their good property in exploration and uncertainty quantification.
Ranked #1 on
MuJoCo Games
on Ant-v3
Exploiting Class Activation Value for Partial-Label Learning
As the first contribution, we empirically show that the class activation map (CAM), a simple technique for discriminating the learning patterns of each class in images, is surprisingly better at making accurate predictions than the model itself on selecting the true label from candidate labels.
Target-Side Data Augmentation for Sequence Generation
Autoregressive sequence generation, a prevalent task in machine learning and natural language processing, generates every target token conditioned on both a source input and previously generated target tokens.
FastCorrect 2: Fast Error Correction on Multiple Candidates for Automatic Speech Recognition
Although multiple candidates are generated by an ASR system through beam search, current error correction approaches can only correct one sentence at a time, failing to leverage the voting effect from multiple candidates to better detect and correct error tokens.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
GNN is a Counter? Revisiting GNN for Question Answering
To open the black box of GNN and investigate these problems, we dissect state-of-the-art GNN modules for QA and analyze their reasoning capability.
Ranked #12 on
Question Answering
on OpenBookQA
Distributional Reinforcement Learning for Multi-Dimensional Reward Functions
To fully inherit the benefits of distributional RL and hybrid reward architectures, we introduce Multi-Dimensional Distributional DQN (MD3QN), which extends distributional RL to model the joint return distribution from multiple reward sources.
Distributional Reinforcement Learning
reinforcement-learning
+1
Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning
Behavioral cloning has proven to be effective for learning sequential decision-making policies from expert demonstrations.
Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model
The key problem in the protein sequence representation learning is to capture the co-evolutionary information reflected by the inter-residue co-variation in the sequences.
Towards Generating Real-World Time Series Data
In this paper, we propose a novel generative framework for RTS data - RTSGAN to tackle the aforementioned challenges.
Recovering Latent Causal Factor for Generalization to Distributional Shifts
To avoid such a spurious correlation, we propose \textbf{La}tent \textbf{C}ausal \textbf{I}nvariance \textbf{M}odels (LaCIM) that specifies the underlying causal structure of the data and the source of distributional shifts, guiding us to pursue only causal factor for prediction.
Personalized Federated Learning with Adaptive Batchnorm for Healthcare
There is a growing interest in applying machine learning techniques to healthcare.
Speech-T: Transducer for Text to Speech and Beyond
Experiments on LJSpeech datasets demonstrate that Speech-T 1) is more robust than the attention based autoregressive TTS model due to its inherent monotonic alignments between text and speech; 2) naturally supports streaming TTS with good voice quality; and 3) enjoys the benefit of joint modeling TTS and ASR in a single network.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Adaptive Memory Networks with Self-supervised Learning for Unsupervised Anomaly Detection
In this paper, we propose a novel approach called Adaptive Memory Network with Self-supervised Learning (AMSL) to address these challenges and enhance the generalization ability in unsupervised anomaly detection.
You May Not Need Ratio Clipping in PPO
Furthermore, we show that ESPO can be easily scaled up to distributed training with many workers, delivering strong performance as well.
Direct Molecular Conformation Generation
Molecular conformation generation aims to generate three-dimensional coordinates of all the atoms in a molecule and is an important task in bioinformatics and pharmacology.
Towards Deployment-Efficient Reinforcement Learning: Lower Bound and Optimality
Deployment efficiency is an important criterion for many real-world applications of reinforcement learning (RL).
Revisiting Over-Smoothness in Text to Speech
Then we conduct a comprehensive study on NAR-TTS models that use some advanced modeling methods.
Mixed-Phoneme BERT: Improving BERT with Mixed Phoneme and Sup-Phoneme Representations for Text to Speech
However, the works apply pre-training with character-based units to enhance the TTS phoneme encoder, which is inconsistent with the TTS fine-tuning that takes phonemes as input.
AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarios
We model the speaker characteristics systematically to improve the generalization on new speakers.
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
While NAR generation can significantly accelerate inference speed for machine translation, the speedup comes at the cost of sacrificed translation accuracy compared to its counterpart, autoregressive (AR) generation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+11
NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Quality
In this paper, we answer these questions by first defining the human-level quality based on the statistical significance of subjective measure and introducing appropriate guidelines to judge it, and then developing a TTS system called NaturalSpeech that achieves human-level quality on a benchmark dataset.
Ranked #1 on
Text-To-Speech Synthesis
on LJSpeech
(using extra training data)
Transcormer: Transformer for Sentence Scoring with Sliding Language Modeling
Previous works on sentence scoring mainly adopted either causal language modeling (CLM) like GPT or masked language modeling (MLM) like BERT, which have some limitations: 1) CLM only utilizes unidirectional information for the probability estimation of a sentence without considering bidirectional context, which affects the scoring quality; 2) MLM can only estimate the probability of partial tokens at a time and thus requires multiple forward passes to estimate the probability of the whole sentence, which incurs large computation and time cost.
Tiered Reinforcement Learning: Pessimism in the Face of Uncertainty and Constant Regret
We propose a new learning framework that captures the tiered structure of many real-world user-interaction applications, where the users can be divided into two groups based on their different tolerance on exploration risks and should be treated separately.
BinauralGrad: A Two-Stage Conditional Diffusion Probabilistic Model for Binaural Audio Synthesis
Combining this novel perspective of two-stage synthesis with advanced generative models (i. e., the diffusion models), the proposed BinauralGrad is able to generate accurate and high-fidelity binaural audio samples.
SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction
Accurate prediction of Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery, facilitating the identification of drugs that can effectively interact with specific targets and regulate their activities.
Ranked #1 on
Drug Discovery
on KIBA
RetroGraph: Retrosynthetic Planning with Graph Search
We observe that the same intermediate molecules are visited many times in the searching process, and they are usually independently treated in previous tree-based methods (e. g., AND-OR tree search, Monte Carlo tree search).
Ranked #2 on
Multi-step retrosynthesis
on USPTO-190
Building Multilingual Machine Translation Systems That Serve Arbitrary X-Y Translations
Multilingual Neural Machine Translation (MNMT) enables one system to translate sentences from multiple source languages to multiple target languages, greatly reducing deployment costs compared with conventional bilingual systems.
A Study of Syntactic Multi-Modality in Non-Autoregressive Machine Translation
Furthermore, we take the best of both and design a new loss function to better handle the complicated syntactic multi-modality in real-world datasets.
Unified 2D and 3D Pre-Training of Molecular Representations
The model is pre-trained on three tasks: reconstruction of masked atoms and coordinates, 3D conformation generation conditioned on 2D graph, and 2D graph generation conditioned on 3D conformation.
Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation
In this work, we propose using only screenshots/pixels as input for automated game testing and build a general game testing agent, Inspector, that can be easily applied to different games without deep integration with games.
Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation
These characteristics cannot be well handled by neural generation models that learn lyric-to-melody mapping in an end-to-end way, due to several issues: (1) lack of aligned lyric-melody training data to sufficiently learn lyric-melody feature alignment; (2) lack of controllability in generation to better and explicitly align the lyric-melody features.
MeloForm: Generating Melody with Musical Form based on Expert Systems and Neural Networks
Specifically, 1) we design an expert system to generate a melody by developing musical elements from motifs to phrases then to sections with repetitions and variations according to pre-given musical form; 2) considering the generated melody is lack of musical richness, we design a Transformer based refinement model to improve the melody without changing its musical form.
Tailoring Molecules for Protein Pockets: a Transformer-based Generative Solution for Structured-based Drug Design
Structure-based drug design is drawing growing attentions in computer-aided drug discovery.
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation
A recent trend is to use Transformer or its variants in music generation, which is, however, suboptimal, because the full attention cannot efficiently model the typically long music sequences (e. g., over 10, 000 tokens), and the existing models have shortcomings in generating musical repetition structures.
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain.
Ranked #1 on
Document Classification
on HOC
(Micro F1 metric)
Incorporating Pre-training Paradigm for Antibody Sequence-Structure Co-design
Specifically, we first pre-train an antibody language model based on the sequence data, then propose a one-shot way for sequence and structure generation of CDR to avoid the heavy cost and error propagation from an autoregressive manner, and finally leverage the pre-trained antibody model for the antigen-specific antibody generation model with some carefully designed modules.
SoftCorrect: Error Correction with Soft Detection for Automatic Speech Recognition
In this paper, we propose SoftCorrect with a soft error detection mechanism to avoid the limitations of both explicit and implicit error detection.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
TD3 with Reverse KL Regularizer for Offline Reinforcement Learning from Mixed Datasets
There are two challenges for this setting: 1) The optimal trade-off between optimizing the RL signal and the behavior cloning (BC) signal changes on different states due to the variation of the action coverage induced by different behavior policies.
An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Context
One of the key challenges in deploying RL to real-world applications is to adapt to variations of unknown environment contexts, such as changing terrains in robotic tasks and fluctuated bandwidth in congestion control.
Regeneration Learning: A Learning Paradigm for Data Generation
Regeneration learning extends the concept of representation learning to data generation tasks, and can be regarded as a counterpart of traditional representation learning, since 1) regeneration learning handles the abstraction (Y') of the target data Y for data generation while traditional representation learning handles the abstraction (X') of source data X for data understanding; 2) both the processes of Y'-->Y in regeneration learning and X-->X' in representation learning can be learned in a self-supervised way (e. g., pre-training); 3) both the mappings from X to Y' in regeneration learning and from X' to Y in representation learning are simpler than the direct mapping from X to Y.
N-Gram Nearest Neighbor Machine Translation
Nearest neighbor machine translation augments the Autoregressive Translation~(AT) with $k$-nearest-neighbor retrieval, by comparing the similarity between the token-level context representations of the target tokens in the query and the datastore.
Retrosynthetic Planning with Dual Value Networks
Retrosynthesis, which aims to find a route to synthesize a target molecule from commercially available starting materials, is a critical task in drug discovery and materials design.
Ranked #1 on
Multi-step retrosynthesis
on USPTO-190
De Novo Molecular Generation via Connection-aware Motif Mining
The obtained motif vocabulary consists of not only molecular motifs (i. e., the frequent fragments), but also their connection information, indicating how the motifs are connected with each other.
AMOM: Adaptive Masking over Masking for Conditional Masked Language Model
Experiments on \textbf{3} different tasks (neural machine translation, summarization, and code generation) with \textbf{15} datasets in total confirm that our proposed simple method achieves significant performance improvement over the strong CMLM model.
NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers
To enhance the zero-shot capability that is important to achieve diverse speech synthesis, we design a speech prompting mechanism to facilitate in-context learning in the diffusion model and the duration/pitch predictor.
Pointerformer: Deep Reinforced Multi-Pointer Transformer for the Traveling Salesman Problem
Traveling Salesman Problem (TSP), as a classic routing optimization problem originally arising in the domain of transportation and logistics, has become a critical task in broader domains, such as manufacturing and biology.
ResiDual: Transformer with Dual Residual Connections
In this paper, we propose ResiDual, a novel Transformer architecture with Pre-Post-LN (PPLN), which fuses the connections in Post-LN and Pre-LN together and inherits their advantages while avoids their limitations.
O-GNN: Incorporating Ring Priors into Molecular Modeling
Despite the recent success of molecular modeling with graph neural networks (GNNs), few models explicitly take rings in compounds into consideration, consequently limiting the expressiveness of the models.
Ranked #1 on
Graph Regression
on PCQM4M-LSC
(Validation MAE metric)
MolXPT: Wrapping Molecules with Text for Generative Pre-training
Considering that text is the most important record for scientific discovery, in this paper, we propose MolXPT, a unified language model of text and molecules pre-trained on SMILES (a sequence representation of molecules) wrapped by text.
Ranked #1 on
Molecular Property Prediction
on ClinTox
Extract and Attend: Improving Entity Translation in Neural Machine Translation
Inspired by this translation process, we propose an Extract-and-Attend approach to enhance entity translation in NMT, where the translation candidates of source entities are first extracted from a dictionary and then attended to by the NMT model to generate the target sentence.
Retrosynthesis Prediction with Local Template Retrieval
Furthermore, we propose a lightweight adapter to adjust the weights when combing neural network and KNN predictions conditioned on the hidden representation and the retrieved templates.
PromptTTS 2: Describing and Generating Voices with Text Prompt
TTS approaches based on the text prompt face two main challenges: 1) the one-to-many problem, where not all details about voice variability can be described in the text prompt, and 2) the limited availability of text prompt datasets, where vendors and large cost of data labeling are required to write text prompts for speech.
FABind: Fast and Accurate Protein-Ligand Binding
In this work, we propose $\mathbf{FABind}$, an end-to-end model that combines pocket prediction and docking to achieve accurate and fast protein-ligand binding.
Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
We find that prompting innovation can unlock deeper specialist capabilities and show that GPT-4 easily tops prior leading results for medical benchmarks.
Ranked #1 on
Question Answering
on MedQA
BioT5+: Towards Generalized Biological Understanding with IUPAC Integration and Multi-task Tuning
However, previous efforts like BioT5 faced challenges in generalizing across diverse tasks and lacked a nuanced understanding of molecular structures, particularly in their textual representations (e. g., IUPAC).
Ranked #1 on
Molecule Captioning
on ChEBI-20
Leveraging Biomolecule and Natural Language through Multi-Modal Learning: A Survey
The integration of biomolecular modeling with natural language (BL) has emerged as a promising interdisciplinary area at the intersection of artificial intelligence, chemistry and biology.
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models
Specifically, 1) we design a neural codec with factorized vector quantization (FVQ) to disentangle speech waveform into subspaces of content, prosody, timbre, and acoustic details; 2) we propose a factorized diffusion model to generate attributes in each subspace following its corresponding prompt.
Building Multilingual Machine Translation Systems That Serve Arbitrary XY Translations
Multilingual Neural Machine Translation (MNMT) enables one system to translate sentences from multiple source languages to multiple target languages, greatly reducing deployment costs compared with conventional bilingual systems.
Machine Translation With Weakly Paired Bilingual Documents
Neural machine translation, which achieves near human-level performance in some languages, strongly relies on the availability of large amounts of parallel sentences, which hinders its applicability to low-resource language pairs.
mixSeq: A Simple Data Augmentation Methodfor Neural Machine Translation
Data augmentation, which refers to manipulating the inputs (e. g., adding random noise, masking specific parts) to enlarge the dataset, has been widely adopted in machine learning.
