Search Results for author:
Found 55 papers, 28 papers with code
How to leverage the multimodal EHR data for better medical prediction?
Therefore, in this paper, we first extract the accompanying clinical notes from EHR and propose a method to integrate these data, we also comprehensively study the different models and the data leverage methods for better medical task prediction performance.
Machine Translation With Weakly Paired Bilingual Documents
Neural machine translation, which achieves near human-level performance in some languages, strongly relies on the availability of large amounts of parallel sentences, which hinders its applicability to low-resource language pairs.
mixSeq: A Simple Data Augmentation Methodfor Neural Machine Translation
Data augmentation, which refers to manipulating the inputs (e. g., adding random noise, masking specific parts) to enlarge the dataset, has been widely adopted in machine learning.
Retrosynthesis Prediction with Local Template Retrieval
Furthermore, we propose a lightweight adapter to adjust the weights when combing neural network and KNN predictions conditioned on the hidden representation and the retrieved templates.
MolXPT: Wrapping Molecules with Text for Generative Pre-training
Considering that text is the most important record for scientific discovery, in this paper, we propose MolXPT, a unified language model of text and molecules pre-trained on SMILES (a sequence representation of molecules) wrapped by text.
Ranked #1 on
Molecular Property Prediction
on SIDER
O-GNN: Incorporating Ring Priors into Molecular Modeling
Despite the recent success of molecular modeling with graph neural networks (GNNs), few models explicitly take rings in compounds into consideration, consequently limiting the expressiveness of the models.
Ranked #1 on
Graph Regression
on PCQM4M-LSC
(Validation MAE metric)
Filler Word Detection with Hard Category Mining and Inter-Category Focal Loss
Filler words like ``um" or ``uh" are common in spontaneous speech.
AMOM: Adaptive Masking over Masking for Conditional Masked Language Model
Experiments on \textbf{3} different tasks (neural machine translation, summarization, and code generation) with \textbf{15} datasets in total confirm that our proposed simple method achieves significant performance improvement over the strong CMLM model.
Retrieved Sequence Augmentation for Protein Representation Learning
RSA links query protein sequences to a set of sequences with similar structures or properties in the database and combines these sequences for downstream prediction.
De Novo Molecular Generation via Connection-aware Motif Mining
The obtained motif vocabulary consists of not only molecular motifs (i. e., the frequent fragments), but also their connection information, indicating how the motifs are connected with each other.
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change
Existing methods mainly perform continual training to mitigate such a misalignment.
Incorporating Pre-training Paradigm for Antibody Sequence-Structure Co-design
Specifically, we first pre-train an antibody language model based on the sequence data, then propose a one-shot way for sequence and structure generation of CDR to avoid the heavy cost and error propagation from an autoregressive manner, and finally leverage the pre-trained antibody model for the antigen-specific antibody generation model with some carefully designed modules.
Tailoring Molecules for Protein Pockets: a Transformer-based Generative Solution for Structured-based Drug Design
Structure-based drug design is drawing growing attentions in computer-aided drug discovery.
Unified 2D and 3D Pre-Training of Molecular Representations
The model is pre-trained on three tasks: reconstruction of masked atoms and coordinates, 3D conformation generation conditioned on 2D graph, and 2D graph generation conditioned on 3D conformation.
RetroGraph: Retrosynthetic Planning with Graph Search
We observe that the same intermediate molecules are visited many times in the searching process, and they are usually independently treated in previous tree-based methods (e. g., AND-OR tree search, Monte Carlo tree search).
Ranked #2 on
Multi-step retrosynthesis
on USPTO-190
SMT-DTA: Improving Drug-Target Affinity Prediction with Semi-supervised Multi-task Training
This seriously hinders the application of deep learning based methods, which require a large scale of supervised data.
Ranked #1 on
Drug Discovery
on KIBA
A Survey on Non-Autoregressive Generation for Neural Machine Translation and Beyond
While NAR generation can significantly accelerate inference speed for machine translation, the speedup comes at the cost of sacrificed translation accuracy compared to its counterpart, autoregressive (AR) generation.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+11
A Mutually Reinforced Framework for Pretrained Sentence Embeddings
Instead of relying on annotation heuristics defined by humans, it leverages the sentence representation model itself and realizes the following iterative self-supervision process: on one hand, the improvement of sentence representation may contribute to the quality of data annotation; on the other hand, more effective data annotation helps to generate high-quality positive samples, which will further improve the current sentence representation model.
Dynamic Relation Discovery and Utilization in Multi-Entity Time Series Forecasting
How to discover the useful implicit relation between entities and effectively utilize the relations for each entity under various circumstances is crucial.
AF$_2$: Adaptive Focus Framework for Aerial Imagery Segmentation
As a specific semantic segmentation task, aerial imagery segmentation has been widely employed in high spatial resolution (HSR) remote sensing images understanding.
Direct Molecular Conformation Generation
Molecular conformation generation aims to generate three-dimensional coordinates of all the atoms in a molecule and is an important task in bioinformatics and pharmacology.
CT4Rec: Simple yet Effective Consistency Training for Sequential Recommendation
State-of-the-art sequential recommendation models proposed very recently combine contrastive learning techniques for obtaining high-quality user representations.
Learning from Inside: Self-driven Siamese Sampling and Reasoning for Video Question Answering
To consider the interdependent knowledge between contextual clips into the network inference, we propose a Siamese Sampling and Reasoning (SiaSamRea) approach, which consists of a siamese sampling mechanism to generate sparse and similar clips (i. e., siamese clips) from the same video, and a novel reasoning strategy for integrating the interdependent knowledge between contextual clips into the network.
Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model
The key problem in the protein sequence representation learning is to capture the co-evolutionary information reflected by the inter-residue co-variation in the sequences.
How to Leverage Multimodal EHR Data for Better Medical Predictions?
Therefore, in this paper, we first extract the accompanying clinical notes from EHR and propose a method to integrate these data, we also comprehensively study the different models and the data leverage methods for better medical task prediction.
DM-CT: Consistency Training with Data and Model Perturbation
The common approach of consistency training is performed on the data-level, which typically utilizes the data augmentation strategy (or adversarial training) to make the predictions from the augmented input and the original input to be consistent, so that the model is more robust and attains better generalization ability.
Are BERT Families Zero-Shot Learners? A Study on Their Potential and Limitations
Starting from the resurgence of deep learning, language models (LMs) have never been so popular.
Target-Side Data Augmentation for Sequence Generation
Autoregressive sequence generation, a prevalent task in machine learning and natural language processing, generates every target token conditioned on both a source input and previously generated target tokens.
Discovering Drug-Target Interaction Knowledge from Biomedical Literature
We regard the DTI triplets as a sequence and use a Transformer-based model to directly generate them without using the detailed annotations of entities and relations.
R-Drop: Regularized Dropout for Neural Networks
Dropout is a powerful and widely used technique to regularize the training of deep neural networks.
Ranked #4 on
Machine Translation
on WMT2014 English-French
Incorporating NODE with Pre-trained Neural Differential Operator for Learning Dynamics
In this paper, to reduce the reliance on the numerical solver, we propose to enhance the supervised signal in the training of NODE.
Distance-Enhanced Graph Neural Network for Link Prediction
To overcome this difficulty, we propose an anchorbased distance: First, we randomly select K anchor vertices from the graph and then calculate the shortest distances of all vertices in the graph to them.
Ranked #1 on
Link Property Prediction
on ogbl-ddi
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
Therefore, in this paper, we integrate different dropout techniques into the training of Transformer models.
Ranked #3 on
Machine Translation
on IWSLT2014 English-German
IOT: Instance-wise Layer Reordering for Transformer Structures
Based on this observation, in this work, we break the assumption of the fixed layer order in the Transformer and introduce instance-wise layer reordering into the model structure.
Learning to Use Future Information in Simultaneous Translation
For wait-k inference, we observe that wait-m training with $m>k$ in simultaneous NMT (i. e., using more future information for training than inference) generally outperforms wait-k training.
ADD: Augmented Disentanglement Distillation Framework for Improving Stock Trend Forecasting
Stock trend forecasting has become a popular research direction that attracts widespread attention in the financial field.
Masked Contrastive Representation Learning for Reinforcement Learning
During inference, the CNN encoder and the policy network are used to take actions, and the Transformer module is discarded.
Temporally Correlated Task Scheduling for Sequence Learning
In many applications, a sequence learning task is usually associated with multiple temporally correlated auxiliary tasks, which are different in terms of how much input information to use or which future step to predict.
Learning to Reweight with Deep Interactions
Recently, the concept of teaching has been introduced into machine learning, in which a teacher model is used to guide the training of a student model (which will be used in real tasks) through data selection, loss function design, etc.
Multi-branch Attentive Transformer
While the multi-branch architecture is one of the key ingredients to the success of computer vision tasks, it has not been well investigated in natural language processing, especially sequence learning tasks.
Ranked #4 on
Machine Translation
on WMT2014 English-German
(SacreBLEU metric)
Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detection
Specifically, we merge the quality estimation into the class prediction vector to form a joint representation of localization quality and classification, and use a vector to represent arbitrary distribution of box locations.
Ranked #107 on
Object Detection
on COCO test-dev
Incorporating BERT into Neural Machine Translation
While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning.
Microsoft Research Asia's Systems for WMT19
We Microsoft Research Asia made submissions to 11 language directions in the WMT19 news translation tasks.
Exploiting Monolingual Data at Scale for Neural Machine Translation
In this work, we study how to use both the source-side and target-side monolingual data for NMT, and propose an effective strategy leveraging both of them.
Ranked #1 on
Machine Translation
on WMT2016 English-German
(SacreBLEU metric, using extra
training data)
Machine Translation With Weakly Paired Documents
1) We provide a simple approach to mine implicitly bilingual sentence pairs from document pairs which can then be used as supervised training signals.
Efficient Bidirectional Neural Machine Translation
In this paper, we propose an efficient method to generate a sequence in both left-to-right and right-to-left manners using a single encoder and decoder, combining the advantages of both generation directions.
Depth Growing for Neural Machine Translation
While very deep neural networks have shown effectiveness for computer vision and text classification applications, how to increase the network depth of neural machine translation (NMT) models for better translation quality remains a challenging problem.
Ranked #11 on
Machine Translation
on WMT2014 English-French
Soft Contextual Data Augmentation for Neural Machine Translation
While data augmentation is an important trick to boost the accuracy of deep learning methods in computer vision tasks, its study in natural language tasks is still very limited.
Learning to Teach with Dynamic Loss Functions
Different from typical learning settings in which the loss function of a machine learning model is predefined and fixed, in our framework, the loss function of a machine learning model (we call it student) is defined by another machine learning model (we call it teacher).
Beyond Error Propagation in Neural Machine Translation: Characteristics of Language Also Matter
Many previous works have discussed the relationship between error propagation and the \emph{accuracy drop} (i. e., the left part of the translated sentence is often better than its right part in left-to-right decoding models) problem.
A Study of Reinforcement Learning for Neural Machine Translation
Recent studies have shown that reinforcement learning (RL) is an effective approach for improving the performance of neural machine translation (NMT) system.
Efficient Sequence Learning with Group Recurrent Networks
Recurrent neural networks have achieved state-of-the-art results in many artificial intelligence tasks, such as language modeling, neural machine translation, speech recognition and so on.
Achieving Human Parity on Automatic Chinese to English News Translation
Machine translation has made rapid advances in recent years.
Ranked #3 on
Machine Translation
on WMT 2017 English-Chinese
Deliberation Networks: Sequence Generation Beyond One-Pass Decoding
In this work, we introduce the deliberation process into the encoder-decoder framework and propose deliberation networks for sequence generation.
Adversarial Neural Machine Translation
The goal of the adversary is to differentiate the translation result generated by the NMT model from that by human.
