Search Results for author:
Found 65 papers, 34 papers with code
mixSeq: A Simple Data Augmentation Methodfor Neural Machine Translation
Data augmentation, which refers to manipulating the inputs (e. g., adding random noise, masking specific parts) to enlarge the dataset, has been widely adopted in machine learning.
UniGenX: Unified Generation of Sequence and Structure with Autoregressive Diffusion
We validate the effectiveness of UniGenX on material and small molecule generation tasks, achieving a significant leap in state-of-the-art performance for material crystal structure prediction and establishing new state-of-the-art results for small molecule structure prediction, de novo design, and conditional generation.
HybriDNA: A Hybrid Transformer-Mamba2 Long-Range DNA Language Model
To address these challenges, we propose HybriDNA, a decoder-only DNA language model that incorporates a hybrid Transformer-Mamba2 architecture, seamlessly integrating the strengths of attention mechanisms with selective state-space models.
NatureLM: Deciphering the Language of Nature for Scientific Discovery
Foundation models have revolutionized natural language processing and artificial intelligence, significantly enhancing how machines comprehend and generate human languages.
Exploiting Pre-trained Models for Drug Target Affinity Prediction with Nearest Neighbors
Specifically, we propose a \emph{label aggregation} with \emph{pair-wise retrieval} and a \emph{representation aggregation} with \emph{point-wise retrieval} of the nearest neighbors.
BioT5: Enriching Cross-modal Integration in Biology with Chemical Knowledge and Natural Language Associations
Recent advancements in biological research leverage the integration of molecules, proteins, and natural language to enhance drug discovery.
Ranked #4 on
Text-based de novo Molecule Generation
on ChEBI-20
FABind: Fast and Accurate Protein-Ligand Binding
In this work, we propose $\mathbf{FABind}$, an end-to-end model that combines pocket prediction and docking to achieve accurate and fast protein-ligand binding.
Ranked #4 on
Blind Docking
on PDBBind
Retrosynthesis Prediction with Local Template Retrieval
Furthermore, we propose a lightweight adapter to adjust the weights when combing neural network and KNN predictions conditioned on the hidden representation and the retrieved templates.
MolXPT: Wrapping Molecules with Text for Generative Pre-training
Considering that text is the most important record for scientific discovery, in this paper, we propose MolXPT, a unified language model of text and molecules pre-trained on SMILES (a sequence representation of molecules) wrapped by text.
Ranked #1 on
Molecular Property Prediction
on BACE
What are the Desired Characteristics of Calibration Sets? Identifying Correlates on Long Form Scientific Summarization
Summarization models often generate text that is poorly calibrated to quality metrics because they are trained to maximize the likelihood of a single reference (MLE).
O-GNN: Incorporating Ring Priors into Molecular Modeling
Despite the recent success of molecular modeling with graph neural networks (GNNs), few models explicitly take rings in compounds into consideration, consequently limiting the expressiveness of the models.
Ranked #1 on
Graph Regression
on PCQM4M-LSC
(Validation MAE metric)
De Novo Molecular Generation via Connection-aware Motif Mining
The obtained motif vocabulary consists of not only molecular motifs (i. e., the frequent fragments), but also their connection information, indicating how the motifs are connected with each other.
Retrosynthetic Planning with Dual Value Networks
Retrosynthesis, which aims to find a route to synthesize a target molecule from commercially available starting materials, is a critical task in drug discovery and materials design.
Ranked #1 on
Multi-step retrosynthesis
on USPTO-190
Incorporating Pre-training Paradigm for Antibody Sequence-Structure Co-design
Specifically, we first pre-train an antibody language model based on the sequence data, then propose a one-shot way for sequence and structure generation of CDR to avoid the heavy cost and error propagation from an autoregressive manner, and finally leverage the pre-trained antibody model for the antigen-specific antibody generation model with some carefully designed modules.
BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining
Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain.
Ranked #1 on
Document Classification
on HOC
(Micro F1 metric)
Tailoring Molecules for Protein Pockets: a Transformer-based Generative Solution for Structured-based Drug Design
Structure-based drug design is drawing growing attentions in computer-aided drug discovery.
Unified 2D and 3D Pre-Training of Molecular Representations
The model is pre-trained on three tasks: reconstruction of masked atoms and coordinates, 3D conformation generation conditioned on 2D graph, and 2D graph generation conditioned on 3D conformation.
RetroGraph: Retrosynthetic Planning with Graph Search
We observe that the same intermediate molecules are visited many times in the searching process, and they are usually independently treated in previous tree-based methods (e. g., AND-OR tree search, Monte Carlo tree search).
Ranked #2 on
Multi-step retrosynthesis
on USPTO-190
SSM-DTA: Breaking the Barriers of Data Scarcity in Drug-Target Affinity Prediction
Accurate prediction of Drug-Target Affinity (DTA) is of vital importance in early-stage drug discovery, facilitating the identification of drugs that can effectively interact with specific targets and regulate their activities.
Ranked #1 on
Drug Discovery
on KIBA
Direct Molecular Conformation Generation
Molecular conformation generation aims to generate three-dimensional coordinates of all the atoms in a molecule and is an important task in bioinformatics and pharmacology.
DDG-DA: Data Distribution Generation for Predictable Concept Drift Adaptation
To handle concept drift, previous methods first detect when/where the concept drift happens and then adapt models to fit the distribution of the latest data.
SHGNN: Structure-Aware Heterogeneous Graph Neural Network
Next, we use a tree-attention aggregator to incorporate the graph structure information into the aggregation module on the meta-path.
Stylized Dialogue Generation with Multi-Pass Dual Learning
Considering there is no parallel data between the contexts and the responses of target style S1, existing works mainly use back translation to generate stylized synthetic data for training, where the data about context, target style S1 and an intermediate style S0 is used.
Pre-training Co-evolutionary Protein Representation via A Pairwise Masked Language Model
The key problem in the protein sequence representation learning is to capture the co-evolutionary information reflected by the inter-residue co-variation in the sequences.
HIST: A Graph-based Framework for Stock Trend Forecasting via Mining Concept-Oriented Shared Information
To overcome the shortcomings of previous work, we proposed a novel stock trend forecasting framework that can adequately mine the concept-oriented shared information from predefined concepts and hidden concepts.
MS$^2$-Transformer: An End-to-End Model for MS/MS-assisted Molecule Identification
Manually interpreting the MS/MS spectrum into the molecules (i. e., the simplified molecular-input line-entry system, SMILES) is often costly and cumbersome, mainly due to the synthesis and labeling of isotopes and the requirement of expert knowledge.
Target-Side Data Augmentation for Sequence Generation
Autoregressive sequence generation, a prevalent task in machine learning and natural language processing, generates every target token conditioned on both a source input and previously generated target tokens.
Discovering Drug-Target Interaction Knowledge from Biomedical Literature
We regard the DTI triplets as a sequence and use a Transformer-based model to directly generate them without using the detailed annotations of entities and relations.
Dual-view Molecule Pre-training
After pre-training, we can use either the Transformer branch (this one is recommended according to empirical results), the GNN branch, or both for downstream tasks.
Ranked #1 on
Molecular Property Prediction
on HIV dataset
Distance-Enhanced Graph Neural Network for Link Prediction
To overcome this difficulty, we propose an anchorbased distance: First, we randomly select K anchor vertices from the graph and then calculate the shortest distances of all vertices in the graph to them.
Ranked #1 on
Link Property Prediction
on ogbl-ddi
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
Therefore, in this paper, we integrate different dropout techniques into the training of Transformer models.
Ranked #5 on
Machine Translation
on IWSLT2014 English-German
IOT: Instance-wise Layer Reordering for Transformer Structures
Based on this observation, in this work, we break the assumption of the fixed layer order in the Transformer and introduce instance-wise layer reordering into the model structure.
Learning to Use Future Information in Simultaneous Translation
For wait-k inference, we observe that wait-m training with $m>k$ in simultaneous NMT (i. e., using more future information for training than inference) generally outperforms wait-k training.
COSEA: Convolutional Code Search with Layer-wise Attention
However, most existing studies overlook the code's intrinsic structural logic, which indeed contains a wealth of semantic information, and fails to capture intrinsic features of codes.
Masked Contrastive Representation Learning for Reinforcement Learning
During inference, the CNN encoder and the policy network are used to take actions, and the Transformer module is discarded.
Temporally Correlated Task Scheduling for Sequence Learning
In many applications, a sequence learning task is usually associated with multiple temporally correlated auxiliary tasks, which are different in terms of how much input information to use or which future step to predict.
Learning to Reweight with Deep Interactions
Recently, the concept of teaching has been introduced into machine learning, in which a teacher model is used to guide the training of a student model (which will be used in real tasks) through data selection, loss function design, etc.
Multi-branch Attentive Transformer
While the multi-branch architecture is one of the key ingredients to the success of computer vision tasks, it has not been well investigated in natural language processing, especially sequence learning tasks.
Ranked #4 on
Machine Translation
on WMT2014 English-German
(SacreBLEU metric)
Dual Learning: Theoretical Study and an Algorithmic Extension
Dual learning has been successfully applied in many machine learning applications including machine translation, image-to-image transformation, etc.
TuiGAN: Learning Versatile Image-to-Image Translation with Two Unpaired Images
With TuiGAN, an image is translated in a coarse-to-fine manner where the generated image is gradually refined from global structures to local details.
Incorporating BERT into Neural Machine Translation
While BERT is more commonly used as fine-tuning instead of contextual embedding for downstream language understanding tasks, in NMT, our preliminary exploration of using BERT as contextual embedding is better than using for fine-tuning.
Neural Machine Translation with Soft Prototype
Neural machine translation models usually use the encoder-decoder framework and generate translation from left to right (or right to left) without fully utilizing the target-side global information.
Microsoft Research Asia's Systems for WMT19
We Microsoft Research Asia made submissions to 11 language directions in the WMT19 news translation tasks.
Exploiting Monolingual Data at Scale for Neural Machine Translation
In this work, we study how to use both the source-side and target-side monolingual data for NMT, and propose an effective strategy leveraging both of them.
Ranked #1 on
Machine Translation
on WMT2016 English-German
(SacreBLEU metric, using extra
training data)
Efficient Bidirectional Neural Machine Translation
In this paper, we propose an efficient method to generate a sequence in both left-to-right and right-to-left manners using a single encoder and decoder, combining the advantages of both generation directions.
Multilingual Neural Machine Translation with Language Clustering
We study two methods for language clustering: (1) using prior knowledge, where we cluster languages according to language family, and (2) using language embedding, in which we represent each language by an embedding vector and cluster them in the embedding space.
Depth Growing for Neural Machine Translation
While very deep neural networks have shown effectiveness for computer vision and text classification applications, how to increase the network depth of neural machine translation (NMT) models for better translation quality remains a challenging problem.
Ranked #11 on
Machine Translation
on WMT2014 English-French
ZstGAN: An Adversarial Approach for Unsupervised Zero-Shot Image-to-Image Translation
Image-to-image translation models have shown remarkable ability on transferring images among different domains.
Image-to-Image Translation with Multi-Path Consistency Regularization
In this work, we introduce a new kind of loss, multi-path consistency loss, which evaluates the differences between direct translation $\mathcal{D}_s\to\mathcal{D}_t$ and indirect translation $\mathcal{D}_s\to\mathcal{D}_a\to\mathcal{D}_t$ with $\mathcal{D}_a$ as an auxiliary domain, to regularize training.
Soft Contextual Data Augmentation for Neural Machine Translation
While data augmentation is an important trick to boost the accuracy of deep learning methods in computer vision tasks, its study in natural language tasks is still very limited.
Multi-Agent Dual Learning
Dual learning has attracted much attention in machine learning, computer vision and natural language processing communities.
Ranked #1 on
Machine Translation
on WMT2016 English-German
Dual Learning: Theoretical Study and Algorithmic Extensions
Based on the theoretical discoveries, we extend dual learning by introducing more related mappings and propose highly symmetric frameworks, cycle dual learning and multipath dual learning, in both of which we can leverage the feedback signals from additional domains to improve the qualities of the mappings.
Exploring Explicit Domain Supervision for Latent Space Disentanglement in Unpaired Image-to-Image Translation
After pre-training, this network is used to extract the domain-specific features of each image.
Layer-Wise Coordination between Encoder and Decoder for Neural Machine Translation
Neural Machine Translation (NMT) has achieved remarkable progress with the quick evolvement of model structures.
Learning to Teach with Dynamic Loss Functions
Different from typical learning settings in which the loss function of a machine learning model is predefined and fixed, in our framework, the loss function of a machine learning model (we call it student) is defined by another machine learning model (we call it teacher).
Model-Level Dual Learning
Many artificial intelligence tasks appear in dual forms like English$\leftrightarrow$French translation and speech$\leftrightarrow$text transformation.
Finite Sample Analysis of LSTD with Random Projections and Eligibility Traces
Policy evaluation with linear function approximation is an important problem in reinforcement learning.
Conditional Image-to-Image Translation
In this paper, we study a new problem, conditional image-to-image translation, which is to translate an image from the source domain to the target domain conditioned on a given image in the target domain.
Achieving Human Parity on Automatic Chinese to English News Translation
Machine translation has made rapid advances in recent years.
Ranked #3 on
Machine Translation
on WMT 2017 English-Chinese
Deliberation Networks: Sequence Generation Beyond One-Pass Decoding
In this work, we introduce the deliberation process into the encoder-decoder framework and propose deliberation networks for sequence generation.
Ranked #22 on
Machine Translation
on WMT2014 English-French
Decoding with Value Networks for Neural Machine Translation
Inspired by the success and methodology of AlphaGo, in this paper we propose using a prediction network to improve beam search, which takes the source sentence $x$, the currently available decoding output $y_1,\cdots, y_{t-1}$ and a candidate word $w$ at step $t$ as inputs and predicts the long-term value (e. g., BLEU score) of the partial target sentence if it is completed by the NMT model.
Dual Supervised Learning
Many supervised learning tasks are emerged in dual forms, e. g., English-to-French translation vs. French-to-English translation, speech recognition vs. text to speech, and image classification vs. image generation.
Adversarial Neural Machine Translation
The goal of the adversary is to differentiate the translation result generated by the NMT model from that by human.
Dual Learning for Machine Translation
Based on the feedback signals generated during this process (e. g., the language-model likelihood of the output of a model, and the reconstruction error of the original sentence after the primal and dual translations), we can iteratively update the two models until convergence (e. g., using the policy gradient methods).
Thompson Sampling for Budgeted Multi-armed Bandits
In this paper, we extend the Thompson sampling to Budgeted MAB, where there is random cost for pulling an arm and the total cost is constrained by a budget.
