Search Results for author:
Found 40 papers, 20 papers with code
PGL at TextGraphs 2020 Shared Task: Explanation Regeneration using Language and Graph Learning Methods
To address this problem, we use a pre-trained language model to recall the top-K relevant explanations for each question.
Pre-training with Fractional Denoising to Enhance Molecular Property Prediction
Deep learning methods have been considered promising for accelerating molecular screening in drug discovery and material design.
MoleculeCLA: Rethinking Molecular Benchmark via Computational Ligand-Target Binding Analysis
We believe this dataset will serve as a more accurate and reliable benchmark for molecular representation learning, thereby expediting progress in the field of artificial intelligence-driven drug discovery.
UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning
Recently, a noticeable trend has emerged in developing pre-trained foundation models in the domains of CV and NLP.
Contextual Molecule Representation Learning from Chemical Reaction Knowledge
REMO offers a novel solution to MRL by exploiting the underlying shared patterns in chemical reactions as \textit{context} for pre-training, which effectively infers meaningful representations of common chemistry knowledge.
Named Entity Driven Zero-Shot Image Manipulation
We introduced StyleEntity a zero-shot image manipulation model that utilizes named entities as proxies during its training phase.
Protein-ligand binding representation learning from fine-grained interactions
The binding between proteins and ligands plays a crucial role in the realm of drug discovery.
Sliced Denoising: A Physics-Informed Molecular Pre-Training Method
By aligning with physical principles, SliDe shows a 42\% improvement in the accuracy of estimated force fields compared to current state-of-the-art denoising methods, and thus outperforms traditional baselines on various molecular property prediction tasks.
UniMAP: Universal SMILES-Graph Representation Learning
Molecular representation learning is fundamental for many drug related applications.
DocPrompt: Large-scale continue pretrain for zero-shot and few-shot document question answering
In this paper, we propose Docprompt for document question answering tasks with powerful zero-shot and few-shot performance.
Fractional Denoising for 3D Molecular Pre-training
Theoretically, the objective is equivalent to learning the force field, which is revealed helpful for downstream tasks.
Multimodal Molecular Pretraining via Modality Blending
Self-supervised learning has recently gained growing interest in molecular modeling for scientific tasks such as AI-assisted drug discovery.
ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images
It is hoped that this competition will attract many researchers in the field of CV and NLP, and bring some new thoughts to the field of Document AI.
Efficient Text-Guided 3D-Aware Portrait Generation with Score Distillation Sampling on Distribution
These elaborated designs enable our model to generate portraits with robust multi-view semantic consistency, eliminating the need for optimization-based methods.
Spectral Heterogeneous Graph Convolutions via Positive Noncommutative Polynomials
Furthermore, these methods cannot learn arbitrary valid heterogeneous graph filters within the spectral domain, which have limited expressiveness.
Ranked #6 on
Node Property Prediction
on ogbn-mag
Label Information Enhanced Fraud Detection against Low Homophily in Graphs
Then, we combine the group aggregation and the learnable encodings into a Transformer encoder to capture the semantic information.
Layout-aware Webpage Quality Assessment
To assess webpage quality from complex DOM tree data, we propose a graph neural network (GNN) based method that extracts rich layout-aware information that implies webpage quality in an end-to-end manner.
ERNIE 3.0 Tiny: Frustratingly Simple Method to Improve Task-Agnostic Distillation Generalization
Experimental results demonstrate that our method yields a student with much better generalization, significantly outperforms existing baselines, and establishes a new state-of-the-art result on in-domain, out-domain, and low-resource datasets in the setting of task-agnostic distillation.
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts
Recent progress in diffusion models has revolutionized the popular technology of text-to-image generation.
Ranked #13 on
Text-to-Image Generation
on MS COCO
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding
Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding.
Ranked #2 on
Semantic entity labeling
on FUNSD
ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding
At first, a document graph is proposed to model complex relationships among multi-grained multimodal elements, in which salient visual regions are detected by a cluster-based method.
DuETA: Traffic Congestion Propagation Pattern Modeling via Efficient Graph Learning for ETA Prediction at Baidu Maps
In this work, we focus on modeling traffic congestion propagation patterns to improve ETA performance.
Simple and Effective Relation-based Embedding Propagation for Knowledge Representation Learning
Notably, it averagely brings about 10% relative improvement to triplet-based embedding methods on OGBL-WikiKG2 and takes 5%-83% time to achieve comparable results as the state-of-the-art GC-OTE.
ERNIE-SPARSE: Learning Hierarchical Efficient Transformer Through Regularized Self-Attention
We argue that two factors, information bottleneck sensitivity and inconsistency between different attention topologies, could affect the performance of the Sparse Transformer.
ERNIE-GeoL: A Geography-and-Language Pre-trained Model and its Applications in Baidu Maps
One of the main reasons for this plateau is the lack of readily available geographic knowledge in generic PTMs.
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
A unified framework named ERNIE 3. 0 was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters.
Graph4Rec: A Universal Toolkit with Graph Neural Networks for Recommender Systems
In recent years, owing to the outstanding performance in graph representation learning, graph neural network (GNN) techniques have gained considerable interests in many real-world scenarios, such as recommender systems and social networks.
ERNIE-SPARSE: Robust Efficient Transformer Through Hierarchically Unifying Isolated Information
The first factor is information bottleneck sensitivity, which is caused by the key feature of Sparse Transformer — only a small number of global tokens can attend to all other tokens.
abcbpc at SemEval-2021 Task 7: ERNIE-based Multi-task Model for Detecting and Rating Humor and Offense
This paper describes our system participated in Task 7 of SemEval-2021: Detecting and Rating Humor and Offense.
Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification
This paper describes our system participated in Task 6 of SemEval-2021: the task focuses on multimodal propaganda technique classification and it aims to classify given image and text into 22 classes.
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph.
NOTE: Solution for KDD-CUP 2021 WikiKG90M-LSC
In addition to the representations, we also use various statistical probabilities among the head entities, the relations and the tail entities for the final prediction.
ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression
Specifically, the first stage, General Distillation, performs distillation with guidance from pretrained teacher, gerenal data and latent distillation loss.
Molecular Representation Learning by Leveraging Chemical Information
As graph neural networks have achieved great success in many domains, some studies apply graph neural networks to molecular property prediction and regard each molecule as a graph.
Ranked #6 on
Graph Property Prediction
on ogbg-molhiv
ERNIE at SemEval-2020 Task 10: Learning Word Emphasis Selection by Pre-trained Language Model
This paper describes the system designed by ERNIE Team which achieved the first place in SemEval-2020 Task 10: Emphasis Selection For Written Text in Visual Media.
kk2018 at SemEval-2020 Task 9: Adversarial Training for Code-Mixing Sentiment Classification
Code switching is a linguistic phenomenon that may occur within a multilingual setting where speakers share more than one language.
Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification
Graph neural network (GNN) and label propagation algorithm (LPA) are both message passing algorithms, which have achieved superior performance in semi-supervised classification.
Ranked #1 on
Node Property Prediction
on ogbn-proteins
ERNIE 2.0: A Continual Pre-training Framework for Language Understanding
Recently, pre-trained models have achieved state-of-the-art results in various language understanding tasks, which indicates that pre-training on large-scale corpora may play a crucial role in natural language processing.
Ranked #1 on
Chinese Sentence Pair Classification
on LCQMC Dev
Chinese Named Entity Recognition
Chinese Reading Comprehension
+8
ERNIE: Enhanced Representation through Knowledge Integration
We present a novel language representation model enhanced by knowledge called ERNIE (Enhanced Representation through kNowledge IntEgration).
Ranked #3 on
Natural Language Inference
on XNLI Chinese Dev
Chinese Named Entity Recognition
Chinese Sentence Pair Classification
+8
Functional Hashing for Compressing Neural Networks
As the complexity of deep neural networks (DNNs) trend to grow to absorb the increasing sizes of data, memory and energy consumption has been receiving more and more attentions for industrial applications, especially on mobile devices.
