Search Results for author:
Found 38 papers, 26 papers with code
A Framework for Cost-Effective and Self-Adaptive LLM Shaking and Recovery Mechanism
As Large Language Models (LLMs) gain great success in real-world applications, an increasing number of users are seeking to develop and deploy their customized LLMs through cloud services.
Spectral Heterogeneous Graph Convolutions via Positive Noncommutative Polynomials
These spatial-based HGNNs neglect the utilization of spectral graph convolutions, which are the foundation of Graph Convolutional Networks (GCN) on homogeneous graphs.
Ranked #5 on
Node Property Prediction
on ogbn-mag
Label Information Enhanced Fraud Detection against Low Homophily in Graphs
Then, we combine the group aggregation and the learnable encodings into a Transformer encoder to capture the semantic information.
TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training
In this paper, we propose TA-MoE, a topology-aware routing strategy for large-scale MoE trainging, from a model-system co-design perspective, which can dynamically adjust the MoE dispatch pattern according to the network topology.
PP-YOLOE-R: An Efficient Anchor-Free Rotated Object Detector
With multi-scale training and testing, PP-YOLOE-R-l and PP-YOLOE-R-x further improve the detection precision to 80. 02 and 80. 73 mAP.
Ranked #6 on
Oriented Object Detection
on DOTA 1.0
PP-StructureV2: A Stronger Document Analysis System
For Table Recognition model, we utilize PP-LCNet, CSP-PAN and SLAHead to optimize the backbone module, feature fusion module and decoding module, respectively, which improved the table structure accuracy by 6\% with comparable inference speed.
Ranked #1 on
Network Pruning
on CIFAR-100
(Inference Time (ms) metric)
ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding
At first, a document graph is proposed to model complex relationships among multi-grained multimodal elements, in which salient visual regions are detected by a cluster-based method.
Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources
Although more layers and more parameters generally improve the accuracy of the models, such big models generally have high computational complexity and require big memory, which exceed the capacity of small devices for inference and incurs long training time.
HelixFold: An Efficient Implementation of AlphaFold2 using PaddlePaddle
Due to the complex model architecture and large memory consumption, it requires lots of computational resources and time to implement the training and inference of AlphaFold2 from scratch.
PP-OCRv3: More Attempts for the Improvement of Ultra Lightweight OCR System
For text recognizer, the base model is replaced from CRNN to SVTR, and we introduce lightweight text recognition network SVTR LCNet, guided training of CTC by attention, data augmentation strategy TextConAug, better pre-trained model by self-supervised TextRotNet, UDML, and UIM to accelerate the model and improve the effect.
PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkit
PaddleSpeech is an open-source all-in-one speech toolkit.
Automatic Speech Recognition (ASR)
Environmental Sound Classification
+9
SE-MoE: A Scalable and Efficient Mixture-of-Experts Distributed Training and Inference System
With the increasing diversity of ML infrastructures nowadays, distributed training over heterogeneous computing systems is desired to facilitate the production of big models.
Nebula-I: A General Framework for Collaboratively Training Deep Learning Models on Low-Bandwidth Cloud Clusters
We took natural language processing (NLP) as an example to show how Nebula-I works in different training phases that include: a) pre-training a multilingual language model using two remote clusters; and b) fine-tuning a machine translation model using knowledge distilled from pre-trained models, which run through the most popular paradigm of recent deep learning.
Cross-Lingual Natural Language Inference
Distributed Computing
+2
Simple and Effective Relation-based Embedding Propagation for Knowledge Representation Learning
Notably, it averagely brings about 10% relative improvement to triplet-based embedding methods on OGBL-WikiKG2 and takes 5%-83% time to achieve comparable results as the state-of-the-art GC-OTE.
PP-Matting: High-Accuracy Natural Image Matting
Also, we propose a semantic context branch (SCB) that adopts a semantic segmentation subtask.
Ranked #4 on
Image Matting
on Distinctions-646
PP-LiteSeg: A Superior Real-Time Semantic Segmentation Model
Real-world applications have high demands for semantic segmentation methods.
Ranked #4 on
Real-Time Semantic Segmentation
on Cityscapes val
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
A unified framework named ERNIE 3. 0 was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters.
End-to-end Adaptive Distributed Training on PaddlePaddle
The experiments demonstrate that our framework can satisfy various requirements from the diversity of applications and the heterogeneity of resources with highly competitive performance.
HeterPS: Distributed Deep Learning With Reinforcement Learning Based Scheduling in Heterogeneous Environments
The training process generally exploits distributed computing resources to reduce training time.
PP-ShiTu: A Practical Lightweight Image Recognition System
In recent years, image recognition applications have developed rapidly.
PP-PicoDet: A Better Real-Time Object Detector on Mobile Devices
We investigate the applicability of the anchor-free strategy on lightweight object detection models.
Ranked #1 on
Object Detection
on MSCOCO
Exploiting Cross-Modal Prediction and Relation Consistency for Semi-Supervised Image Captioning
In detail, considering that the heterogeneous gap between modalities always leads to the supervision difficulty of using the global embedding directly, CPRC turns to transform both the raw image and corresponding generated sentence into the shared semantic space, and measure the generated sentence from two aspects: 1) Prediction consistency.
PP-LCNet: A Lightweight CPU Convolutional Neural Network
We propose a lightweight CPU network based on the MKLDNN acceleration strategy, named PP-LCNet, which improves the performance of lightweight models on multiple tasks.
PP-OCRv2: Bag of Tricks for Ultra Lightweight OCR System
Optical Character Recognition (OCR) systems have been widely used in various of application scenarios.
Optical Character Recognition
Optical Character Recognition (OCR)
ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation
We trained the model with 10 billion parameters on a 4TB corpus consisting of plain texts and a large-scale knowledge graph.
PP-YOLOv2: A Practical Object Detector
To meet these two concerns, we comprehensively evaluate a collection of existing refinements to improve the performance of PP-YOLO while almost keep the infer time unchanged.
Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones
Recently, research efforts have been concentrated on revealing how pre-trained model makes a difference in neural network performance.
Distilling Knowledge from Pre-trained Language Models via Text Smoothing
As an alternative, we propose a new method for BERT distillation, i. e., asking the teacher to generate smoothed word ids, rather than labels, for teaching the student model in knowledge distillation.
How to Evaluate the Next System: Automatic Dialogue Evaluation from the Perspective of Continual Learning
Automatic dialogue evaluation plays a crucial role in open-domain dialogue research.
RLTM: An Efficient Neural IR Framework for Long Documents
This paper proposes a novel End-to-End neural ranking framework called Reinforced Long Text Matching (RLTM) which matches a query with long documents efficiently and effectively.
Multi-Turn Response Selection for Chatbots with Deep Attention Matching Network
Human generates responses relying on semantic and functional dependencies, including coreference relation, among dialogue elements and their context.
Ranked #6 on
Conversational Response Selection
on RRS
A New Method of Region Embedding for Text Classification
In this paper, we propose a new method of learning and utilizing task-specific distributed representations of n-grams, referred to as “region embeddings”.
