Search Results for author:
Found 8 papers, 4 papers with code
ERNIE-UniX2: A Unified Cross-lingual Cross-modal Framework for Understanding and Generation
Recent cross-lingual cross-modal works attempt to extend Vision-Language Pre-training (VLP) models to non-English inputs and achieve impressive performance.
Ranked #1 on
Multimodal Machine Translation
on Multi30K
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts
Recent progress in diffusion models has revolutionized the popular technology of text-to-image generation.
Ranked #12 on
Text-to-Image Generation
on MS COCO
ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding
Recent years have witnessed the rise and success of pre-training techniques in visually-rich document understanding.
Ranked #2 on
Semantic entity labeling
on FUNSD
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training
They attempt to learn cross-modal representation using contrastive learning on image-text pairs, however, the built inter-modal correlations only rely on a single view for each modality.
Ranked #1 on
Image Retrieval
on AIC-ICC
ERNIE-mmLayout: Multi-grained MultiModal Transformer for Document Understanding
At first, a document graph is proposed to model complex relationships among multi-grained multimodal elements, in which salient visual regions are detected by a cluster-based method.
ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation
To explore the landscape of large-scale pre-training for bidirectional text-image generation, we train a 10-billion parameter ERNIE-ViLG model on a large-scale dataset of 145 million (Chinese) image-text pairs which achieves state-of-the-art performance for both text-to-image and image-to-text tasks, obtaining an FID of 7. 9 on MS-COCO for text-to-image synthesis and best results on COCO-CN and AIC-ICC for image captioning.
Ranked #42 on
Text-to-Image Generation
on MS COCO
Alpha at SemEval-2021 Task 6: Transformer Based Propaganda Classification
This paper describes our system participated in Task 6 of SemEval-2021: the task focuses on multimodal propaganda technique classification and it aims to classify given image and text into 22 classes.
ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph
Thus, ERNIE-ViL can learn the joint representations characterizing the alignments of the detailed semantics across vision and language.
Ranked #2 on
Visual Question Answering (VQA)
on VCR (QA-R) test
