Search Results for author:
Found 22 papers, 10 papers with code
XFUND: A Benchmark Dataset for Multilingual Visually Rich Form Understanding
Multimodal pre-training with text, layout, and image has achieved SOTA performance for visually rich document understanding tasks recently, which demonstrates the great potential for joint learning across different modalities.
Citrus: Leveraging Expert Cognitive Pathways in a Medical Language Model for Advanced Medical Decision Support
Large language models (LLMs), particularly those with reasoning capabilities, have rapidly advanced in recent years, demonstrating significant potential across a wide range of applications.
JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generation
Specifically, in the first stage, we introduce a decoupled facial representation framework that separates dynamic facial expressions from static 3D facial representations.
JoyType: A Robust Design for Multilingual Visual Text Creation
In this paper, we introduce a novel approach for multilingual visual text creation, named JoyType, designed to maintain the font style of text during the image generation process.
JoyHallo: Digital human model for Mandarin
In audio-driven video generation, creating Mandarin videos presents significant challenges.
MHAD: Multimodal Home Activity Dataset with Multi-Angle Videos and Synchronized Physiological Signals
This non-contact, real-time monitoring method holds great potential for home settings.
ECG Biometric Authentication Using Self-Supervised Learning for IoT Edge Sensors
This paper investigates an electrocardiogram (ECG) based biometric user authentication system using features derived from the Convolutional Neural Network (CNN) and self-supervised contrastive learning.
3SHNet: Boosting Image-Sentence Retrieval via Visual Semantic-Spatial Self-Highlighting
In this paper, we propose a novel visual Semantic-Spatial Self-Highlighting Network (termed 3SHNet) for high-precision, high-efficiency and high-generalization image-sentence retrieval.
Ranked #1 on
Cross-Modal Retrieval
on MSCOCO
PYRA: Parallel Yielding Re-Activation for Training-Inference Efficient Task Adaptation
Consequently, a simple combination of them cannot guarantee accomplishing both training efficiency and inference efficiency with minimal costs.
A Bi-Pyramid Multimodal Fusion Method for the Diagnosis of Bipolar Disorders
Previous research on the diagnosis of Bipolar disorder has mainly focused on resting-state functional magnetic resonance imaging.
Unsupervised Pre-Training Using Masked Autoencoders for ECG Analysis
Unsupervised learning methods have become increasingly important in deep learning due to their demonstrated large utilization of datasets and higher accuracy in computer vision and natural language processing tasks.
Multi-Dimension-Embedding-Aware Modality Fusion Transformer for Psychiatric Disorder Clasification
In this work, we proposed a multi-dimension-embedding-aware modality fusion transformer (MFFormer) for schizophrenia and bipolar disorder classification using rs-fMRI and T1 weighted structural MRI (T1w sMRI).
KOSMOS-2.5: A Multimodal Literate Model
In this paper we present KOSMOS-2. 5, a multimodal literate model for machine reading of text-intensive images.
Unifying Vision, Text, and Layout for Universal Document Processing
UDOP leverages the spatial correlation between textual content and document image to model image, text, and layout modalities with one uniform representation.
Ranked #5 on
Visual Question Answering (VQA)
on InfographicVQA
(using extra training data)
Understanding Long Documents with Different Position-Aware Attentions
Despite several successes in document understanding, the practical task for long document understanding is largely under-explored due to several challenges in computation and how to efficiently absorb long multimodal input.
BoningKnife: Joint Entity Mention Detection and Typing for Nested NER via prior Boundary Knowledge
While named entity recognition (NER) is a key task in natural language processing, most approaches only target flat entities, ignoring nested structures which are common in many scenarios.
Ranked #1 on
Nested Mention Recognition
on ACE 2005
Understanding Chinese Video and Language via Contrastive Multimodal Pre-Training
The pre-trained neural models have recently achieved impressive performances in understanding multimodal content.
LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
In this paper, we present LayoutXLM, a multimodal pre-trained model for multilingual document understanding, which aims to bridge the language barriers for visually-rich document understanding.
Ranked #6 on
Key-value Pair Extraction
on SIBR
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Pre-training of text and layout has proved effective in a variety of visually-rich document understanding tasks due to its effective model architecture and the advantage of large-scale unlabeled scanned/digital-born documents.
Ranked #1 on
Key Information Extraction
on SROIE
Pre-training Graph Transformer with Multimodal Side Information for Recommendation
Side information of items, e. g., images and text description, has shown to be effective in contributing to accurate recommendations.
Enhanced Meta-Learning for Cross-lingual Named Entity Recognition with Minimal Resources
For languages with no annotated resources, transferring knowledge from rich-resource languages is an effective solution for named entity recognition (NER).
Ranked #1 on
Cross-Lingual NER
on MSRA
CAN-NER: Convolutional Attention Network for Chinese Named Entity Recognition
Named entity recognition (NER) in Chinese is essential but difficult because of the lack of natural delimiters.
Ranked #1 on
Chinese Named Entity Recognition
on Weibo NER
(Accuracy-NE metric)
Chinese Named Entity Recognition
named-entity-recognition
+4
