Search Results for author:
Found 42 papers, 17 papers with code
MarkupLM: Pre-training of Text and Markup Language for Visually Rich Document Understanding
Multimodal pre-training with text, layout, and image has made significant progress for Visually Rich Document Understanding (VRDU), especially the fixed-layout documents such as scanned document images.
XFUND: A Benchmark Dataset for Multilingual Visually Rich Form Understanding
Multimodal pre-training with text, layout, and image has achieved SOTA performance for visually rich document understanding tasks recently, which demonstrates the great potential for joint learning across different modalities.
Language Is Not All You Need: Aligning Perception with Language Models
A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence.
Ranked #2 on
Image Captioning
on Flickr30k Captions test
(CIDEr metric)
Neighbor Regularized Bayesian Optimization for Hyperparameter Optimization
Bayesian Optimization (BO) is a common solution to search optimal hyperparameters based on sample observations of a machine learning model.
XDoc: Unified Pre-training for Cross-Format Document Understanding
The surge of pre-training has witnessed the rapid development of document understanding recently.
Ranked #4 on
Semantic entity labeling
on FUNSD
ADTR: Anomaly Detection Transformer with Feature Reconstruction
Second, CNN tends to reconstruct both normal samples and anomalies well, making them still hard to distinguish.
A Unified Model for Multi-class Anomaly Detection
For example, when learning a unified model for 15 categories in MVTec-AD, we surpass the second competitor on the tasks of both anomaly detection (from 88. 1% to 96. 5%) and anomaly localization (from 89. 5% to 96. 8%).
Invariant Content Synergistic Learning for Domain Generalization of Medical Image Segmentation
In this paper, we propose a method, named Invariant Content Synergistic Learning (ICSL), to improve the generalization ability of DCNNs on unseen datasets by controlling the inductive bias.
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
In this paper, we propose \textbf{LayoutLMv3} to pre-train multimodal Transformers for Document AI with unified text and image masking.
Ranked #1 on
Key information extraction
on EPHOIE
DiT: Self-supervised Pre-training for Document Image Transformer
We leverage DiT as the backbone network in a variety of vision-based Document AI tasks, including document image classification, document layout analysis, table detection as well as text detection for OCR.
Ranked #1 on
Table Detection
on cTDaR
Few-shot Object Counting with Similarity-Aware Feature Enhancement
This work studies the problem of few-shot object counting, which counts the number of exemplar objects (i. e., described by one or several support images) occurring in the query image.
Ranked #2 on
Object Counting
on CARPK
Document AI: Benchmarks, Models and Applications
Document AI, or Document Intelligence, is a relatively new research topic that refers to the techniques for automatically reading, understanding, and analyzing business documents.
MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding
Multimodal pre-training with text, layout, and image has made significant progress for Visually Rich Document Understanding (VRDU), especially the fixed-layout documents such as scanned document images.
TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
Text recognition is a long-standing research problem for document digitalization.
LayoutReader: Pre-training of Text and Layout for Reading Order Detection
Reading order detection is the cornerstone to understanding visually-rich documents (e. g., receipts and forms).
Glancing at the Patch: Anomaly Localization With Global and Local Feature Comparison
More concretely, we employ a Local-Net and Global-Net to extract features from any individual patch and its surrounding respectively.
VT-SSum: A Benchmark Dataset for Video Transcript Segmentation and Summarization
Video transcript summarization is a fundamental task for video understanding.
AutoSampling: Search for Effective Data Sampling Schedules
Data sampling acts as a pivotal role in training deep learning models.
LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
In this paper, we present LayoutXLM, a multimodal pre-trained model for multilingual document understanding, which aims to bridge the language barriers for visually-rich document understanding.
Ranked #10 on
Document Image Classification
on RVL-CDIP
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Pre-training of text and layout has proved effective in a variety of visually-rich document understanding tasks due to its effective model architecture and the advantage of large-scale unlabeled scanned/digital-born documents.
Ranked #1 on
Key information extraction
on SROIE
Unsupervised Fine-tuning for Text Clustering
Fine-tuning with pre-trained language models (e. g. BERT) has achieved great success in many language understanding tasks in supervised settings (e. g. text classification).
DocBank: A Benchmark Dataset for Document Layout Analysis
DocBank is constructed using a simple yet effective way with weak supervision from the \LaTeX{} documents available on the arXiv. com.
TableBank: Table Benchmark for Image-based Table Detection and Recognition
We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet.
Multimodal Matching Transformer for Live Commenting
In this work, we aim to improve the relevance between live comments and videos by modeling the cross-modal interactions among different modalities.
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
In this paper, we propose the \textbf{LayoutLM} to jointly model interactions between text and layout information across scanned document images, which is beneficial for a great number of real-world document image understanding tasks such as information extraction from scanned documents.
Ranked #12 on
Document Image Classification
on RVL-CDIP
Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
In order to overcome these challenges, we propose to use cross-modality attention with semantic graph embedding for multi label classification.
Ranked #8 on
Multi-Label Classification
on NUS-WIDE
Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension
Most machine reading comprehension (MRC) models separately handle encoding and matching with different network architectures.
TableBank: A Benchmark Dataset for Table Detection and Recognition
We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet.
Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition
This paper proposes to study fine-grained coordinated cross-lingual text stream alignment through a novel information network decipherment paradigm.
Unsupervised Machine Commenting with Neural Variational Topic Model
To fully exploit the unpaired data, we completely remove the need for parallel data and propose a novel unsupervised approach to train an automatic article commenting model, relying on nothing but unpaired articles and comments.
LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts
We introduce the task of automatic live commenting.
Neural Melody Composition from Lyrics
In this paper, we study a novel task that learns to compose music from natural language.
Retrieval-Enhanced Adversarial Training for Neural Response Generation
Dialogue systems are usually built on either generation-based or retrieval-based approaches, yet they do not benefit from the advantages of different models.
Neural Open Information Extraction
Conventional Open Information Extraction (Open IE) systems are usually built on hand-crafted patterns from other NLP tools such as syntactic parsing, yet they face problems of error propagation.
Event Detection with Burst Information Networks
Retrospective event detection is an important task for discovering previously unidentified events in a text stream.
Aligning Coordinated Text Streams through Burst Information Network Construction and Decipherment
We introduce a novel Burst Information Network (BINet) representation that can display the most important information and illustrate the connections among bursty entities, events and keywords in the corpus.
