Search Results for author:
Found 46 papers, 18 papers with code
Aligning Coordinated Text Streams through Burst Information Network Construction and Decipherment
We introduce a novel Burst Information Network (BINet) representation that can display the most important information and illustrate the connections among bursty entities, events and keywords in the corpus.
Event Detection with Burst Information Networks
Retrospective event detection is an important task for discovering previously unidentified events in a text stream.
Neural Open Information Extraction
Conventional Open Information Extraction (Open IE) systems are usually built on hand-crafted patterns from other NLP tools such as syntactic parsing, yet they face problems of error propagation.
Retrieval-Enhanced Adversarial Training for Neural Response Generation
Dialogue systems are usually built on either generation-based or retrieval-based approaches, yet they do not benefit from the advantages of different models.
Neural Melody Composition from Lyrics
In this paper, we study a novel task that learns to compose music from natural language.
Unsupervised Machine Commenting with Neural Variational Topic Model
To fully exploit the unpaired data, we completely remove the need for parallel data and propose a novel unsupervised approach to train an automatic article commenting model, relying on nothing but unpaired articles and comments.
LiveBot: Generating Live Video Comments Based on Visual and Textual Contexts
We introduce the task of automatic live commenting.
Fine-grained Coordinated Cross-lingual Text Stream Alignment for Endless Language Knowledge Acquisition
This paper proposes to study fine-grained coordinated cross-lingual text stream alignment through a novel information network decipherment paradigm.
TableBank: A Benchmark Dataset for Table Detection and Recognition
We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet.
Inspecting Unification of Encoding and Matching with Transformer: A Case Study of Machine Reading Comprehension
Most machine reading comprehension (MRC) models separately handle encoding and matching with different network architectures.
Cross-Modality Attention with Semantic Graph Embedding for Multi-Label Classification
In order to overcome these challenges, we propose to use cross-modality attention with semantic graph embedding for multi label classification.
Ranked #8 on
Multi-Label Classification
on NUS-WIDE
LayoutLM: Pre-training of Text and Layout for Document Image Understanding
In this paper, we propose the \textbf{LayoutLM} to jointly model interactions between text and layout information across scanned document images, which is beneficial for a great number of real-world document image understanding tasks such as information extraction from scanned documents.
Ranked #7 on
Relation Extraction
on FUNSD
Multimodal Matching Transformer for Live Commenting
In this work, we aim to improve the relevance between live comments and videos by modeling the cross-modal interactions among different modalities.
TableBank: Table Benchmark for Image-based Table Detection and Recognition
We present TableBank, a new image-based table detection and recognition dataset built with novel weak supervision from Word and Latex documents on the internet.
DocBank: A Benchmark Dataset for Document Layout Analysis
DocBank is constructed using a simple yet effective way with weak supervision from the \LaTeX{} documents available on the arXiv. com.
Unsupervised Fine-tuning for Text Clustering
Fine-tuning with pre-trained language models (e. g. BERT) has achieved great success in many language understanding tasks in supervised settings (e. g. text classification).
LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understanding
Pre-training of text and layout has proved effective in a variety of visually-rich document understanding tasks due to its effective model architecture and the advantage of large-scale unlabeled scanned/digital-born documents.
Ranked #1 on
Key Information Extraction
on SROIE
LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding
In this paper, we present LayoutXLM, a multimodal pre-trained model for multilingual document understanding, which aims to bridge the language barriers for visually-rich document understanding.
Ranked #13 on
Document Image Classification
on RVL-CDIP
AutoSampling: Search for Effective Data Sampling Schedules
Data sampling acts as a pivotal role in training deep learning models.
VT-SSum: A Benchmark Dataset for Video Transcript Segmentation and Summarization
Video transcript summarization is a fundamental task for video understanding.
Glancing at the Patch: Anomaly Localization With Global and Local Feature Comparison
More concretely, we employ a Local-Net and Global-Net to extract features from any individual patch and its surrounding respectively.
LayoutReader: Pre-training of Text and Layout for Reading Order Detection
Reading order detection is the cornerstone to understanding visually-rich documents (e. g., receipts and forms).
Ranked #2 on
Reading Order Detection
on ReadingBank
Document Layout Analysis
Optical Character Recognition (OCR)
+1
TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models
Text recognition is a long-standing research problem for document digitalization.
Ranked #3 on
Handwritten Text Recognition
on IAM
MarkupLM: Pre-training of Text and Markup Language for Visually-rich Document Understanding
Multimodal pre-training with text, layout, and image has made significant progress for Visually Rich Document Understanding (VRDU), especially the fixed-layout documents such as scanned document images.
Document AI: Benchmarks, Models and Applications
Document AI, or Document Intelligence, is a relatively new research topic that refers to the techniques for automatically reading, understanding, and analyzing business documents.
Few-shot Object Counting with Similarity-Aware Feature Enhancement
This work studies the problem of few-shot object counting, which counts the number of exemplar objects (i. e., described by one or several support images) occurring in the query image.
Ranked #2 on
Object Counting
on CARPK
DiT: Self-supervised Pre-training for Document Image Transformer
We leverage DiT as the backbone network in a variety of vision-based Document AI tasks, including document image classification, document layout analysis, table detection as well as text detection for OCR.
Ranked #1 on
Table Detection
on ICDAR 2019
LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking
In this paper, we propose \textbf{LayoutLMv3} to pre-train multimodal Transformers for Document AI with unified text and image masking.
Ranked #1 on
Key Information Extraction
on EPHOIE
Invariant Content Synergistic Learning for Domain Generalization of Medical Image Segmentation
In this paper, we propose a method, named Invariant Content Synergistic Learning (ICSL), to improve the generalization ability of DCNNs on unseen datasets by controlling the inductive bias.
A Unified Model for Multi-class Anomaly Detection
For example, when learning a unified model for 15 categories in MVTec-AD, we surpass the second competitor on the tasks of both anomaly detection (from 88. 1% to 96. 5%) and anomaly localization (from 89. 5% to 96. 8%).
ADTR: Anomaly Detection Transformer with Feature Reconstruction
Second, CNN tends to reconstruct both normal samples and anomalies well, making them still hard to distinguish.
XDoc: Unified Pre-training for Cross-Format Document Understanding
The surge of pre-training has witnessed the rapid development of document understanding recently.
Ranked #7 on
Semantic entity labeling
on FUNSD
Neighbor Regularized Bayesian Optimization for Hyperparameter Optimization
Bayesian Optimization (BO) is a common solution to search optimal hyperparameters based on sample observations of a machine learning model.
Language Is Not All You Need: Aligning Perception with Language Models
A big convergence of language, multimodal perception, action, and world modeling is a key step toward artificial general intelligence.
TextDiffuser: Diffusion Models as Text Painters
Diffusion models have gained increasing attention for their impressive generation abilities but currently struggle with rendering accurate and coherent text.
Kosmos-2.5: A Multimodal Literate Model
We present Kosmos-2. 5, a multimodal literate model for machine reading of text-intensive images.
TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering
The diffusion model has been proven a powerful generative model in recent years, yet remains a challenge in generating visual text.
Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models
Large language models (LLMs) have exhibited impressive performance in language comprehension and various reasoning tasks.
MarkupLM: Pre-training of Text and Markup Language for Visually Rich Document Understanding
Multimodal pre-training with text, layout, and image has made significant progress for Visually Rich Document Understanding (VRDU), especially the fixed-layout documents such as scanned document images.
XFUND: A Benchmark Dataset for Multilingual Visually Rich Form Understanding
Multimodal pre-training with text, layout, and image has achieved SOTA performance for visually rich document understanding tasks recently, which demonstrates the great potential for joint learning across different modalities.
