Search Results for author:
Found 52 papers, 37 papers with code
TextSquare: Scaling up Text-Centric Visual Instruction Tuning
Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of leading models like GPT4V and Gemini, partly due to a lack of extensive, high-quality instruction tuning data.
Bridging the Gap Between End-to-End and Two-Step Text Spotting
Subsequently, we introduce a Bridge that connects the locked detector and recognizer through a zero-initialized neural network.
OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition
Recently, visually-situated text parsing (VsTP) has experienced notable advancements, driven by the increasing demand for automated document understanding and the emergence of Generative Large Language Models (LLMs) capable of processing document-based questions.
TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document
We present TextMonkey, a large multimodal model (LMM) tailored for text-centric tasks.
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization
In light of recent advances in multimodal Large Language Models (LLMs), there is increasing attention to scaling them from image-text data to more informative real-world videos.
Ranked #59 on
Visual Question Answering
on MM-Vet
An open dataset for oracle bone script recognition and decipherment
Additionally, all images and labels have been reviewed and corrected by experts in oracle bone studies.
An open dataset for the evolution of oracle bone characters: EVOBC
We conducted validation and simulated deciphering on the constructed dataset, and the results demonstrate its high efficacy in aiding the study of oracle bone script.
SwinTextSpotter v2: Towards Better Synergy for Scene Text Spotting
In this paper, we propose a new end-to-end scene text spotting framework termed SwinTextSpotter v2, which seeks to find a better synergy between text detection and recognition.
Progressive Evolution from Single-Point to Polygon for Scene Text
We demonstrate the accuracy of the generated polygons through extensive experiments: 1) By creating polygons from ground truth points, we achieved an accuracy of 82. 0% on ICDAR 2015; 2) In training detectors with polygons generated by our method, we attained 86% of the accuracy relative to training with ground truth (GT); 3) Additionally, the proposed Point2Polygon can be seamlessly integrated to empower single-point spotters to generate polygons.
Toward Real Text Manipulation Detection: New Dataset and New Solution
With the surge in realistic text tampering, detecting fraudulent text in images has gained prominence for maintaining information security.
Enhancing Scene Text Detectors with Realistic Text Image Synthesis Using Diffusion Models
We contend that one main limitation of existing generation methods is the insufficient integration of foreground text with the background.
ML-Bench: Evaluating Large Language Models for Code Generation in Repository-Level Machine Learning Tasks
While Large Language Models (LLMs) have demonstrated proficiency in code generation benchmarks, translating these results into practical development scenarios - where leveraging existing repository-level libraries is the norm - remains challenging.
Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Models
Additionally, experiments on 18 datasets further demonstrate that Monkey surpasses existing LMMs in many tasks like Image Captioning and various Visual Question Answering formats.
KwaiYiiMath: Technical Report
Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on mathematical tasks requiring multi-step reasoning.
Ranked #87 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Turning a CLIP Model into a Scene Text Spotter
Utilizing only 10% of the supervised data, FastTCM-CR50 improves performance by an average of 26. 5% and 5. 5% for text detection and spotting tasks, respectively.
ESTextSpotter: Towards Better Scene Text Spotting with Explicit Synergy in Transformer
To this end, we introduce a new model named Explicit Synergy-based Text Spotting Transformer framework (ESTextSpotter), which achieves explicit synergy by modeling discriminative and interactive features for text detection and recognition within a single decoder.
On Point Affiliation in Feature Upsampling
We introduce the notion of point affiliation into feature upsampling.
Box-DETR: Understanding and Boxing Conditional Spatial Queries
In DAB-DETR, such queries are modulated by the so-called conditional linear projection at each decoder stage, aiming to search for positions of interest such as the four extremities of the box.
ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining
As ViTEraser implicitly integrates text localization and inpainting, we propose a novel end-to-end pretraining method, termed SegMIM, which focuses the encoder and decoder on the text box segmentation and masked image modeling tasks, respectively.
Looking and Listening: Audio Guided Text Recognition
Text recognition in the wild is a long-standing problem in computer vision.
ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images
It is hoped that this competition will attract many researchers in the field of CV and NLP, and bring some new thoughts to the field of Document AI.
On the Hidden Mystery of OCR in Large Multimodal Models
In this paper, we conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks including Text Recognition, Scene Text-Centric Visual Question Answering (VQA), Document-Oriented VQA, Key Information Extraction (KIE), and Handwritten Mathematical Expression Recognition (HMER).
ICDAR 2023 Competition on Reading the Seal Title
To promote research in this area, we organized ICDAR 2023 competition on reading the seal title (ReST), which included two tasks: seal title text detection (Task 1) and end-to-end seal title recognition (Task 2).
Accountable Textual-Visual Chat Learns to Reject Human Instructions in Image Re-creation
This stream is subsequently fed into the decoder-based transformer to generate visual re-creations and textual feedback in the second stage.
Turning a CLIP Model into a Scene Text Detector
Recently, pretraining approaches based on vision language models have made effective progresses in the field of text detection.
Colossal-Auto: Unified Automation of Parallelization and Activation Checkpoint for Large-scale Models
To address these challenges, we introduce a system that can jointly optimize distributed execution and gradient checkpointing plans.
SPTS v2: Single-Point Scene Text Spotting
Within the context of our SPTS v2 framework, our experiments suggest a potential preference for single-point representation in scene text spotting when compared to other representations.
Ranked #15 on
Text Spotting
on ICDAR 2015
Towards Robust Tampered Text Detection in Document Image: New Dataset and New Solution
In this paper, we propose a novel framework to capture more fine-grained clues in complex scenarios for tampered text detection, termed as Document Tampering Detector (DTD), which consists of a Frequency Perception Head (FPH) to compensate the deficiencies caused by the inconspicuous visual features, and a Multi-view Iterative Decoder (MID) for fully utilizing the information of features in different scales.
MSDS: A Large-Scale Chinese Signature and Token Digit String Dataset for Handwriting Verification
MSDS-ChS consists of handwritten Chinese signatures, which, to the best of our knowledge, is the largest publicly available Chinese signature dataset for handwriting verification, at least eight times larger than existing online datasets.
SAPA: Similarity-Aware Point Affiliation for Feature Upsampling
We introduce point affiliation into feature upsampling, a notion that describes the affiliation of each upsampled point to a semantic cluster formed by local decoder feature points with semantic similarity.
Ranked #5 on
Feature Upsampling
on ImageNet
PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition
Utilizing the proposed weakly supervised learning framework, PageNet requires only transcripts to be annotated for real data; however, it can still output detection and recognition results at both the character and line levels, avoiding the labor and cost of labeling bounding boxes of characters and text lines.
Don't Forget Me: Accurate Background Recovery for Text Removal via Modeling Local-Global Context
To address this issue, we propose a Contextual-guided Text Removal Network, termed as CTRNet.
DELTA: Dynamically Optimizing GPU Memory beyond Tensor Recomputation
To the best of our knowledge, we are the first to make a reasonable dynamic runtime scheduler on the combination of tensor swapping and tensor recomputation without user oversight.
SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition
End-to-end scene text spotting has attracted great attention in recent years due to the success of excavating the intrinsic synergy of the scene text detection and recognition.
Ranked #3 on
Text Spotting
on Inverse-Text
SPTS: Single-Point Text Spotting
For the first time, we demonstrate that training scene text spotting models can be achieved with an extremely low-cost annotation of a single-point for each instance.
Ranked #3 on
Text Spotting
on SCUT-CTW1500
Colossal-AI: A Unified Deep Learning System For Large-Scale Parallel Training
The success of Transformer models has pushed the deep learning model scale to billions of parameters.
ICDAR 2021 Competition on Integrated Circuit Text Spotting and Aesthetic Assessment
With hundreds of thousands of electronic chip components are being manufactured every day, chip manufacturers have seen an increasing demand in seeking a more efficient and effective way of inspecting the quality of printed texts on chip components.
ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting
Previous methods can be roughly categorized into two groups: character-based and segmentation-based, which often require character-level annotations and/or complex post-processing due to the unstructured output.
Ranked #7 on
Text Spotting
on Inverse-Text
Structured Multimodal Attentions for TextVQA
In this paper, we propose an end-to-end structured multimodal attention (SMA) neural network to mainly solve the first two issues above.
ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network
Our contributions are three-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve.
Ranked #9 on
Text Spotting
on Inverse-Text
On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering
Visual Question Answering (VQA) methods have made incredible progress, but suffer from a failure to generalize.
Separating Content from Style Using Adversarial Learning for Recognizing Text in the Wild
Furthermore, to tackle the issue of lacking paired training samples, we design an interactive joint training scheme, which shares attention masks from the recognizer to the discriminator, and enables the discriminator to extract the features of each character for further adversarial training.
Exploring the Capacity of an Orderless Box Discretization Network for Multi-orientation Scene Text Detection
More importantly, based on OBD, we provide a detailed analysis of the impact of a collection of refinements, which may inspire others to build state-of-the-art text detectors.
Ranked #3 on
Scene Text Detection
on ICDAR 2017 MLT
ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVT
Robust text reading from street view images provides valuable information for various applications.
ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)
This paper reports the ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT) that consists of three major challenges: i) scene text detection, ii) scene text recognition, and iii) scene text spotting.
Omnidirectional Scene Text Detection with Sequential-free Box Discretization
Scene text in the wild is commonly presented with high variant characteristics.
Ranked #1 on
Scene Text Detection
on IC19-ReCTs
(using extra training data)
Aggregation Cross-Entropy for Sequence Recognition
In this paper, we propose a novel method, aggregation cross-entropy (ACE), for sequence recognition from a brand new perspective.
Tightness-aware Evaluation Protocol for Scene Text Detection
Evaluation protocols play key role in the developmental progress of text detection methods.
EnsNet: Ensconce Text in the Wild
The feature of the former is first enhanced by a novel lateral connection structure and then refined by four carefully designed losses: multiscale regression loss and content loss, which capture the global discrepancy of different level features; texture loss and total variation loss, which primarily target filling the text region and preserving the reality of the background.
DeRPN: Taking a further step toward more general object detection
However, the detection performance is sensitive to the setting of the anchor boxes.
Feature Enhancement Network: A Refined Scene Text Detector
In this paper, we propose a refined scene text detector with a \textit{novel} Feature Enhancement Network (FEN) for Region Proposal and Text Detection Refinement.
Deep Matching Prior Network: Toward Tighter Multi-oriented Text Detection
The effectiveness of our approach is evaluated on a public word-level, multi-oriented scene text database, ICDAR 2015 Robust Reading Competition Challenge 4 "Incidental scene text localization".
