Search Results for author:
Found 95 papers, 46 papers with code
LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding
LiLT can be pre-trained on the structured documents of a single language and then directly fine-tuned on other languages with the corresponding off-the-shelf monolingual/multilingual pre-trained textual models.
Ranked #5 on
Key Information Extraction
on CORD
Learn to Augment: Joint Data Augmentation and Network Optimization for Text Recognition
An agent network learns from the output of the recognition network and controls the fiducial points to generate more proper training samples for the recognition network.
Fourier Contour Embedding for Arbitrary-Shaped Text Detection
One of the main challenges for arbitrary-shaped text detection is to design a good text instance representation that allows networks to learn diverse text geometry variances.
ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network
Our contributions are three-fold: 1) For the first time, we adaptively fit arbitrarily-shaped text by a parameterized Bezier curve.
Ranked #9 on
Text Spotting
on Inverse-Text
Revisiting Scene Text Recognition: A Data Perspective
To this end, we consolidate a large-scale real STR dataset, namely Union14M, which comprises 4 million labeled images and 10 million unlabeled images, to assess the performance of STR models in more complex real-world scenarios.
ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)
This paper reports the ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT) that consists of three major challenges: i) scene text detection, ii) scene text recognition, and iii) scene text spotting.
SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction
Previous works have formulated the recognition of facial beauty as a specific supervised learning problem of classification, regression or ranking, which indicates that FBP is intrinsically a computation problem with multiple paradigms.
Ranked #2 on
Facial Beauty Prediction
on SCUT-FBP
A Multi-Object Rectified Attention Network for Scene Text Recognition
It decreases the difficulty of recognition and enables the attention-based sequence recognition network to more easily read irregular text.
Text Recognition in the Wild: A Survey
This paper aims to (1) summarize the fundamental problems and the state-of-the-art associated with scene text recognition; (2) introduce new insights and ideas; (3) provide a comprehensive review of publicly available resources; (4) point out directions for future work.
Datasets for Large Language Models: A Comprehensive Survey
Additionally, a comprehensive review of the existing available dataset resources is also provided, including statistics from 444 datasets, covering 8 language categories and spanning 32 domains.
Decoupled Attention Network for Text Recognition
To remedy this issue, we propose a decoupled attention network (DAN), which decouples the alignment operation from using historical decoding results.
Ranked #4 on
Scene Text Recognition
on ICDAR 2003
Aggregation Cross-Entropy for Sequence Recognition
In this paper, we propose a novel method, aggregation cross-entropy (ACE), for sequence recognition from a brand new perspective.
On the Hidden Mystery of OCR in Large Multimodal Models
In this paper, we conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks including Text Recognition, Scene Text-Centric Visual Question Answering (VQA), Document-Oriented VQA, Key Information Extraction (KIE), and Handwritten Mathematical Expression Recognition (HMER).
Omnidirectional Scene Text Detection with Sequential-free Box Discretization
Scene text in the wild is commonly presented with high variant characteristics.
Ranked #1 on
Scene Text Detection
on IC19-ReCTs
(using extra training data)
Exploring the Capacity of an Orderless Box Discretization Network for Multi-orientation Scene Text Detection
More importantly, based on OBD, we provide a detailed analysis of the impact of a collection of refinements, which may inspire others to build state-of-the-art text detectors.
Ranked #3 on
Scene Text Detection
on ICDAR 2017 MLT
SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognition
End-to-end scene text spotting has attracted great attention in recent years due to the success of excavating the intrinsic synergy of the scene text detection and recognition.
Ranked #3 on
Text Spotting
on Inverse-Text
ESTextSpotter: Towards Better Scene Text Spotting with Explicit Synergy in Transformer
To this end, we introduce a new model named Explicit Synergy-based Text Spotting Transformer framework (ESTextSpotter), which achieves explicit synergy by modeling discriminative and interactive features for text detection and recognition within a single decoder.
Bridging the Gap Between End-to-End and Two-Step Text Spotting
Subsequently, we introduce a Bridge that connects the locked detector and recognizer through a zero-initialized neural network.
DeepText: A Unified Framework for Text Proposal Generation and Text Detection in Natural Images
In this paper, we develop a novel unified framework called DeepText for text region proposal generation and text detection in natural images via a fully convolutional neural network (CNN).
EnsNet: Ensconce Text in the Wild
The feature of the former is first enhanced by a novel lateral connection structure and then refined by four carefully designed losses: multiscale regression loss and content loss, which capture the global discrepancy of different level features; texture loss and total variation loss, which primarily target filling the text region and preserving the reality of the background.
Tightness-aware Evaluation Protocol for Scene Text Detection
Evaluation protocols play key role in the developmental progress of text detection methods.
FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning
Automatic font generation is an imitation task, which aims to create a font library that mimics the style of reference images while preserving the content from source images.
Scale-Aware Modulation Meet Transformer
Specifically, SMT with 11. 5M / 2. 4GFLOPs and 32M / 7. 7GFLOPs can achieve 82. 2% and 84. 3% top-1 accuracy on ImageNet-1K, respectively.
DeRPN: Taking a further step toward more general object detection
However, the detection performance is sensitive to the setting of the anchor boxes.
SPTS: Single-Point Text Spotting
For the first time, we demonstrate that training scene text spotting models can be achieved with an extremely low-cost annotation of a single-point for each instance.
Ranked #3 on
Text Spotting
on SCUT-CTW1500
SPTS v2: Single-Point Scene Text Spotting
Within the context of our SPTS v2 framework, our experiments suggest a potential preference for single-point representation in scene text spotting when compared to other representations.
Ranked #15 on
Text Spotting
on ICDAR 2015
High Performance Offline Handwritten Chinese Character Recognition Using GoogLeNet and Directional Feature Maps
We design a streamlined version of GoogLeNet [13], which was original proposed for image classification in recent years with very deep architecture, for HCCR (denoted as HCCR-GoogLeNet).
Image Classification
Offline Handwritten Chinese Character Recognition
Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation
We assess the model's performance across a range of OCR tasks, including scene text recognition, handwritten text recognition, handwritten mathematical expression recognition, table structure recognition, and information extraction from visually-rich document.
Handwritten Text Recognition
Optical Character Recognition
+2
Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solution
Visual information extraction (VIE) has attracted considerable attention recently owing to its various advanced applications such as document understanding, automatic marking and intelligent education.
Towards Robust Tampered Text Detection in Document Image: New Dataset and New Solution
In this paper, we propose a novel framework to capture more fine-grained clues in complex scenarios for tampered text detection, termed as Document Tampering Detector (DTD), which consists of a Frequency Perception Head (FPH) to compensate the deficiencies caused by the inconspicuous visual features, and a Multi-view Iterative Decoder (MID) for fully utilizing the information of features in different scales.
Look Closer to Supervise Better: One-Shot Font Generation via Component-Based Discriminator
Automatic font generation remains a challenging research issue due to the large amounts of characters with complicated structures.
M6Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis
Document layout analysis is a crucial prerequisite for document understanding, including document retrieval and conversion.
Don't Forget Me: Accurate Background Recovery for Text Removal via Modeling Local-Global Context
To address this issue, we propose a Contextual-guided Text Removal Network, termed as CTRNet.
PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition
Utilizing the proposed weakly supervised learning framework, PageNet requires only transcripts to be annotated for real data; however, it can still output detection and recognition results at both the character and line levels, avoiding the labor and cost of labeling bounding boxes of characters and text lines.
Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter
Specifically, we integrate IFA into the two most prevailing text recognition streams (attention-based and CTC-based) and propose attention-guided dense prediction (ADP) and Extended CTC (ExCTC).
Optical Character Recognition
Optical Character Recognition (OCR)
+1
ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting
Previous methods can be roughly categorized into two groups: character-based and segmentation-based, which often require character-level annotations and/or complex post-processing due to the unstructured output.
Ranked #7 on
Text Spotting
on Inverse-Text
Joint Layout Analysis, Character Detection and Recognition for Historical Document Digitization
In this framework, two branches named character branch and layout branch are added behind the feature extraction network.
MSDS: A Large-Scale Chinese Signature and Token Digit String Dataset for Handwriting Verification
MSDS-ChS consists of handwritten Chinese signatures, which, to the best of our knowledge, is the largest publicly available Chinese signature dataset for handwriting verification, at least eight times larger than existing online datasets.
SimAN: Exploring Self-Supervised Representation Learning of Scene Text via Similarity-Aware Normalization
Motivated by this common sense, we augment one image patch and use its neighboring patch as guidance to recover itself.
Marior: Margin Removal and Iterative Content Rectification for Document Dewarping in the Wild
To address this issue, we propose a novel approach called Marior (Margin Removal and \Iterative Content Rectification).
ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining
As ViTEraser implicitly integrates text localization and inpainting, we propose a novel end-to-end pretraining method, termed SegMIM, which focuses the encoder and decoder on the text box segmentation and masked image modeling tasks, respectively.
ICDAR 2021 Competition on Integrated Circuit Text Spotting and Aesthetic Assessment
With hundreds of thousands of electronic chip components are being manufactured every day, chip manufacturers have seen an increasing demand in seeking a more efficient and effective way of inspecting the quality of printed texts on chip components.
Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module
In this paper, we first leverage a robust feature descriptor, path signature (PS), and propose three PS features to explicitly represent the spatial and temporal motion characteristics, i. e., spatial PS (S_PS), temporal PS (T_PS) and temporal spatial PS (T_S_PS).
Ranked #1 on
Gesture Recognition
on ChaLearn 2013
SCUT-FBP: A Benchmark Dataset for Facial Beauty Perception
In this paper, a novel face dataset with attractiveness ratings, namely, the SCUT-FBP dataset, is developed for automatic facial beauty perception.
ICDAR 2021 Competition on On-Line Signature Verification
This paper describes the experimental framework and results of the ICDAR 2021 Competition on On-Line Signature Verification (SVC 2021).
SVC-onGoing: Signature Verification Competition
This article presents SVC-onGoing, an on-going competition for on-line signature verification where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases, such as DeepSignDB and SVC2021_EvalDB, and standard experimental protocols.
Detecting Heads using Feature Refine Net and Cascaded Multi-Scale Architecture
To improve the performance of small head detection, we propose a cascaded multi-scale architecture which has two detectors.
Feature Enhancement Network: A Refined Scene Text Detector
In this paper, we propose a refined scene text detector with a \textit{novel} Feature Enhancement Network (FEN) for Region Proposal and Text Detection Refinement.
Developing the Path Signature Methodology and its Application to Landmark-based Human Action Recognition
To this end, we regard the evolving landmark data as a high-dimensional path and apply non-linear path signature techniques to provide an expressive, robust, non-linear, and interpretable representation for the sequential events.
Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition
Online handwritten Chinese text recognition (OHCTR) is a challenging problem as it involves a large-scale character set, ambiguous segmentation, and variable-length input sequences.
Online Signature Verification using Recurrent Neural Network and Length-normalized Path Signature
Inspired by the great success of recurrent neural networks (RNNs) in sequential modeling, we introduce a novel RNN system to improve the performance of online signature verification.
Design of a Very Compact CNN Classifier for Online Handwritten Chinese Character Recognition Using DropWeight and Global Pooling
Currently, owing to the ubiquity of mobile devices, online handwritten Chinese character recognition (HCCR) has become one of the suitable choice for feeding input to cell phones and tablet devices.
DropRegion Training of Inception Font Network for High-Performance Chinese Font Recognition
Chinese font recognition (CFR) has gained significant attention in recent years.
Deep Matching Prior Network: Toward Tighter Multi-oriented Text Detection
The effectiveness of our approach is evaluated on a public word-level, multi-oriented scene text database, ICDAR 2015 Robust Reading Competition Challenge 4 "Incidental scene text localization".
Building Fast and Compact Convolutional Neural Networks for Offline Handwritten Chinese Character Recognition
We design a nine-layer CNN for HCCR consisting of 3, 755 classes, and devise an algorithm that can reduce the networks computational cost by nine times and compress the network to 1/18 of the original size of the baseline model, with only a 0. 21% drop in accuracy.
Toward high-performance online HCCR: a CNN approach with DropDistortion, path signature and spatial stochastic max-pooling
This paper presents an investigation of several techniques that increase the accuracy of online handwritten Chinese character recognition (HCCR).
Fully Convolutional Recurrent Network for Handwritten Chinese Text Recognition
This paper proposes an end-to-end framework, namely fully convolutional recurrent network (FCRN) for handwritten Chinese text recognition (HCTR).
Handwriting Recognition
Handwritten Chinese Text Recognition
+1
Character Proposal Network for Robust Text Extraction
Maximally stable extremal regions (MSER), which is a popular method to generate character proposals/candidates, has shown superior performance in scene text detection.
DeepWriterID: An End-to-end Online Text-independent Writer Identification System
A key feature of DeepWriterID is a new method we are proposing, called DropSegment.
A new humanlike facial attractiveness predictor with cascaded fine-tuning deep learning model
This paper proposes a deep leaning method to address the challenging facial attractiveness prediction problem.
Fingertip in the Eye: A cascaded CNN pipeline for the real-time fingertip detection in egocentric videos
We introduce a new pipeline for hand localization and fingertip detection.
An Open Source Testing Tool for Evaluating Handwriting Input Methods
This paper presents an open source tool for testing the recognition accuracy of Chinese handwriting input methods.
Improved Deep Convolutional Neural Network For Online Handwritten Chinese Character Recognition using Domain-Specific Knowledge
Deep convolutional neural networks (DCNNs) have achieved great success in various computer vision and pattern recognition applications, including those for handwritten Chinese character recognition (HCCR).
Recognition Confidence Analysis of Handwritten Chinese Character with CNN
In this paper, we present an effective method to analyze the recognition confidence of handwritten Chinese character, based on the softmax regression score of a high performance convolutional neural networks (CNN).
DropSample: A New Training Method to Enhance Deep Convolutional Neural Networks for Large-Scale Unconstrained Handwritten Chinese Character Recognition
Inspired by the theory of Leitners learning box from the field of psychology, we propose DropSample, a new method for training deep convolutional neural networks (DCNNs), and apply it to large-scale online handwritten Chinese character recognition (HCCR).
Character-level Chinese Writer Identification using Path Signature Feature, DropStroke and Deep CNN
The results reveal that the path-signature feature is useful for writer identification, and the proposed DropStroke technique enhances the generalization and significantly improves performance.
Logic Rules Powered Knowledge Graph Embedding
And then, to put both triples and mined logic rules within the same semantic space, all triples in the knowledge graph are represented as first-order logic.
Offline Writer Identification based on the Path Signature Feature
In this paper, we propose a novel set of features for offline writer identification based on the path signature approach, which provides a principled way to express information contained in a path.
Adaptive Embedding Gate for Attention-Based Scene Text Recognition
Scene text recognition has attracted particular research interest because it is a very challenging problem and has various applications.
ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVT
Robust text reading from street view images provides valuable information for various applications.
Adaptive GNN for Image Analysis and Editing
In practical verification, we design a new regularization structure with guided feature to produce GNN-based filtering and propagation diffusion to tackle the ill-posed inverse problems of quotient image analysis (QIA), which recovers the reflectance ratio as a signature for image analysis or adjustment.
SynSig2Vec: Learning Representations from Synthetic Dynamic Signatures for Real-world Verification
To tackle this issue, this paper proposes to learn dynamic signature representations through ranking synthesized signatures.
Air-Writing Translater: A Novel Unsupervised Domain Adaptation Method for Inertia-Trajectory Translation of In-air Handwriting
As a new way of human-computer interaction, inertial sensor based in-air handwriting can provide a natural and unconstrained interaction to express more complex and richer information in 3D space.
Separating Content from Style Using Adversarial Learning for Recognizing Text in the Wild
Furthermore, to tackle the issue of lacking paired training samples, we design an interactive joint training scheme, which shares attention masks from the recognizer to the discriminator, and enables the discriminator to extract the features of each character for further adversarial training.
On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering
Visual Question Answering (VQA) methods have made incredible progress, but suffer from a failure to generalize.
Improving Attention-Based Handwritten Mathematical Expression Recognition with Scale Augmentation and Drop Attention
Handwritten mathematical expression recognition (HMER) is an important research direction in handwriting recognition.
RD-GAN: Few/Zero-Shot Chinese Character Style Transfer via Radical Decomposition and Rendering
Style transfer has attracted much interest owing to its various applications.
Towards an efficient framework for Data Extraction from Chart Images
For building a robust point detector, a fully convolutional network with feature fusion module is adopted, which can distinguish close points compared to traditional methods.
Tag, Copy or Predict: A Unified Weakly-Supervised Learning Framework for Visual Information Extraction using Sequences
Visual information extraction (VIE) has attracted increasing attention in recent years.
Optical Character Recognition
Optical Character Recognition (OCR)
+2
MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction
Through key-value matching based on relevancy evaluation, the proposed MatchVIE can bypass the recognitions to various semantics, and simply focuses on the strong relevancy between entities.
SLOGAN: Handwriting Style Synthesis for Arbitrary-Length and Out-of-Vocabulary Text
Specifically, we propose a style bank to parameterize the specific handwriting styles as latent vectors, which are input to a generator as style priors to achieve the corresponding handwritten styles.
Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understanding
Multi-modal document pre-trained models have proven to be very effective in a variety of visually-rich document understanding (VrDU) tasks.
Recognition of Handwritten Chinese Text by Segmentation: A Segment-annotation-free Approach
A novel weakly supervised learning method is proposed to enable the network to be trained using only transcript annotations; thus, the expensive character segmentation annotations required by previous segmentation-based methods can be avoided.
M$^{6}$Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysis
Document layout analysis is a crucial prerequisite for document understanding, including document retrieval and conversion.
ConaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval
Large-scale pre-trained text-image models with dual-encoder architectures (such as CLIP) are typically adopted for various vision-language applications, including text-image retrieval.
DocAligner: Annotating Real-world Photographic Document Images by Simply Taking Pictures
Furthermore, considering the importance of fine-grained elements in document images, we present a details recurrent refinement module to enhance the output in a high-resolution space.
Hierarchical Side-Tuning for Vision Transformers
Fine-tuning pre-trained Vision Transformers (ViT) has consistently demonstrated promising performance in the realm of visual recognition.
UPOCR: Towards Unified Pixel-Level OCR Interface
Without bells and whistles, the experimental results showcase that the proposed method can simultaneously achieve state-of-the-art performance on three tasks with a unified single model, which provides valuable strategies and insights for future research on generalist OCR models.
Optical Character Recognition
Optical Character Recognition (OCR)
+2
Progressive Evolution from Single-Point to Polygon for Scene Text
We demonstrate the accuracy of the generated polygons through extensive experiments: 1) By creating polygons from ground truth points, we achieved an accuracy of 82. 0% on ICDAR 2015; 2) In training detectors with polygons generated by our method, we attained 86% of the accuracy relative to training with ground truth (GT); 3) Additionally, the proposed Point2Polygon can be seamlessly integrated to empower single-point spotters to generate polygons.
PEneo: Unifying Line Extraction, Line Grouping, and Entity Linking for End-to-end Document Pair Extraction
However, simply concatenating SER and RE serially can lead to severe error propagation, and it fails to handle cases like multi-line entities in real scenarios.
SwinTextSpotter v2: Towards Better Synergy for Scene Text Spotting
In this paper, we propose a new end-to-end scene text spotting framework termed SwinTextSpotter v2, which seeks to find a better synergy between text detection and recognition.
An open dataset for the evolution of oracle bone characters: EVOBC
We conducted validation and simulated deciphering on the constructed dataset, and the results demonstrate its high efficacy in aiding the study of oracle bone script.
An open dataset for oracle bone script recognition and decipherment
Additionally, all images and labels have been reviewed and corrected by experts in oracle bone studies.
DiffChat: Learning to Chat with Text-to-Image Synthesis Models for Interactive Image Creation
We present DiffChat, a novel method to align Large Language Models (LLMs) to "chat" with prompt-as-input Text-to-Image Synthesis (TIS) models (e. g., Stable Diffusion) for interactive image creation.
HierCode: A Lightweight Hierarchical Codebook for Zero-shot Chinese Text Recognition
Text recognition, especially for complex scripts like Chinese, faces unique challenges due to its intricate character structures and vast vocabulary.
