Search Results for author: Lianwen Jin

Found 110 papers, 56 papers with code

Omni-IML: Towards Unified Image Manipulation Localization

no code implementations22 Nov 2024 Chenfan Qu, Yiwu Zhong, Fengjun Guo, Lianwen Jin

To this end, we propose Omni-IML, the first generalist model to unify diverse IML tasks.

Decoder Image Forensics +2

Mini-Monkey: Alleviating the Semantic Sawtooth Effect for Lightweight MLLMs via Complementary Image Pyramid

1 code implementation4 Aug 2024 Mingxin Huang, Yuliang Liu, Dingkang Liang, Lianwen Jin, Xiang Bai

To address this issue, we introduce a Complementary Image Pyramid (CIP), a simple, effective, and plug-and-play solution designed to mitigate semantic discontinuity during high-resolution image processing.

document understanding

LEGO: Self-Supervised Representation Learning for Scene Text Images

no code implementations4 Aug 2024 Yujin Ren, Jiaxin Zhang, Lianwen Jin

To tackle this problem, a highly promising approach is to utilize massive amounts of unlabeled real data for self-supervised training, which has been widely proven effective in many NLP and CV tasks.

Representation Learning Scene Text Recognition

Generalized Tampered Scene Text Detection in the era of Generative AI

no code implementations31 Jul 2024 Chenfan Qu, Yiwu Zhong, Fengjun Guo, Lianwen Jin

To tackle this, we propose a novel task: open-set tampered scene text detection, which evaluates forensics models on their ability to identify both seen and previously unseen forgery types.

Misinformation Scene Text Detection +1

TongGu: Mastering Classical Chinese Understanding with Knowledge-Grounded Large Language Models

1 code implementation4 Jul 2024 Jiahuan Cao, Dezhi Peng, Peirong Zhang, Yongxin Shi, Yang Liu, Kai Ding, Lianwen Jin

Classical Chinese is a gateway to the rich heritage and wisdom of ancient China, yet its complexities pose formidable comprehension barriers for most modern people without specialized knowledge.

RAG

DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming

no code implementations27 Jun 2024 Jiaxin Zhang, Wentao Yang, Songxuan Lai, Zecheng Xie, Lianwen Jin

Current multimodal large language models (MLLMs) face significant challenges in visual document understanding (VDU) tasks due to the high resolution, dense text, and complex layouts typical of document images.

document understanding

Deciphering Oracle Bone Language with Diffusion Models

1 code implementation2 Jun 2024 Haisu Guan, Huanxin Yang, Xinyu Wang, Shengwei Han, Yongge Liu, Lianwen Jin, Xiang Bai, Yuliang Liu

Originating from China's Shang Dynasty approximately 3, 000 years ago, the Oracle Bone Script (OBS) is a cornerstone in the annals of linguistic history, predating many established writing systems.

Decipherment Image Generation

C$^{3}$Bench: A Comprehensive Classical Chinese Understanding Benchmark for Large Language Models

no code implementations28 May 2024 Jiahuan Cao, Yongxin Shi, Dezhi Peng, Yang Liu, Lianwen Jin

To fill this gap, this paper introduces C$^{3}$bench, a Comprehensive Classical Chinese understanding benchmark, which comprises 50, 000 text pairs for five primary CCU tasks, including classification, retrieval, named entity recognition, punctuation, and translation.

named-entity-recognition Named Entity Recognition

VimTS: A Unified Video and Image Text Spotter for Enhancing the Cross-domain Generalization

1 code implementation30 Apr 2024 Yuliang Liu, Mingxin Huang, Hao Yan, Linger Deng, Weijia Wu, Hao Lu, Chunhua Shen, Lianwen Jin, Xiang Bai

Typically, we propose a Prompt Queries Generation Module and a Tasks-aware Adapter to effectively convert the original single-task model into a multi-task model suitable for both image and video scenarios with minimal additional parameters.

Domain Generalization Text Spotting

Bridging the Gap Between End-to-End and Two-Step Text Spotting

4 code implementations CVPR 2024 Mingxin Huang, Hongliang Li, Yuliang Liu, Xiang Bai, Lianwen Jin

Subsequently, we introduce a Bridge that connects the locked detector and recognizer through a zero-initialized neural network.

Text Spotting

HierCode: A Lightweight Hierarchical Codebook for Zero-shot Chinese Text Recognition

no code implementations20 Mar 2024 Yuyi Zhang, Yuanzhi Zhu, Dezhi Peng, Peirong Zhang, Zhenhua Yang, Zhibo Yang, Cong Yao, Lianwen Jin

Text recognition, especially for complex scripts like Chinese, faces unique challenges due to its intricate character structures and vast vocabulary.

Zero-Shot Learning

DiffChat: Learning to Chat with Text-to-Image Synthesis Models for Interactive Image Creation

no code implementations8 Mar 2024 Jiapeng Wang, Chengyu Wang, Tingfeng Cao, Jun Huang, Lianwen Jin

We present DiffChat, a novel method to align Large Language Models (LLMs) to "chat" with prompt-as-input Text-to-Image Synthesis (TIS) models (e. g., Stable Diffusion) for interactive image creation.

Image Generation Instruction Following +1

Datasets for Large Language Models: A Comprehensive Survey

1 code implementation28 Feb 2024 Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, Lianwen Jin

Additionally, a comprehensive review of the existing available dataset resources is also provided, including statistics from 444 datasets, covering 8 language categories and spanning 32 domains.

Language Modelling Large Language Model +1

An open dataset for oracle bone script recognition and decipherment

2 code implementations27 Jan 2024 Pengjie Wang, Kaile Zhang, Xinyu Wang, Shengwei Han, Yongge Liu, Jinpeng Wan, Haisu Guan, Zhebin Kuang, Lianwen Jin, Xiang Bai, Yuliang Liu

Oracle bone script, one of the earliest known forms of ancient Chinese writing, presents invaluable research materials for scholars studying the humanities and geography of the Shang Dynasty, dating back 3, 000 years.

Decipherment

An open dataset for the evolution of oracle bone characters: EVOBC

no code implementations23 Jan 2024 Haisu Guan, Jinpeng Wan, Yuliang Liu, Pengjie Wang, Kaile Zhang, Zhebin Kuang, Xinyu Wang, Xiang Bai, Lianwen Jin

We conducted validation and simulated deciphering on the constructed dataset, and the results demonstrate its high efficacy in aiding the study of oracle bone script.

Decipherment

SwinTextSpotter v2: Towards Better Synergy for Scene Text Spotting

no code implementations15 Jan 2024 Mingxin Huang, Dezhi Peng, Hongliang Li, Zhenghao Peng, Chongyu Liu, Dahua Lin, Yuliang Liu, Xiang Bai, Lianwen Jin

In this paper, we propose a new end-to-end scene text spotting framework termed SwinTextSpotter v2, which seeks to find a better synergy between text detection and recognition.

Text Detection Text Spotting

PEneo: Unifying Line Extraction, Line Grouping, and Entity Linking for End-to-end Document Pair Extraction

1 code implementation7 Jan 2024 Zening Lin, Jiapeng Wang, Teng Li, Wenhui Liao, Dayi Huang, Longfei Xiong, Lianwen Jin

However, simply concatenating SER and RE serially can lead to severe error propagation, and it fails to handle cases like multi-line entities in real scenarios.

Key-value Pair Extraction Relation Extraction +1

Progressive Evolution from Single-Point to Polygon for Scene Text

no code implementations21 Dec 2023 Linger Deng, Mingxin Huang, Xudong Xie, Yuliang Liu, Lianwen Jin, Xiang Bai

We demonstrate the accuracy of the generated polygons through extensive experiments: 1) By creating polygons from ground truth points, we achieved an accuracy of 82. 0% on ICDAR 2015; 2) In training detectors with polygons generated by our method, we attained 86% of the accuracy relative to training with ground truth (GT); 3) Additionally, the proposed Point2Polygon can be seamlessly integrated to empower single-point spotters to generate polygons.

Text Detection

FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning

1 code implementation19 Dec 2023 Zhenhua Yang, Dezhi Peng, Yuxin Kong, Yuyi Zhang, Cong Yao, Lianwen Jin

Automatic font generation is an imitation task, which aims to create a font library that mimics the style of reference images while preserving the content from source images.

Contrastive Learning Denoising +3

UPOCR: Towards Unified Pixel-Level OCR Interface

no code implementations5 Dec 2023 Dezhi Peng, Zhenhua Yang, Jiaxin Zhang, Chongyu Liu, Yongxin Shi, Kai Ding, Fengjun Guo, Lianwen Jin

Without bells and whistles, the experimental results showcase that the proposed method can simultaneously achieve state-of-the-art performance on three tasks with a unified single model, which provides valuable strategies and insights for future research on generalist OCR models.

Decoder Optical Character Recognition +3

Exploring OCR Capabilities of GPT-4V(ision) : A Quantitative and In-depth Evaluation

1 code implementation25 Oct 2023 Yongxin Shi, Dezhi Peng, Wenhui Liao, Zening Lin, Xinhong Chen, Chongyu Liu, Yuyi Zhang, Lianwen Jin

We assess the model's performance across a range of OCR tasks, including scene text recognition, handwritten text recognition, handwritten mathematical expression recognition, table structure recognition, and information extraction from visually-rich document.

Handwritten Text Recognition Key Information Extraction +3

Hierarchical Side-Tuning for Vision Transformers

no code implementations9 Oct 2023 Weifeng Lin, Ziheng Wu, Wentao Yang, Mingxin Huang, Jun Huang, Lianwen Jin

In this paper, we introduce Hierarchical Side-Tuning (HST), an innovative PETL method facilitating the transfer of ViT models to diverse downstream tasks.

Image Classification Instance Segmentation +5

ESTextSpotter: Towards Better Scene Text Spotting with Explicit Synergy in Transformer

3 code implementations ICCV 2023 Mingxin Huang, Jiaxin Zhang, Dezhi Peng, Hao Lu, Can Huang, Yuliang Liu, Xiang Bai, Lianwen Jin

To this end, we introduce a new model named Explicit Synergy-based Text Spotting Transformer framework (ESTextSpotter), which achieves explicit synergy by modeling discriminative and interactive features for text detection and recognition within a single decoder.

Decoder Text Detection +1

Revisiting Scene Text Recognition: A Data Perspective

1 code implementation ICCV 2023 Qing Jiang, Jiapeng Wang, Dezhi Peng, Chongyu Liu, Lianwen Jin

To this end, we consolidate a large-scale real STR dataset, namely Union14M, which comprises 4 million labeled images and 10 million unlabeled images, to assess the performance of STR models in more complex real-world scenarios.

Scene Text Recognition

Scale-Aware Modulation Meet Transformer

1 code implementation ICCV 2023 Weifeng Lin, Ziheng Wu, Jiayu Chen, Jun Huang, Lianwen Jin

Specifically, SMT with 11. 5M / 2. 4GFLOPs and 32M / 7. 7GFLOPs can achieve 82. 2% and 84. 3% top-1 accuracy on ImageNet-1K, respectively.

object-detection Object Detection +1

ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining

1 code implementation21 Jun 2023 Dezhi Peng, Chongyu Liu, Yuliang Liu, Lianwen Jin

As ViTEraser implicitly integrates text localization and inpainting, we propose a novel end-to-end pretraining method, termed SegMIM, which focuses the encoder and decoder on the text box segmentation and masked image modeling tasks, respectively.

Decoder Long-range modeling +2

DocAligner: Annotating Real-world Photographic Document Images by Simply Taking Pictures

no code implementations9 Jun 2023 Jiaxin Zhang, Bangdong Chen, Hiuyi Cheng, Fengjun Guo, Kai Ding, Lianwen Jin

Furthermore, considering the importance of fine-grained elements in document images, we present a details recurrent refinement module to enhance the output in a high-resolution space.

Self-Supervised Learning

ConaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval

no code implementations28 May 2023 Jiapeng Wang, Chengyu Wang, Xiaodan Wang, Jun Huang, Lianwen Jin

Large-scale pre-trained text-image models with dual-encoder architectures (such as CLIP) are typically adopted for various vision-language applications, including text-image retrieval.

Image Retrieval Knowledge Distillation +2

OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models

1 code implementation13 May 2023 Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, XuCheng Yin, Cheng-Lin Liu, Lianwen Jin, Xiang Bai

In this paper, we conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks including Text Recognition, Scene Text-Centric Visual Question Answering (VQA), Document-Oriented VQA, Key Information Extraction (KIE), and Handwritten Mathematical Expression Recognition (HMER).

Key Information Extraction Nutrition +4

SPTS v2: Single-Point Scene Text Spotting

3 code implementations4 Jan 2023 Yuliang Liu, Jiaxin Zhang, Dezhi Peng, Mingxin Huang, Xinyu Wang, Jingqun Tang, Can Huang, Dahua Lin, Chunhua Shen, Xiang Bai, Lianwen Jin

Within the context of our SPTS v2 framework, our experiments suggest a potential preference for single-point representation in scene text spotting when compared to other representations.

Decoder Text Detection +1

Towards Robust Tampered Text Detection in Document Image: New Dataset and New Solution

1 code implementation CVPR 2023 Chenfan Qu, Chongyu Liu, Yuliang Liu, Xinhong Chen, Dezhi Peng, Fengjun Guo, Lianwen Jin

In this paper, we propose a novel framework to capture more fine-grained clues in complex scenarios for tampered text detection, termed as Document Tampering Detector (DTD), which consists of a Frequency Perception Head (FPH) to compensate the deficiencies caused by the inconspicuous visual features, and a Multi-view Iterative Decoder (MID) for fully utilizing the information of features in different scales.

Decoder Image and Video Forgery Detection +2

MSDS: A Large-Scale Chinese Signature and Token Digit String Dataset for Handwriting Verification

1 code implementation17 Oct 2022 Peirong Zhang, Jiajia Jiang, Yuliang Liu, Lianwen Jin

MSDS-ChS consists of handwritten Chinese signatures, which, to the best of our knowledge, is the largest publicly available Chinese signature dataset for handwriting verification, at least eight times larger than existing online datasets.

Handwriting Verification

Recognition of Handwritten Chinese Text by Segmentation: A Segment-annotation-free Approach

no code implementations29 Jul 2022 Dezhi Peng, Lianwen Jin, Weihong Ma, Canyu Xie, Hesuo Zhang, Shenggao Zhu, Jing Li

A novel weakly supervised learning method is proposed to enable the network to be trained using only transcript annotations; thus, the expensive character segmentation annotations required by previous segmentation-based methods can be avoided.

Handwritten Chinese Text Recognition Segmentation +1

PageNet: Towards End-to-End Weakly Supervised Page-Level Handwritten Chinese Text Recognition

4 code implementations29 Jul 2022 Dezhi Peng, Lianwen Jin, Yuliang Liu, Canjie Luo, Songxuan Lai

Utilizing the proposed weakly supervised learning framework, PageNet requires only transcripts to be annotated for real data; however, it can still output detection and recognition results at both the character and line levels, avoiding the labor and cost of labeling bounding boxes of characters and text lines.

Handwritten Chinese Text Recognition Line Detection +1

Marior: Margin Removal and Iterative Content Rectification for Document Dewarping in the Wild

1 code implementation23 Jul 2022 Jiaxin Zhang, Canjie Luo, Lianwen Jin, Fengjun Guo, Kai Ding

To address this issue, we propose a novel approach called Marior (Margin Removal and \Iterative Content Rectification).

Optical Character Recognition (OCR)

Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understanding

no code implementations27 Jun 2022 Chuwei Luo, Guozhi Tang, Qi Zheng, Cong Yao, Lianwen Jin, Chenliang Li, Yang Xue, Luo Si

Multi-modal document pre-trained models have proven to be very effective in a variety of visually-rich document understanding (VrDU) tasks.

Document Classification document understanding +2

LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding

4 code implementations ACL 2022 Jiapeng Wang, Lianwen Jin, Kai Ding

LiLT can be pre-trained on the structured documents of a single language and then directly fine-tuned on other languages with the corresponding off-the-shelf monolingual/multilingual pre-trained textual models.

Document Image Classification document understanding +2

SLOGAN: Handwriting Style Synthesis for Arbitrary-Length and Out-of-Vocabulary Text

no code implementations23 Feb 2022 Canjie Luo, Yuanzhi Zhu, Lianwen Jin, Zhe Li, Dezhi Peng

Specifically, we propose a style bank to parameterize the specific handwriting styles as latent vectors, which are input to a generator as style priors to achieve the corresponding handwritten styles.

Attribute Diversity +1

SPTS: Single-Point Text Spotting

1 code implementation15 Dec 2021 Dezhi Peng, Xinyu Wang, Yuliang Liu, Jiaxin Zhang, Mingxin Huang, Songxuan Lai, Shenggao Zhu, Jing Li, Dahua Lin, Chunhua Shen, Xiang Bai, Lianwen Jin

For the first time, we demonstrate that training scene text spotting models can be achieved with an extremely low-cost annotation of a single-point for each instance.

Language Modelling Text Detection +1

SVC-onGoing: Signature Verification Competition

1 code implementation13 Aug 2021 Ruben Tolosana, Ruben Vera-Rodriguez, Carlos Gonzalez-Garcia, Julian Fierrez, Aythami Morales, Javier Ortega-Garcia, Juan Carlos Ruiz-Garcia, Sergio Romero-Tapiador, Santiago Rengifo, Miguel Caruana, Jiajia Jiang, Songxuan Lai, Lianwen Jin, Yecheng Zhu, Javier Galbally, Moises Diaz, Miguel Angel Ferrer, Marta Gomez-Barrero, Ilya Hodashinsky, Konstantin Sarin, Artem Slezkin, Marina Bardamova, Mikhail Svetlakov, Mohammad Saleem, Cintia Lia Szucs, Bence Kovari, Falk Pulsmeyer, Mohamad Wehbi, Dario Zanca, Sumaiya Ahmad, Sarthak Mishra, Suraiya Jabin

This article presents SVC-onGoing, an on-going competition for on-line signature verification where researchers can easily benchmark their systems against the state of the art in an open common platform using large-scale public databases, such as DeepSignDB and SVC2021_EvalDB, and standard experimental protocols.

Task 2

ICDAR 2021 Competition on Integrated Circuit Text Spotting and Aesthetic Assessment

1 code implementation12 Jul 2021 Chun Chet Ng, Akmalul Khairi Bin Nazaruddin, Yeong Khang Lee, Xinyu Wang, Yuliang Liu, Chee Seng Chan, Lianwen Jin, Yipeng Sun, Lixin Fan

With hundreds of thousands of electronic chip components are being manufactured every day, chip manufacturers have seen an increasing demand in seeking a more efficient and effective way of inspecting the quality of printed texts on chip components.

Text Spotting

MatchVIE: Exploiting Match Relevancy between Entities for Visual Information Extraction

no code implementations24 Jun 2021 Guozhi Tang, Lele Xie, Lianwen Jin, Jiapeng Wang, Jingdong Chen, Zhen Xu, Qianying Wang, Yaqiang Wu, Hui Li

Through key-value matching based on relevancy evaluation, the proposed MatchVIE can bypass the recognitions to various semantics, and simply focuses on the strong relevancy between entities.

Graph Neural Network

Implicit Feature Alignment: Learn to Convert Text Recognizer to Text Spotter

1 code implementation CVPR 2021 Tianwei Wang, Yuanzhi Zhu, Lianwen Jin, Dezhi Peng, Zhe Li, Mengchao He, Yongpan Wang, Canjie Luo

Specifically, we integrate IFA into the two most prevailing text recognition streams (attention-based and CTC-based) and propose attention-guided dense prediction (ADP) and Extended CTC (ExCTC).

Optical Character Recognition Optical Character Recognition (OCR) +1

ABCNet v2: Adaptive Bezier-Curve Network for Real-time End-to-end Text Spotting

1 code implementation8 May 2021 Yuliang Liu, Chunhua Shen, Lianwen Jin, Tong He, Peng Chen, Chongyu Liu, Hao Chen

Previous methods can be roughly categorized into two groups: character-based and segmentation-based, which often require character-level annotations and/or complex post-processing due to the unstructured output.

Text Spotting

Towards an efficient framework for Data Extraction from Chart Images

no code implementations5 May 2021 Weihong Ma, Hesuo Zhang, Shuang Yan, Guangshun Yao, Yichao Huang, Hui Li, Yaqiang Wu, Lianwen Jin

For building a robust point detector, a fully convolutional network with feature fusion module is adopted, which can distinguish close points compared to traditional methods.

Fourier Contour Embedding for Arbitrary-Shaped Text Detection

8 code implementations CVPR 2021 2021 Yiqin Zhu, Jianyong Chen, Lingyu Liang, Zhanghui Kuang, Lianwen Jin, Wayne Zhang

One of the main challenges for arbitrary-shaped text detection is to design a good text instance representation that allows networks to learn diverse text geometry variances.

Scene Text Detection Text Detection

Towards Robust Visual Information Extraction in Real World: New Dataset and Novel Solution

1 code implementation24 Jan 2021 Jiapeng Wang, Chongyu Liu, Lianwen Jin, Guozhi Tang, Jiaxin Zhang, Shuaitao Zhang, Qianying Wang, Yaqiang Wu, Mingxiang Cai

Visual information extraction (VIE) has attracted considerable attention recently owing to its various advanced applications such as document understanding, automatic marking and intelligent education.

3D Feature Matching document understanding +2

Joint Layout Analysis, Character Detection and Recognition for Historical Document Digitization

1 code implementation14 Jul 2020 Weihong Ma, Hesuo Zhang, Lianwen Jin, Sihang Wu, Jiapeng Wang, Yongpan Wang

In this framework, two branches named character branch and layout branch are added behind the feature extraction network.

Line Detection

Text Recognition in the Wild: A Survey

1 code implementation7 May 2020 Xiaoxue Chen, Lianwen Jin, Yuanzhi Zhu, Canjie Luo, Tianwei Wang

This paper aims to (1) summarize the fundamental problems and the state-of-the-art associated with scene text recognition; (2) introduce new insights and ideas; (3) provide a comprehensive review of publicly available resources; (4) point out directions for future work.

Scene Text Recognition Survey

Learn to Augment: Joint Data Augmentation and Network Optimization for Text Recognition

3 code implementations CVPR 2020 Canjie Luo, Yuanzhi Zhu, Lianwen Jin, Yongpan Wang

An agent network learns from the output of the recognition network and controls the fiducial points to generate more proper training samples for the recognition network.

Diversity Image Augmentation

Separating Content from Style Using Adversarial Learning for Recognizing Text in the Wild

no code implementations13 Jan 2020 Canjie Luo, Qingxiang Lin, Yuliang Liu, Lianwen Jin, Chunhua Shen

Furthermore, to tackle the issue of lacking paired training samples, we design an interactive joint training scheme, which shares attention masks from the recognizer to the discriminator, and enables the discriminator to extract the features of each character for further adversarial training.

Style Transfer

Decoupled Attention Network for Text Recognition

4 code implementations21 Dec 2019 Tianwei Wang, Yuanzhi Zhu, Lianwen Jin, Canjie Luo, Xiaoxue Chen, Yaqiang Wu, Qianying Wang, Mingxiang Cai

To remedy this issue, we propose a decoupled attention network (DAN), which decouples the alignment operation from using historical decoding results.

Decoder Handwritten Text Recognition +1

Exploring the Capacity of an Orderless Box Discretization Network for Multi-orientation Scene Text Detection

1 code implementation20 Dec 2019 Yuliang Liu, Tong He, Hao Chen, Xinyu Wang, Canjie Luo, Shuaitao Zhang, Chunhua Shen, Lianwen Jin

More importantly, based on OBD, we provide a detailed analysis of the impact of a collection of refinements, which may inspire others to build state-of-the-art text detectors.

Scene Text Detection Text Detection

Adaptive GNN for Image Analysis and Editing

no code implementations NeurIPS 2019 Lingyu Liang, Lianwen Jin, Yong Xu

In practical verification, we design a new regularization structure with guided feature to produce GNN-based filtering and propagation diffusion to tackle the ill-posed inverse problems of quotient image analysis (QIA), which recovers the reflectance ratio as a signature for image analysis or adjustment.

Graph Neural Network Low-Light Image Enhancement

SynSig2Vec: Learning Representations from Synthetic Dynamic Signatures for Real-world Verification

no code implementations13 Nov 2019 Songxuan Lai, Lianwen Jin, Luojun Lin, Yecheng Zhu, Huiyun Mao

To tackle this issue, this paper proposes to learn dynamic signature representations through ranking synthesized signatures.

Representation Learning

Air-Writing Translater: A Novel Unsupervised Domain Adaptation Method for Inertia-Trajectory Translation of In-air Handwriting

no code implementations1 Nov 2019 Songbin Xu, Yang Xue, Xin Zhang, Lianwen Jin

As a new way of human-computer interaction, inertial sensor based in-air handwriting can provide a natural and unconstrained interaction to express more complex and richer information in 3D space.

Translation Unsupervised Domain Adaptation

ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)

1 code implementation16 Sep 2019 Chee-Kheng Chng, Yuliang Liu, Yipeng Sun, Chun Chet Ng, Canjie Luo, Zihan Ni, ChuanMing Fang, Shuaitao Zhang, Junyu Han, Errui Ding, Jingtuo Liu, Dimosthenis Karatzas, Chee Seng Chan, Lianwen Jin

This paper reports the ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT) that consists of three major challenges: i) scene text detection, ii) scene text recognition, and iii) scene text spotting.

Scene Text Detection Scene Text Recognition +2

Adaptive Embedding Gate for Attention-Based Scene Text Recognition

no code implementations26 Aug 2019 Xiaoxue Chen, Tianwei Wang, Yuanzhi Zhu, Lianwen Jin, Canjie Luo

Scene text recognition has attracted particular research interest because it is a very challenging problem and has various applications.

Decoder Scene Text Recognition

Offline Writer Identification based on the Path Signature Feature

no code implementations3 May 2019 Songxuan Lai, Lianwen Jin

In this paper, we propose a novel set of features for offline writer identification based on the path signature approach, which provides a principled way to express information contained in a path.

Aggregation Cross-Entropy for Sequence Recognition

2 code implementations CVPR 2019 Zecheng Xie, Yaoxiong Huang, Yuanzhi Zhu, Lianwen Jin, Yuliang Liu, Lele Xie

In this paper, we propose a novel method, aggregation cross-entropy (ACE), for sequence recognition from a brand new perspective.

Logic Rules Powered Knowledge Graph Embedding

no code implementations9 Mar 2019 Pengwei Wang, Dejing Dou, Fangzhao Wu, Nisansa de Silva, Lianwen Jin

And then, to put both triples and mined logic rules within the same semantic space, all triples in the knowledge graph are represented as first-order logic.

Knowledge Graph Embedding Link Prediction +1

EnsNet: Ensconce Text in the Wild

3 code implementations3 Dec 2018 Shuaitao Zhang, Yuliang Liu, Lianwen Jin, Yaoxiong Huang, Songxuan Lai

The feature of the former is first enhanced by a novel lateral connection structure and then refined by four carefully designed losses: multiscale regression loss and content loss, which capture the global discrepancy of different level features; texture loss and total variation loss, which primarily target filling the text region and preserving the reality of the background.

Generative Adversarial Network Image Text Removal

Skeleton-based Gesture Recognition Using Several Fully Connected Layers with Path Signature Features and Temporal Transformer Module

1 code implementation17 Nov 2018 Chenyang Li, Xin Zhang, Lufan Liao, Lianwen Jin, Weixin Yang

In this paper, we first leverage a robust feature descriptor, path signature (PS), and propose three PS features to explicitly represent the spatial and temporal motion characteristics, i. e., spatial PS (S_PS), temporal PS (T_PS) and temporal spatial PS (T_S_PS).

Computational Efficiency General Classification +1

Detecting Heads using Feature Refine Net and Cascaded Multi-Scale Architecture

no code implementations25 Mar 2018 Dezhi Peng, Zikai Sun, Zirong Chen, Zirui Cai, Lele Xie, Lianwen Jin

To improve the performance of small head detection, we propose a cascaded multi-scale architecture which has two detectors.

Head Detection

SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Prediction

5 code implementations19 Jan 2018 Lingyu Liang, Luojun Lin, Lianwen Jin, Duorui Xie, Mengru Li

Previous works have formulated the recognition of facial beauty as a specific supervised learning problem of classification, regression or ranking, which indicates that FBP is intrinsically a computation problem with multiple paradigms.

Facial Beauty Prediction General Classification +1

Feature Enhancement Network: A Refined Scene Text Detector

no code implementations12 Nov 2017 Sheng Zhang, Yuliang Liu, Lianwen Jin, Canjie Luo

In this paper, we propose a refined scene text detector with a \textit{novel} Feature Enhancement Network (FEN) for Region Proposal and Text Detection Refinement.

object-detection Object Detection +3

Developing the Path Signature Methodology and its Application to Landmark-based Human Action Recognition

no code implementations13 Jul 2017 Weixin Yang, Terry Lyons, Hao Ni, Cordelia Schmid, Lianwen Jin

To this end, we regard the evolving landmark data as a high-dimensional path and apply non-linear path signature techniques to provide an expressive, robust, non-linear, and interpretable representation for the sequential events.

Action Classification Action Recognition In Videos +1

Online Signature Verification using Recurrent Neural Network and Length-normalized Path Signature

no code implementations19 May 2017 Songxuan Lai, Lianwen Jin, Weixin Yang

Inspired by the great success of recurrent neural networks (RNNs) in sequential modeling, we introduce a novel RNN system to improve the performance of online signature verification.

Design of a Very Compact CNN Classifier for Online Handwritten Chinese Character Recognition Using DropWeight and Global Pooling

no code implementations15 May 2017 Xuefeng Xiao, Yafeng Yang, Tasweer Ahmad, Lianwen Jin, Tianhai Chang

Currently, owing to the ubiquity of mobile devices, online handwritten Chinese character recognition (HCCR) has become one of the suitable choice for feeding input to cell phones and tablet devices.

Deep Matching Prior Network: Toward Tighter Multi-oriented Text Detection

no code implementations CVPR 2017 Yuliang Liu, Lianwen Jin

The effectiveness of our approach is evaluated on a public word-level, multi-oriented scene text database, ICDAR 2015 Robust Reading Competition Challenge 4 "Incidental scene text localization".

Text Detection

Building Fast and Compact Convolutional Neural Networks for Offline Handwritten Chinese Character Recognition

no code implementations26 Feb 2017 Xuefeng Xiao, Lianwen Jin, Yafeng Yang, Weixin Yang, Jun Sun, Tianhai Chang

We design a nine-layer CNN for HCCR consisting of 3, 755 classes, and devise an algorithm that can reduce the networks computational cost by nine times and compress the network to 1/18 of the original size of the baseline model, with only a 0. 21% drop in accuracy.

Offline Handwritten Chinese Character Recognition

Toward high-performance online HCCR: a CNN approach with DropDistortion, path signature and spatial stochastic max-pooling

no code implementations24 Feb 2017 Songxuan Lai, Lianwen Jin, Weixin Yang

This paper presents an investigation of several techniques that increase the accuracy of online handwritten Chinese character recognition (HCCR).

Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition

no code implementations9 Oct 2016 Zecheng Xie, Zenghui Sun, Lianwen Jin, Hao Ni, Terry Lyons

Online handwritten Chinese text recognition (OHCTR) is a challenging problem as it involves a large-scale character set, ambiguous segmentation, and variable-length input sequences.

Handwritten Chinese Text Recognition Language Modelling +1

DeepText: A Unified Framework for Text Proposal Generation and Text Detection in Natural Images

5 code implementations24 May 2016 Zhuoyao Zhong, Lianwen Jin, Shuye Zhang, Ziyong Feng

In this paper, we develop a novel unified framework called DeepText for text region proposal generation and text detection in natural images via a fully convolutional neural network (CNN).

Region Proposal Text Classification +1

Fully Convolutional Recurrent Network for Handwritten Chinese Text Recognition

no code implementations18 Apr 2016 Zecheng Xie, Zenghui Sun, Lianwen Jin, Ziyong Feng, Shuye Zhang

This paper proposes an end-to-end framework, namely fully convolutional recurrent network (FCRN) for handwritten Chinese text recognition (HCTR).

Handwriting Recognition Handwritten Chinese Text Recognition +1

Character Proposal Network for Robust Text Extraction

no code implementations13 Feb 2016 Shuye Zhang, Mude Lin, Tianshui Chen, Lianwen Jin, Liang Lin

Maximally stable extremal regions (MSER), which is a popular method to generate character proposals/candidates, has shown superior performance in scene text detection.

Scene Text Detection Text Detection

SCUT-FBP: A Benchmark Dataset for Facial Beauty Perception

1 code implementation8 Nov 2015 Duorui Xie, Lingyu Liang, Lianwen Jin, Jie Xu, Mengru Li

In this paper, a novel face dataset with attractiveness ratings, namely, the SCUT-FBP dataset, is developed for automatic facial beauty perception.

Deep Learning

A new humanlike facial attractiveness predictor with cascaded fine-tuning deep learning model

no code implementations8 Nov 2015 Jie Xu, Lianwen Jin, Lingyu Liang, Ziyong Feng, Duorui Xie

This paper proposes a deep leaning method to address the challenging facial attractiveness prediction problem.

Facial Beauty Prediction

An Open Source Testing Tool for Evaluating Handwriting Input Methods

no code implementations30 May 2015 Liquan Qiu, Lianwen Jin, Ruifen Dai, Yuxiang Zhang, Lei LI

This paper presents an open source tool for testing the recognition accuracy of Chinese handwriting input methods.

Handwriting Recognition

Improved Deep Convolutional Neural Network For Online Handwritten Chinese Character Recognition using Domain-Specific Knowledge

no code implementations28 May 2015 Weixin Yang, Lianwen Jin, Zecheng Xie, Ziyong Feng

Deep convolutional neural networks (DCNNs) have achieved great success in various computer vision and pattern recognition applications, including those for handwritten Chinese character recognition (HCCR).

Diversity

Recognition Confidence Analysis of Handwritten Chinese Character with CNN

no code implementations25 May 2015 Meijun He, Shuye Zhang, Huiyun Mao, Lianwen Jin

In this paper, we present an effective method to analyze the recognition confidence of handwritten Chinese character, based on the softmax regression score of a high performance convolutional neural networks (CNN).

regression

DropSample: A New Training Method to Enhance Deep Convolutional Neural Networks for Large-Scale Unconstrained Handwritten Chinese Character Recognition

no code implementations20 May 2015 Weixin Yang, Lianwen Jin, DaCheng Tao, Zecheng Xie, Ziyong Feng

Inspired by the theory of Leitners learning box from the field of psychology, we propose DropSample, a new method for training deep convolutional neural networks (DCNNs), and apply it to large-scale online handwritten Chinese character recognition (HCCR).

High Performance Offline Handwritten Chinese Character Recognition Using GoogLeNet and Directional Feature Maps

1 code implementation19 May 2015 Zhuoyao Zhong, Lianwen Jin, Zecheng Xie

We design a streamlined version of GoogLeNet [13], which was original proposed for image classification in recent years with very deep architecture, for HCCR (denoted as HCCR-GoogLeNet).

Image Classification Offline Handwritten Chinese Character Recognition

Character-level Chinese Writer Identification using Path Signature Feature, DropStroke and Deep CNN

no code implementations19 May 2015 Weixin Yang, Lianwen Jin, Manfei Liu

The results reveal that the path-signature feature is useful for writer identification, and the proposed DropStroke technique enhances the generalization and significantly improves performance.

Data Augmentation

Cannot find the paper you are looking for? You can Submit a new open access paper.