Search Results for author:
Found 32 papers, 17 papers with code
CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy
The current landscape lacks a comprehensive benchmark to effectively measure the literate capabilities of LMMs.
HIP: Hierarchical Point Modeling and Pre-training for Visual Information Extraction
End-to-end visual information extraction (VIE) aims at integrating the hierarchical subtasks of VIE, including text spotting, word grouping, and entity labeling, into a unified framework.
VL-Reader: Vision and Language Reconstructor is an Effective Scene Text Recognizer
Text recognition is an inherent integration of vision and language, encompassing the visual texture in stroke patterns and the semantic context among the character sequences.
Platypus: A Generalized Specialist Model for Reading Text in Various Forms
Recently, generalist models (such as GPT-4V), trained on tremendous data in a unified way, have shown enormous potential in reading text in various scenarios, but with the drawbacks of limited accuracy and low efficiency.
Handwritten Text Recognition
Optical Character Recognition (OCR)
+1
Look Hear: Gaze Prediction for Speech-directed Human Attention
To predict the gaze scanpaths in this incremental object referral task, we developed the Attention in Referral Transformer model or ART, which predicts the human fixations spurred by each word in a referring expression.
Visual Text Generation in the Wild
However, it is still challenging to render high-quality text images in real-world scenarios, as three critical criteria should be satisfied: (1) Fidelity: the generated text images should be photo-realistic and the contents are expected to be the same as specified in the given conditions; (2) Reasonability: the regions and contents of the generated text should cohere with the scene; (3) Utility: the generated text images can facilitate related tasks (e. g., text detection and recognition).
OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognition
Recently, visually-situated text parsing (VsTP) has experienced notable advancements, driven by the increasing demand for automated document understanding and the emergence of Generative Large Language Models (LLMs) capable of processing document-based questions.
HierCode: A Lightweight Hierarchical Codebook for Zero-shot Chinese Text Recognition
Text recognition, especially for complex scripts like Chinese, faces unique challenges due to its intricate character structures and vast vocabulary.
LORE++: Logical Location Regression Network for Table Structure Recognition with Pre-training
We model TSR as a logical location regression problem and propose a new TSR framework called LORE, standing for LOgical location REgression network, which for the first time regresses logical location as well as spatial location of table cells in a unified network.
OmniParser: A Unified Framework for Text Spotting Key Information Extraction and Table Recognition
Recently visually-situated text parsing (VsTP) has experienced notable advancements driven by the increasing demand for automated document understanding and the emergence of Generative Large Language Models (LLMs) capable of processing document-based questions.
Efficient Monaural Speech Enhancement using Spectrum Attention Fusion
Speech enhancement is a demanding task in automated speech processing pipelines, focusing on separating clean speech from noisy channels.
Gazeformer: Scalable, Effective and Fast Prediction of Goal-Directed Human Attention
In response, we pose a new task called ZeroGaze, a new variant of zero-shot learning where gaze is predicted for never-before-searched objects, and we develop a novel model, Gazeformer, to solve the ZeroGaze problem.
Modeling Entities as Semantic Points for Visual Information Extraction in the Wild
As the first contribution of this work, we curate and release a new dataset for VIE, in which the document images are much more challenging in that they are taken from real applications, and difficulties such as blur, partial occlusion, and printing shift are quite common.
Unifying Top-down and Bottom-up Scanpath Prediction Using Transformers
Most models of visual attention aim at predicting either top-down or bottom-up control, as studied using different visual search and free-viewing tasks.
Unraveling Key Elements Underlying Molecular Property Prediction: A Systematic Study
Herein, we conduct an extensive evaluation of representative models using various representations on the MoleculeNet datasets, a suite of opioids-related datasets and two additional activity datasets from the literature.
Target-absent Human Attention
In this paper, we propose the first data-driven computational model that addresses the search-termination problem and predicts the scanpath of search fixations made by people searching for targets that do not appear in images.
Vision-Language Pre-Training for Boosting Scene Text Detectors
In this paper, we specifically adapt vision-language joint learning for scene text detection, a task that intrinsically involves cross-modal interaction between the two modalities: vision and language, since text is the written form of language.
Revisiting Document Image Dewarping by Grid Regularization
This paper addresses the problem of document image dewarping, which aims at eliminating the geometric distortion in document images for document digitization.
Ranked #3 on
Local Distortion
on DocUNet
ARTS: Eliminating Inconsistency between Text Detection and Recognition with Auto-Rectification Text Spotter
Recent approaches for end-to-end text spotting have achieved promising results.
Parsing Table Structures in the Wild
In contrast to existing studies that mainly focus on parsing well-aligned tabular images with simple layouts from scanned PDF documents, we aim to establish a practical table structure parsing system for real-world scenarios where tabular input images are taken or scanned with severe deformation, bending or occlusions.
Artificial Intelligence in Drug Discovery: Applications and Techniques
Artificial intelligence (AI) has been transforming the practice of drug discovery in the past decade.
PAN++: Towards Efficient and Accurate End-to-End Spotting of Arbitrarily-Shaped Text
By systematically comparing with existing scene text representations, we show that our kernel representation can not only describe arbitrarily-shaped text but also well distinguish adjacent text.
MOST: A Multi-Oriented Scene Text Detector with Localization Refinement
Over the past few years, the field of scene text detection has progressed rapidly that modern text detectors are able to hunt text in various challenging scenarios.
Hierarchical Proxy-based Loss for Deep Metric Learning
In this paper, we present a framework that leverages this implicit hierarchy by imposing a hierarchical structure on the proxies and can be used with any existing proxy-based loss.
Defending against black-box adversarial attacks with gradient-free trained sign activation neural networks
The non-transferability in our ensemble also makes it a powerful defense to substitute model black box attacks that we show require a much greater distortion than binary and full precision networks to bring our model to zero adversarial accuracy.
AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting
Unlike previous works that merely employed visual features for text detection, this work proposes a novel text spotter, named Ambiguity Eliminating Text Spotter (AE TextSpotter), which learns both visual and linguistic features to significantly reduce ambiguity in text detection.
Predicting Goal-directed Human Attention Using Inverse Reinforcement Learning
These maps were learned by IRL and then used to predict behavioral scanpaths for multiple target categories.
Towards Better Opioid Antagonists Using Deep Reinforcement Learning
Naloxone, an opioid antagonist, has been widely used to save lives from opioid overdose, a leading cause for death in the opioid epidemic.
Predicting Goal-directed Attention Control Using Inverse-Reinforcement Learning
Using machine learning and the psychologically-meaningful principle of reward, it is possible to learn the visual features used in goal-directed attention control.
TextField: Learning A Deep Direction Field for Irregular Scene Text Detection
Experimental results show that the proposed TextField outperforms the state-of-the-art methods by a large margin (28% and 8%) on two curved text datasets: Total-Text and CTW1500, respectively, and also achieves very competitive performance on multi-oriented datasets: ICDAR 2015 and MSRA-TD500.
Ranked #22 on
Scene Text Detection
on Total-Text
Robust and Fast Decoding of High-Capacity Color QR Codes for Mobile Applications
Particularly, we further discover a new type of chromatic distortion in high-density color QR codes, cross-module color interference, caused by the high density which also makes the geometric distortion correction more challenging.
Similar Handwritten Chinese Character Discrimination by Weakly Supervised Learning
The unconstrained property makes our method well adapted to high variance in the size and position of discriminative regions in similar handwritten Chinese characters.
