Search Results for author:
Found 24 papers, 19 papers with code
OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning
Scoring the Optical Character Recognition (OCR) capabilities of Large Multimodal Models (LMMs) has witnessed growing interest recently.
Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLM
The application of Large Vision-Language Models (LVLMs) for analyzing images and videos is an exciting and rapidly evolving field.
Grounding Natural Language to SQL Translation with Data-Based Self-Explanations
Natural Language Interfaces for Databases empower non-technical users to interact with data using natural language (NL).
MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark
The comprehension of text-rich visual scenes has become a focal point for evaluating Multi-modal Large Language Models (MLLMs) due to their widespread applications.
UNA: Unifying Alignments of RLHF/PPO, DPO and KTO by a Generalized Implicit Reward Function
DPO proposes a mapping between an optimal policy and a reward, greatly simplifying the training process of RLHF.
ParGo: Bridging Vision-Language with Partial and Global Views
This work presents ParGo, a novel Partial-Global projector designed to connect the vision and language modalities for Multimodal Large Language Models (MLLMs).
Harmonizing Visual Text Comprehension and Generation
Our work delineates the viability of an integrated approach to multimodal generation within the visual text domain, setting a foundation for subsequent inquiries.
A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding
Recently, many studies have demonstrated that exclusively incorporating OCR-derived text and spatial layouts with large language models (LLMs) can be highly effective for document understanding tasks.
TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy
In this mechanism, all the involved diverse visual table understanding (VTU) tasks and multi-source visual embeddings are abstracted as concepts.
MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering
Text-Centric Visual Question Answering (TEC-VQA) in its proper format not only facilitates human-machine interaction in text-centric visual environments but also serves as a de facto gold proxy to evaluate AI models in the domain of text-centric scene understanding.
TextSquare: Scaling up Text-Centric Visual Instruction Tuning
Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of leading models like GPT4V and Gemini, partly due to a lack of extensive, high-quality instruction tuning data.
PURPLE: Making a Large Language Model a Better SQL Writer
LLMs can learn to organize operator compositions from the input demonstrations for the given task.
Elysium: Exploring Object-level Perception in Videos via MLLM
Multi-modal Large Language Models (MLLMs) have demonstrated their ability to perceive objects in still images, but their application in video-related tasks, such as object tracking, remains understudied.
Ranked #2 on
Zero-Shot Single Object Tracking
on LaSOT
Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translation
While these translation models have greatly improved the overall translation accuracy, surpassing 70% on NLIDB benchmarks, the use of auto-regressive decoding to generate single SQL queries may result in sub-optimal outputs, potentially leading to erroneous translations.
PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognition
In this study, we aim to reduce generation latency for Named Entity Recognition (NER) with Large Language Models (LLMs).
GloTSFormer: Global Video Text Spotting Transformer
In this paper, we propose a novel Global Video Text Spotting Transformer GloTSFormer to model the tracking problem as global associations and utilize the Gaussian Wasserstein distance to guide the morphological correlation between frames.
Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizer
A straightforward solution is performing model fine-tuning tailored to a specific scenario, but it is computationally intensive and requires multiple model copies for various scenarios.
DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding
This work presents DocPedia, a novel large multimodal model (LMM) for versatile OCR-free document understanding, capable of parsing images up to 2, 560$\times$2, 560 resolution.
ESTextSpotter: Towards Better Scene Text Spotting with Explicit Synergy in Transformer
To this end, we introduce a new model named Explicit Synergy-based Text Spotting Transformer framework (ESTextSpotter), which achieves explicit synergy by modeling discriminative and interactive features for text detection and recognition within a single decoder.
UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding
However, existing advanced algorithms are limited to effectively utilizing the immense representation capabilities and rich world knowledge inherent to these large pre-trained models, and the beneficial connections among tasks within the context of text-rich scenarios have not been sufficiently explored.
SPTS v2: Single-Point Scene Text Spotting
Within the context of our SPTS v2 framework, our experiments suggest a potential preference for single-point representation in scene text spotting when compared to other representations.
Ranked #15 on
Text Spotting
on ICDAR 2015
Knowing Where and What: Unified Word Block Pretraining for Document Understanding
In this paper, we focus on the embedding learning of word blocks containing text and layout information, and propose UTel, a language model with Unified TExt and Layout pre-training.
M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Network
To improve the robustness and completeness of point cloud reconstruction, we propose a novel multi-metric loss function that combines pixel-wise and feature-wise loss function to learn the inherent constraints from different perspectives of matching correspondences.
M^3VSNet: Unsupervised Multi-metric Multi-view Stereo Network
To improve the robustness and completeness of point cloud reconstruction, we propose a novel multi-metric loss function that combines pixel-wise and feature-wise loss function to learn the inherent constraints from different perspectives of matching correspondences.
