Search Results for author:
Found 60 papers, 27 papers with code
Multimodal Transformer for Comics Text-Cloze
This work explores a closure task in comics, a medium where visual and textual elements are intricately intertwined.
Privacy-Aware Document Visual Question Answering
We employ a federated learning scheme, that reflects the real-life distribution of documents in different businesses, and we explore the use case where the ID of the invoice issuer is the sensitive information to be protected.
STEP -- Towards Structured Scene-Text Spotting
We introduce the structured scene-text spotting task, which requires a scene-text OCR system to spot text in the wild according to a query regular expression.
Understanding Video Scenes through Text: Insights from Text-based Video Question Answering
Researchers have extensively studied the field of vision and language, discovering that both visual and textual content is crucial for understanding scenes effectively.
Reading Between the Lanes: Text VideoQA on the Road
Text and signs around roads provide crucial information for drivers, vital for safe navigation and situational awareness.
ICDAR 2023 Competition on Structured Text Extraction from Visually-Rich Document Images
It is hoped that this competition will attract many researchers in the field of CV and NLP, and bring some new thoughts to the field of Document AI.
ICDAR 2023 Competition on Reading the Seal Title
To promote research in this area, we organized ICDAR 2023 competition on reading the seal title (ReST), which included two tasks: seal title text detection (Task 1) and end-to-end seal title recognition (Task 2).
ICDAR 2023 Video Text Reading Competition for Dense and Small Text
In this competition report, we establish a video text reading benchmark, DSText, which focuses on dense and small text reading challenges in the video with various scenarios.
DocILE Benchmark for Document Information Localization and Extraction
This paper introduces the DocILE benchmark with the largest dataset of business documents for the tasks of Key Information Localization and Extraction and Line Item Recognition.
Hierarchical multimodal transformers for Multi-Page DocVQA
The proposed method is based on a hierarchical transformer architecture where the encoder summarizes the most relevant information of every page and then, the decoder takes this summarized information to generate the final answer.
Watching the News: Towards VideoQA Models that can Read
We demonstrate the limitations of current Scene Text VQA and VideoQA methods and propose ways to incorporate scene text information into VideoQA methods.
Show, Interpret and Tell: Entity-aware Contextualised Image Captioning in Wikipedia
Particularly, a similar Wikimedia image can be used to illustrate different articles, and the produced caption needs to be adapted to a specific context, therefore allowing us to explore the limits of a model to adjust captions to different contextual information.
MUST-VQA: MUltilingual Scene-text VQA
In this paper, we present a framework for Multilingual Scene Text Visual Question Answering that deals with new languages in a zero-shot fashion.
Out-of-Vocabulary Challenge Report
This paper presents final results of the Out-Of-Vocabulary 2022 (OOV) challenge.
Optical Character Recognition
Optical Character Recognition (OCR)
+1
Text-DIAE: A Self-Supervised Degradation Invariant Autoencoders for Text Recognition and Document Enhancement
In this paper, we propose a Text-Degradation Invariant Auto Encoder (Text-DIAE), a self-supervised model designed to tackle two tasks, text recognition (handwritten or scene-text) and document image enhancement.
OCR-IDL: OCR Annotations for Industry Document Library Dataset
It is our hope that OCR-IDL can be a starting point for future works on Document Intelligence.
ICDAR 2021 Competition on Document VisualQuestion Answering
In this report we present results of the ICDAR 2021 edition of the Document Visual Question Challenges.
Is An Image Worth Five Sentences? A New Look into Semantics for Image-Text Matching
In this work, we propose two metrics that evaluate the degree of semantic relevance of retrieved items, independently of their annotated binary relevance.
Let there be a clock on the beach: Reducing Object Hallucination in Image Captioning
Explaining an image with missing or non-existent objects is known as object bias (hallucination) in image captioning.
Asking questions on handwritten document collections
This work addresses the problem of Question Answering (QA) on handwritten document collections.
One-shot Compositional Data Generation for Low Resource Handwritten Text Recognition
Low resource Handwritten Text Recognition (HTR) is a hard problem due to the scarce annotated data and the very limited linguistic information (dictionaries and language models).
Document Collection Visual Question Answering
Current tasks and methods in Document Understanding aims to process documents as single elements.
InfographicVQA
Infographics are documents designed to effectively communicate information using a combination of textual, graphical and visual elements.
ICDAR2019 Competition on Scanned Receipt OCR and Information Extraction
In this competition, we set up three tasks, namely, Scanned Receipt Text Localisation (Task 1), Scanned Receipt OCR (Task 2) and Key Information Extraction from Scanned Receipts (Task 3).
Key Information Extraction
Optical Character Recognition (OCR)
+1
StacMR: Scene-Text Aware Cross-Modal Retrieval
Then, armed with this dataset, we describe several approaches which leverage scene text, including a better scene-text aware cross-modal retrieval method which uses specialized representations for text from the captions and text from the visual scene, and reconcile them in a common embedding space.
Multi-Modal Reasoning Graph for Scene-Text Based Fine-Grained Image Classification and Retrieval
Scene text instances found in natural images carry explicit semantic information that can provide important cues to solve a wide array of computer vision problems.
Document Visual Question Answering Challenge 2020
For the task 1 a new dataset is introduced comprising 50, 000 questions-answer(s) pairs defined over 12, 767 document images.
Retrieval Guided Unsupervised Multi-domain Image-to-Image Translation
In this paper we propose the use of an image retrieval system to assist the image-to-image translation task.
Location Sensitive Image Retrieval and Tagging
People from different parts of the globe describe objects and concepts in distinct manners.
Text Recognition -- Real World Data and Where to Find Them
We present a method for exploiting weakly annotated images to improve text extraction pipelines.
DocVQA: A Dataset for VQA on Document Images
The dataset consists of 50, 000 questions defined on 12, 000+ document images.
Ranked #1 on
Visual Question Answering (VQA)
on DocVQA val
Multimodal grid features and cell pointers for Scene Text Visual Question Answering
This paper presents a new model for the task of scene text visual question answering, in which questions about a given image can only be answered by reading and understanding scene text that is present in it.
Fine-grained Image Classification and Retrieval by Combining Visual and Locally Pooled Textual Features
Text contained in an image carries high-level semantics that can be exploited to achieve richer image understanding.
Ranked #1 on
Fine-Grained Image Classification
on Con-Text
ICDAR 2019 Robust Reading Challenge on Reading Chinese Text on Signboard
21 teams submit results for Task 1, 23 teams submit results for Task 2, 24 teams submit results for Task 3, and 13 teams submit results for Task 4.
Exploring Hate Speech Detection in Multimodal Publications
In this work we target the problem of hate speech detection in multimodal publications formed by a text and an image.
ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVT
Robust text reading from street view images provides valuable information for various applications.
ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)
This paper reports the ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT) that consists of three major challenges: i) scene text detection, ii) scene text recognition, and iii) scene text spotting.
ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019
With the growing cosmopolitan culture of modern cities, the need of robust Multi-Lingual scene Text (MLT) detection and recognition systems has never been more immense.
Cultural Vocal Bursts Intensity Prediction
General Classification
+2
ICDAR 2019 Competition on Scene Text Visual Question Answering
ST-VQA introduces an important aspect that is not addressed by any Visual Question Answering system up to date, namely the incorporation of scene text to answer questions asked about an image.
Selective Style Transfer for Text
This paper explores the possibilities of image style transfer applied to text maintaining the original transcriptions.
Scene Text Visual Question Answering
Current visual question answering datasets do not consider the rich semantic information conveyed by text within an image.
Good News, Everyone! Context driven entity-aware captioning for news images
We propose a novel captioning method that is able to leverage contextual information provided by the text of news articles associated with an image.
Self-Supervised Visual Representations for Cross-Modal Retrieval
Cross-modal retrieval methods have been significantly improved in last years with the use of deep neural networks and large-scale annotated datasets such as ImageNet and Places.
Self-Supervised Learning from Web Data for Multimodal Retrieval
In this work we propose to exploit this free available data to learn a multimodal image and text embedding, aiming to leverage the semantic knowledge learnt in the text domain and transfer it to a visual model for semantic image retrieval.
Soft-PHOC Descriptor for End-to-End Word Spotting in Egocentric Scene Images
In this paper we propose a technique to create and exploit an intermediate representation of images based on text attributes which are character probability maps.
Single Shot Scene Text Retrieval
In this way, the text based image retrieval task can be casted as a simple nearest neighbor search of the query text representation over the outputs of the CNN over the entire image database.
Learning from #Barcelona Instagram data what Locals and Tourists post about its Neighbourhoods
We perform a language separate treatment of the data and show that it can be extrapolated to a tourists and locals separate analysis, and that tourism is reflected in Social Media at a neighborhood level.
Learning to Learn from Web Data through Deep Semantic Embeddings
In this paper we propose to learn a multimodal image and text embedding from Web and Social Media data, aiming to leverage the semantic knowledge learnt in the text domain and transfer it to a visual model for semantic image retrieval.
TextTopicNet - Self-Supervised Learning of Visual Features Through Embedding Images on Semantic Text Spaces
We show that adequate visual features can be learned efficiently by training a CNN to predict the semantic textual context in which a particular image is more probable to appear as an illustration.
Non-deterministic Behavior of Ranking-based Metrics when Evaluating Embeddings
Embedding data into vector spaces is a very popular strategy of pattern recognition methods.
The Robust Reading Competition Annotation and Evaluation Platform
The ICDAR Robust Reading Competition (RRC), initiated in 2003 and re-established in 2011, has become a de-facto evaluation standard for robust reading systems and algorithms.
Self-supervised learning of visual features through embedding images into text topic spaces
End-to-end training from scratch of current deep architectures for new computer vision problems would require Imagenet-scale datasets, and this is not always possible.
Improving Text Proposals for Scene Images with Fully Convolutional Networks
Text Proposals have emerged as a class-dependent version of object proposals - efficient approaches to reduce the search space of possible text object locations in an image.
TextProposals: a Text-specific Selective Search Algorithm for Word Spotting in the Wild
Motivated by the success of powerful while expensive techniques to recognize words in a holistic way, object proposals techniques emerge as an alternative to the traditional text detectors.
Improving patch-based scene text script identification with ensembles of conjoined networks
Instead of resizing input images to a fixed aspect ratio as in the typical use of holistic CNN classifiers, we propose here a patch-based classification framework in order to preserve discriminative parts of the image that are characteristic of its class.
A fine-grained approach to scene text script identification
Although widely studied for document images and handwritten documents, it remains an almost unexplored territory for scene text images.
Visual Script and Language Identification
In this paper we introduce a script identification method based on hand-crafted texture features and an artificial neural network.
Object Proposals for Text Extraction in the Wild
The use of Object Proposals techniques in the scene text understanding field is innovative.
Sparse Radial Sampling LBP for Writer Identification
In this paper we present the use of Sparse Radial Sampling Local Binary Patterns, a variant of Local Binary Patterns (LBP) for text-as-texture classification.
A Fast Hierarchical Method for Multi-script and Arbitrary Oriented Scene Text Extraction
Typography and layout lead to the hierarchical organisation of text in words, text lines, paragraphs.
