Search Results for author:
Found 27 papers, 12 papers with code
Mixed-Query Transformer: A Unified Image Segmentation Architecture
Existing unified image segmentation models either employ a unified architecture across multiple tasks but use separate weights tailored to each dataset, or apply a single set of weights to multiple datasets but are limited to a single task.
On the Scalability of Diffusion-based Text-to-Image Generation
On the data scaling side, we show the quality and diversity of the training set matters more than simply dataset size.
DEED: Dynamic Early Exit on Decoder for Accelerating Encoder-Decoder Transformer Models
Based on the multi-exit model, we perform step-level dynamic early exit during inference, where the model may decide to use fewer decoder layers based on its confidence of the current layer at each individual decoding step.
Multiple-Question Multiple-Answer Text-VQA
We present Multiple-Question Multiple-Answer (MQMA), a novel approach to do text-VQA in encoder-decoder transformer models.
DocTr: Document Transformer for Structured Information Extraction in Documents
We present a new formulation for structured information extraction (SIE) from visually rich documents.
Ranked #2 on
Entity Linking
on FUNSD
DocFormerv2: Local Features for Document Understanding
We propose DocFormerv2, a multi-modal transformer for Visual Document Understanding (VDU).
Ranked #9 on
Visual Question Answering (VQA)
on DocVQA test
(using extra training data)
document understanding
Optical Character Recognition (OCR)
+1
PolyFormer: Referring Image Segmentation as Sequential Polygon Generation
In this work, instead of directly predicting the pixel-level segmentation masks, the problem of referring image segmentation is formulated as sequential polygon generation, and the predicted polygons can be later converted into segmentation masks.
Ranked #1 on
Referring Expression Segmentation
on ReferIt
(using extra training data)
SimCon Loss with Multiple Views for Text Supervised Semantic Segmentation
Instead of purely relying on the alignment from the noisy data, this paper proposes a novel loss function termed SimCon, which accounts for intra-modal similarities to determine the appropriate set of positive samples to align.
YORO -- Lightweight End to End Visual Grounding
We present YORO - a multi-modal transformer encoder-only architecture for the Visual Grounding (VG) task.
GLASS: Global to Local Attention for Scene-Text Spotting
In recent years, the dominant paradigm for text spotting is to combine the tasks of text detection and recognition into a single end-to-end framework.
Ranked #6 on
Text Spotting
on Total-Text
Towards Weakly-Supervised Text Spotting using a Multi-Task Transformer
Text spotting end-to-end methods have recently gained attention in the literature due to the benefits of jointly optimizing the text detection and recognition components.
LaTr: Layout-Aware Transformer for Scene-Text VQA
Accounting for this, we propose a single objective pre-training scheme that requires only text and spatial cues.
DocFormer: End-to-End Transformer for Document Understanding
DocFormer uses text, vision and spatial features and combines them using a novel multi-modal self-attention layer.
Ranked #3 on
Document Image Classification
on RVL-CDIP
On Calibration of Scene-Text Recognition Models
Although the topic of confidence calibration has been an active research area for the last several decades, the case of structured and sequence prediction calibration has been scarcely explored.
Sequence-to-Sequence Contrastive Learning for Text Recognition
We propose a framework for sequence-to-sequence contrastive learning (SeqCLR) of visual representations, which we apply to text recognition.
A Comprehensive Study of Deep Video Action Recognition
Video action recognition is one of the representative tasks for video understanding.
Document Visual Question Answering Challenge 2020
For the task 1 a new dataset is introduced comprising 50, 000 questions-answer(s) pairs defined over 12, 767 document images.
Improving Semantic Segmentation via Self-Training
In the case of semantic segmentation, this means that large amounts of pixelwise annotations are required to learn accurate models.
ResNeSt: Split-Attention Networks
It is well known that featuremap attention and multi-path representation are important for visual recognition.
Ranked #8 on
Instance Segmentation
on COCO test-dev
(APM metric)
SCATTER: Selective Context Attentional Scene Text Recognizer
The first attention step re-weights visual features from a CNN backbone together with contextual features computed by a BiLSTM layer.
Saliency Driven Perceptual Image Compression
The proposed compression model incorporates the salient regions and optimizes on the proposed perceptual similarity metric.
Human Perceptual Evaluations for Image Compression
Recently, there has been much interest in deep learning techniques to do image compression and there have been claims that several of these produce better results than engineered compression schemes (such as JPEG, JPEG2000 or BPG).
Deep Perceptual Compression
In several cases, the MS-SSIM for deep learned techniques is higher than say a conventional, non-deep learned codec such as JPEG-2000 or BPG.
Searching for Apparel Products from Images in the Wild
In particular, Instagram and Twitter influencers often provide images of themselves wearing different outfits and their followers are often inspired to buy similar clothes. We propose a system to automatically find the closest visually similar clothes in the online Catalog (street-to-shop searching).
Compressed Video Action Recognition
), we propose to train a deep network directly on the compressed video.
Ranked #46 on
Action Classification
on Charades
(using extra training data)
Sampling Matters in Deep Embedding Learning
In addition, we show that a simple margin based loss is sufficient to outperform all other loss functions.
Ranked #5 on
Image Retrieval
on CARS196
Deep Decision Network for Multi-Class Image Classification
During the learning phase, starting from the root network node, DDN automatically builds a network that splits the data into disjoint clusters of classes which would be handled by the subsequent expert networks.
