Search Results for author:
Found 19 papers, 2 papers with code
Enhancing Vision-Language Pre-training with Rich Supervisions
We propose Strongly Supervised pre-training with ScreenShots (S4) - a novel pre-training paradigm for Vision-Language Models using data from large-scale web screenshot rendering.
Multiple-Question Multiple-Answer Text-VQA
We present Multiple-Question Multiple-Answer (MQMA), a novel approach to do text-VQA in encoder-decoder transformer models.
DEED: Dynamic Early Exit on Decoder for Accelerating Encoder-Decoder Transformer Models
Based on the multi-exit model, we perform step-level dynamic early exit during inference, where the model may decide to use fewer decoder layers based on its confidence of the current layer at each individual decoding step.
DocTr: Document Transformer for Structured Information Extraction in Documents
We present a new formulation for structured information extraction (SIE) from visually rich documents.
Ranked #2 on
Entity Linking
on FUNSD
PolyFormer: Referring Image Segmentation as Sequential Polygon Generation
In this work, instead of directly predicting the pixel-level segmentation masks, the problem of referring image segmentation is formulated as sequential polygon generation, and the predicted polygons can be later converted into segmentation masks.
Ranked #1 on
Referring Expression Segmentation
on ReferIt
(using extra training data)
MATrIX -- Modality-Aware Transformer for Information eXtraction
In these, text semantics and visual information supplement each other to provide a global understanding of the document.
Towards Differential Relational Privacy and its use in Question Answering
While bounding general memorization can have detrimental effects on the performance of a trained model, bounding RM does not prevent effective learning.
Adaptive Regularization of B-Spline Models for Scientific Data
B-spline models are a powerful way to represent scientific data sets with a functional approximation.
Contrastive Neighborhood Alignment
We present Contrastive Neighborhood Alignment (CNA), a manifold learning approach to maintain the topology of learned features whereby data points that are mapped to nearby representations by the source (teacher) model are also mapped to neighbors by the target (student) model.
Visual Relationship Detection Using Part-and-Sum Transformers with Composite Queries
Computer vision applications such as visual relationship detection and human object interaction can be formulated as a composite (structured) set detection problem in which both the parts (subject, object, and predicate) and the sum (triplet as a whole) are to be detected in a hierarchical fashion.
End-to-End Piece-Wise Unwarping of Document Images
Document unwarping attempts to undo the physical deformation of the paper and recover a 'flatbed' scanned document-image for downstream tasks such as OCR.
Multimodal Attention for Layout Synthesis in Diverse Domains
We address the problem of scene layout generation for diverse domains such as images, mobile applications, documents and 3D objects.
LayoutTransformer: Layout Generation and Completion with Self-attention
Generating a new layout or extending an existing layout requires understanding the relationships between these primitives.
Toward Understanding Catastrophic Forgetting in Continual Learning
As an application, we apply our procedure to study two properties of a task sequence: (1) total complexity and (2) sequential heterogeneity.
VLAD3: Encoding Dynamics of Deep Features for Action Recognition
To account for long-range inhomogeneous dynamics, a VLAD descriptor is derived for the LDS and pooled over the whole video, to arrive at the final VLAD^3 representation.
Learning Optimal Seeds for Diffusion-based Salient Object Detection
The propagation of the resulting saliency seeds, using a diffusion process, is finally shown to outperform the state of the art on a number of salient object detection datasets.
On the connections between saliency and tracking
A model connecting visual tracking and saliency has recently been proposed.
Maximum Covariance Unfolding : Manifold Learning for Bimodal Data
To perform this visualization, we augment MCU with an additional step for metric learning in the high dimensional voxel space.
The discriminant center-surround hypothesis for bottom-up saliency
The classical hypothesis, that bottom-up saliency is a center-surround process, is combined with a more recent hypothesis that all saliency decisions are optimal in a decision-theoretic sense.
