Search Results for author:
Found 14 papers, 7 papers with code
Question Aware Vision Transformer for Multimodal Reasoning
This integration results in dynamic visual features focusing on relevant image aspects to the posed question.
GRAM: Global Reasoning for Multi-Page VQA
The increasing use of transformer-based large language models brings forward the challenge of processing long sequences.
CLIPAG: Towards Generator-Free Text-to-Image Generation
Perceptually Aligned Gradients (PAG) refer to an intriguing property observed in robust image classification models, wherein their input gradients align with human perception and pose semantic meanings.
FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions
Our proposed method, FuseCap, fuses the outputs of such vision experts with the original captions using a large language model (LLM), yielding comprehensive image descriptions.
Ranked #1 on
Image Captioning
on COCO Captions
(CLIPScore metric)
Classifier Robustness Enhancement Via Test-Time Transformation
We show that the proposed method achieves state-of-the-art results and validate our claim through extensive experiments on a variety of defense methods, classifier architectures, and datasets.
Towards Models that Can See and Read
Visual Question Answering (VQA) and Image Captioning (CAP), which are among the most popular vision-language tasks, have analogous scene-text versions that require reasoning from the text in the image.
CLIPTER: Looking at the Bigger Picture in Scene Text Recognition
Reading text in real-world scenarios often requires understanding the context surrounding it, especially when dealing with poor-quality text.
Enhancing Diffusion-Based Image Synthesis with Robust Classifier Guidance
In order to obtain class-conditional generation, it was suggested to guide the diffusion process by gradients from a time-dependent classifier.
Do Perceptually Aligned Gradients Imply Adversarial Robustness?
In this work, we focus on this trait and test whether \emph{Perceptually Aligned Gradients imply Robustness}.
Threat Model-Agnostic Adversarial Defense using Diffusion Models
Deep Neural Networks (DNNs) are highly sensitive to imperceptible malicious perturbations, known as adversarial attacks.
Multimodal Semi-Supervised Learning for Text Recognition
In a novel setup, consistency is enforced on each modality separately.
Improved Image Generation via Sparsity
The interest of the deep learning community in image synthesis has grown massively in recent years.
BIGRoC: Boosting Image Generation via a Robust Classifier
The interest of the machine learning community in image synthesis has grown significantly in recent years, with the introduction of a wide range of deep generative models and means for training them.
Ranked #4 on
Image Generation
on ImageNet 128x128
Improved Image Generation via Sparse Modeling
The interest of the deep learning community in image synthesis has grown massively in recent years.
