Search Results for author:
Found 25 papers, 16 papers with code
To Find Waldo You Need Contextual Cues: Debiasing Who’s Waldo
We find that the original Who’s Waldo dataset compiled for this task contains a large number of biased samples that are solvable simply by heuristic methods; for instance, in many cases the first name in the sentence corresponds to the largest bounding box, or the sequence of names in the sentence corresponds to an exact left-to-right order in the image.
On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation
Recent advances in monocular depth estimation have been made by incorporating natural language as additional guidance.
Improving Shift Invariance in Convolutional Neural Networks with Translation Invariant Polyphase Sampling
Downsampling operators break the shift invariance of convolutional neural networks (CNNs) and this affects the robustness of features learned by CNNs when dealing with even small pixel-level shift.
Getting it Right: Improving Spatial Consistency in Text-to-Image Models
One of the key shortcomings in current text-to-image (T2I) models is their inability to consistently generate images which faithfully follow the spatial relationships specified in the text prompt.
Adversarial Bayesian Augmentation for Single-Source Domain Generalization
Generalizing to unseen image domains is a challenging problem primarily due to the lack of diverse training data, inaccessible target data, and the large domain shift that may exist in many real-world settings.
ConceptBed: Evaluating Concept Learning Abilities of Text-to-Image Diffusion Models
To quantify the ability of T2I models in learning and synthesizing novel visual concepts (a. k. a.
End-to-end Knowledge Retrieval with Multi-modal Queries
We investigate knowledge retrieval with multi-modal queries, i. e. queries containing information split across image and text inputs, a challenging task that differs from previous work on cross-modal retrieval.
Mole Recruitment: Poisoning of Image Classifiers via Selective Batch Sampling
In this work, we present a data poisoning attack that confounds machine learning models without any manipulation of the image or label.
Benchmarking Spatial Relationships in Text-to-Image Generation
We investigate the ability of T2I models to generate correct spatial relationships among objects and present VISOR, an evaluation metric that captures how accurately the spatial relationship described in text is generated in the image.
CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answering
Videos often capture objects, their visible properties, their motion, and the interactions between different objects.
Ranked #1 on
Counterfactual Planning
on CRIPP-VQA
Improving Diversity with Adversarially Learned Transformations for Domain Generalization
To be successful in single source domain generalization, maximizing diversity of synthesized domains has emerged as one of the most effective strategies.
To Find Waldo You Need Contextual Cues: Debiasing Who's Waldo
We find that the original Who's Waldo dataset compiled for this task contains a large number of biased samples that are solvable simply by heuristic methods; for instance, in many cases the first name in the sentence corresponds to the largest bounding box, or the sequence of names in the sentence corresponds to an exact left-to-right order in the image.
Generalized but not Robust? Comparing the Effects of Data Modification Methods on Out-of-Domain Generalization and Adversarial Robustness
However, the effect of data modification on adversarial robustness remains unclear.
Improving Biomedical Information Retrieval with Neural Retrievers
We show that BM25 and our method can complement each other, and a simple hybrid model leads to further gains in the large corpus setting.
Unsupervised Natural Language Inference Using PHL Triplet Generation
Transformer-based models achieve impressive performance on numerous Natural Language Inference (NLI) benchmarks when trained on respective training datasets.
Semantically Distributed Robust Optimization for Vision-and-Language Inference
Analysis of vision-and-language models has revealed their brittleness under linguistic phenomena such as paraphrasing, negation, textual entailment, and word substitutions with synonyms or antonyms.
Weakly Supervised Relative Spatial Reasoning for Visual Question Answering
In this work, we evaluate the faithfulness of V\&L models to such geometric understanding, by formulating the prediction of pair-wise relative locations of objects as a classification as well as a regression task.
Self-Supervised Test-Time Learning for Reading Comprehension
Recent work on unsupervised question answering has shown that models can be trained with procedurally generated question-answer pairs and can achieve performance competitive with supervised methods.
WeaQA: Weak Supervision via Captions for Visual Question Answering
Methodologies for training visual question answering (VQA) models assume the availability of datasets with human-annotated \textit{Image-Question-Answer} (I-Q-A) triplets.
Attribute-Guided Adversarial Training for Robustness to Natural Perturbations
While this deviation may not be exactly known, its broad characterization is specified a priori, in terms of attributes.
MUTANT: A Training Paradigm for Out-of-Distribution Generalization in Visual Question Answering
In this paper, we present MUTANT, a training paradigm that exposes the model to perceptually similar, yet semantically distinct mutations of the input, to improve OOD generalization, such as the VQA-CP challenge.
Halluci-Net: Scene Completion by Exploiting Object Co-occurrence Relationships
The generated dense labelmap can then be used as input by state-of-the-art image synthesis techniques like pix2pixHD to obtain the final image.
Video2Commonsense: Generating Commonsense Descriptions to Enrich Video Captioning
In videos that involve active agents such as humans, the agent's actions can bring about myriad changes in the scene.
VQA-LOL: Visual Question Answering under the Lens of Logic
We propose our {Lens of Logic (LOL)} model which uses question-attention and logic-attention to understand logical connectives in the question, and a novel Fr\'echet-Compatibility Loss, which ensures that the answers of the component questions and the composed question are consistent with the inferred logical operation.
Blocksworld Revisited: Learning and Reasoning to Generate Event-Sequences from Image Pairs
The process of identifying changes or transformations in a scene along with the ability of reasoning about their causes and effects, is a key aspect of intelligence.
