Search Results for author: Piyush Sharma

Found 14 papers, 5 papers with code

CrossVQA: Scalably Generating Benchmarks for Systematically Testing VQA Generalization

no code implementations EMNLP 2021 Arjun Akula, Soravit Changpinyo, Boqing Gong, Piyush Sharma, Song-Chun Zhu, Radu Soricut

One challenge in evaluating visual question answering (VQA) models in the cross-dataset adaptation setting is that the distribution shifts are multi-modal, making it difficult to identify if it is the shifts in visual or language features that play a key role.

Question-Answer-Generation Question Answering +1

Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts

1 code implementation CVPR 2021 Soravit Changpinyo, Piyush Sharma, Nan Ding, Radu Soricut

The availability of large-scale image captioning and visual question answering datasets has contributed significantly to recent successes in vision-and-language pre-training.

Image Captioning Question Answering +1

Understanding Guided Image Captioning Performance across Domains

1 code implementation CoNLL (EMNLP) 2021 Edwin G. Ng, Bo Pang, Piyush Sharma, Radu Soricut

Image captioning models generally lack the capability to take into account user interest, and usually default to global descriptions that try to balance readability, informativeness, and information overload.

Image Captioning Informativeness +1

Denoising Large-Scale Image Captioning from Alt-text Data using Content Selection Models

no code implementations10 Sep 2020 Khyathi Raghavi Chandu, Piyush Sharma, Soravit Changpinyo, Ashish Thapliyal, Radu Soricut

Training large-scale image captioning (IC) models demands access to a rich and diverse set of training examples, gathered from the wild, often from noisy alt-text data.

Denoising Image Captioning

Cross-modal Coherence Modeling for Caption Generation

no code implementations ACL 2020 Malihe Alikhani, Piyush Sharma, Shengjie Li, Radu Soricut, Matthew Stone

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.

Image Captioning

Clue: Cross-modal Coherence Modeling for Caption Generation

no code implementations2 May 2020 Malihe Alikhani, Piyush Sharma, Shengjie Li, Radu Soricut, Matthew Stone

We use coherence relations inspired by computational models of discourse to study the information needs and goals of image captioning.

Image Captioning

Reinforcing an Image Caption Generator Using Off-Line Human Feedback

no code implementations21 Nov 2019 Paul Hongsuck Seo, Piyush Sharma, Tomer Levinboim, Bohyung Han, Radu Soricut

Human ratings are currently the most accurate way to assess the quality of an image captioning model, yet most often the only used outcome of an expensive human rating evaluation is a few overall statistics over the evaluation dataset.

Image Captioning

Neural Naturalist: Generating Fine-Grained Image Comparisons

no code implementations IJCNLP 2019 Maxwell Forbes, Christine Kaeser-Chen, Piyush Sharma, Serge Belongie

We introduce the new Birds-to-Words dataset of 41k sentences describing fine-grained differences between photographs of birds.

Quality Estimation for Image Captions Based on Large-scale Human Evaluations

1 code implementation NAACL 2021 Tomer Levinboim, Ashish V. Thapliyal, Piyush Sharma, Radu Soricut

Automatic image captioning has improved significantly over the last few years, but the problem is far from being solved, with state of the art models still often producing low quality captions when used in the wild.

Image Captioning Model Selection

Informative Image Captioning with External Sources of Information

no code implementations ACL 2019 Sanqiang Zhao, Piyush Sharma, Tomer Levinboim, Radu Soricut

An image caption should fluently present the essential information in a given image, including informative, fine-grained entity mentions and the manner in which these entities interact.

Image Captioning Informativeness

Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning

1 code implementation ACL 2018 Piyush Sharma, Nan Ding, Sebastian Goodman, Radu Soricut

We present a new dataset of image caption annotations, Conceptual Captions, which contains an order of magnitude more images than the MS-COCO dataset (Lin et al., 2014) and represents a wider variety of both images and image caption styles.

Image Captioning

Cannot find the paper you are looking for? You can Submit a new open access paper.