TASK	DATASET	MODEL	METRIC NAME	METRIC VALUE	GLOBAL RANK
Image Paragraph Captioning	Image Paragraph Captioning	SCST training, w/ rep. penalty	BLEU-4	10.58	# 2
Image Paragraph Captioning	Image Paragraph Captioning	SCST training, w/ rep. penalty	METEOR	17.86	# 5
Image Paragraph Captioning	Image Paragraph Captioning	SCST training, w/ rep. penalty	CIDEr	30.63	# 2

Badge	Markdown
	`[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/training-for-diversity-in-image-paragraph/image-paragraph-captioning-on-image-paragraph)](https://paperswithcode.com/sota/image-paragraph-captioning-on-image-paragraph?p=training-for-diversity-in-image-paragraph)`

Training for Diversity in Image Paragraph Captioning

EMNLP 2018 · Luke Melas-Kyriazi, Alex Rush, er, George Han ·

Image paragraph captioning models aim to produce detailed descriptions of a source image. These models use similar techniques as standard image captioning models, but they have encountered issues in text generation, notably a lack of diversity between sentences, that have limited their effectiveness. In this work, we consider applying sequence-level training for this task. We find that standard self-critical training produces poor results, but when combined with an integrated penalty on trigram repetition produces much more diverse paragraphs. This simple training approach improves on the best result on the Visual Genome paragraph captioning dataset from 16.9 to 30.6 CIDEr, with gains on METEOR and BLEU as well, without requiring any architectural changes.

PDF Abstract