WHOOPS!
Introduced by Bitton-Guetta et al. in Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images
WHOOPS! Is a dataset and benchmark for visual commonsense. The dataset is comprised of purposefully commonsense-defying images created by designers using publicly-available image generation tools like Midjourney. It contains commonsense-defying image from a wide range of reasons, deviations from expected social norms and everyday knowledge.
Benchmarks
| Trend | Task | Dataset Variant | Best Model | Paper | Code |
|---|---|---|---|---|---|
|
WHOOPS!
|
Ground-truth Caption -> GPT3
|
|||
|
WHOOPS!
|
BLIP2 FlanT5-XXL
|
|||
|
WHOOPS!
|
BLIP2 FlanT5-XXL
|
|||
|
WHOOPS!
|
BLIP2 FlanT5-XXL
|
Papers
| Paper | Code | Results | Date | Stars |
|---|
