GenAI-Bench is a benchmarking framework designed to evaluate and improve compositional text-to-visual generation models. It was developed by researchers from Carnegie Mellon University and Meta. The key aspects of GenAI-Bench include:

• Compositional Text-to-Visual Generation: It focuses on the ability of generative models to handle compositional text prompts that involve attributes, relationships, and higher-order reasoning such as logic and comparison¹.

• Diverse Text Prompts: GenAI-Bench uses 1,600 text prompts collected from professional graphic designers to cover a wide range of compositional reasoning skills¹.

• Human Studies: The framework includes human annotators who rate the performance of leading generative models like DALL-E 3, Stable Diffusion, and others based on image-text or video-text alignment¹.

• Automated Evaluation Metrics: It aims to benchmark automated evaluation metrics that measure the alignment between an image and a text prompt².

• Improving Generation: GenAI-Bench also explores how VQAScore, an automated metric, can improve image generation by selecting the highest-scoring images from generated candidates¹.

Overall, GenAI-Bench provides a comprehensive and challenging testbed for state-of-the-art text-to-visual generative models, pushing the boundaries of what these models can achieve in terms of understanding and creating complex visual compositions based on textual descriptions.

(1) GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual .... https://linzhiqiu.github.io/papers/genai_bench/. (2) GenAI-Bench: A Holistic Benchmark for Compositional Text-to-Visual .... https://openreview.net/pdf?id=hJm7qnW3ym. (3) Evaluating Text-to-Visual Generation with Image-to-Text Generation. https://linzhiqiu.github.io/papers/vqascore/.