MIGS: Meta Image Generation from Scene Graphs

22 Oct 2021  ·  Azade Farshad, Sabrina Musatian, Helisa Dhamo, Nassir Navab ·

Generation of images from scene graphs is a promising direction towards explicit scene generation and manipulation. However, the images generated from the scene graphs lack quality, which in part comes due to high difficulty and diversity in the data. We propose MIGS (Meta Image Generation from Scene Graphs), a meta-learning based approach for few-shot image generation from graphs that enables adapting the model to different scenes and increases the image quality by training on diverse sets of tasks. By sampling the data in a task-driven fashion, we train the generator using meta-learning on different sets of tasks that are categorized based on the scene attributes. Our results show that using this meta-learning approach for the generation of images from scene graphs achieves state-of-the-art performance in terms of image quality and capturing the semantic relationships in the scene. Project Website: https://migs2021.github.io/

PDF Abstract
Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Image Generation from Scene Graphs BDD100K-Subsets MIGS FID 49.5 # 1
Image Generation from Scene Graphs BDD100K-Subsets SG2Im FID 66.1 # 2
Image Generation from Scene Graphs Home Action Genome MIGS FID 98.1 # 1
Image Generation from Scene Graphs Home Action Genome SG2Im FID 141.3 # 2
Image Generation from Scene Graphs Visual Genome 64x64 MIGS FID 54.24 # 1
Image Generation from Scene Graphs Visual Genome 64x64 SG2Im FID 55.2 # 2

Methods


No methods listed for this paper. Add relevant methods here