Scene Graph Generation
110 papers with code • 5 benchmarks • 7 datasets
A scene graph is a structured representation of an image, where nodes in a scene graph correspond to object bounding boxes with their object categories, and edges correspond to their pairwise relationships between objects. The task of Scene Graph Generation is to generate a visually-grounded scene graph that most accurately correlates with an image.
Benchmarks
These leaderboards are used to track progress in Scene Graph Generation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Scene Graph Generation models and implementations
MS COCO
Visual Genome
VRD
3DSSG
3RScan
PSG Dataset
4D-OR
Latest papers
EGTR: Extracting Graph from Transformer for Scene Graph Generation
We propose a lightweight one-stage SGG model that extracts the relation graph from the various relationships learned in the multi-head self-attention layers of the DETR decoder.
Groupwise Query Specialization and Quality-Aware Multi-Assignment for Transformer-based Visual Relationship Detection
mlvlab/speaq
•
•
Groupwise Query Specialization trains a specialized query by dividing queries and relations into disjoint groups and directing a query in a specific query group solely toward relations in the corresponding relation group.
HiKER-SGG: Hierarchical Knowledge Enhanced Robust Scene Graph Generation
Being able to understand visual scenes is a precursor for many downstream tasks, including autonomous driving, robotics, and other vision-based approaches.
SGTR+: End-to-end Scene Graph Generation with Transformer
scarecrow0/sgtr
•
•
Moreover, we design a graph assembling module to infer the connectivity of the bipartite scene graph based on our entity-aware structure, enabling us to generate the scene graph in an end-to-end manner.
Adaptive Self-training Framework for Fine-grained Scene Graph Generation
To this end, we introduce a Self-Training framework for SGG (ST-SGG) that assigns pseudo-labels for unannotated triplets based on which the SGG models are trained.
Panoptic Video Scene Graph Generation
lilydaytoy/openpvsg
•
•
CVPR 2023
PVSG relates to the existing video scene graph generation (VidSGG) problem, which focuses on temporal interactions between humans and objects grounded with bounding boxes in videos.
VLPrompt: Vision-Language Prompting for Panoptic Scene Graph Generation
Panoptic Scene Graph Generation (PSG) aims at achieving a comprehensive image understanding by simultaneously segmenting objects and predicting relations among objects.
Enhancing Scene Graph Generation with Hierarchical Relationships and Commonsense Knowledge
bowen-upenn/scene_graph_commonsense
•
•
This work presents an enhanced approach to generating scene graphs by incorporating a relationship hierarchy and commonsense knowledge.
NeuSyRE: Neuro-Symbolic Visual Understanding and Reasoning Framework based on Scene Graph Enrichment
jaleedkhan/neusire
•
•
Semantic Web 2023
We present a loosely-coupled neuro-symbolic visual understanding and reasoning framework that employs a DNN-based pipeline for object detection and multi-modal pairwise relationship prediction for scene graph generation and leverages common sense knowledge in heterogenous knowledge graphs to enrich scene graphs for improved downstream reasoning.
LLM4SGG: Large Language Model for Weakly Supervised Scene Graph Generation
Weakly-Supervised Scene Graph Generation (WSSGG) research has recently emerged as an alternative to the fully-supervised approach that heavily relies on costly annotations.
