Search Results for author:
Found 12 papers, 3 papers with code
Scene Graph Parsing via Abstract Meaning Representation in Pre-trained Language Models
In this work, we propose the application of abstract meaning representation (AMR) based semantic parsing models to parse textual descriptions of a visual scene into scene graphs, which is the first work to the best of our knowledge.
SGRAM: Improving Scene Graph Parsing via Abstract Meaning Representation
To this end, we design a simple yet effective two-stage scene graph parsing framework utilizing abstract meaning representation, SGRAM (Scene GRaph parsing via Abstract Meaning representation): 1) transforming a textual description of an image into an AMR graph (Text-to-AMR) and 2) encoding the AMR graph into a Transformer-based language model to generate a scene graph (AMR-to-SG).
Hypergraph Transformer: Weakly-supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering
Knowledge-based visual question answering (QA) aims to answer a question which requires visually-grounded external knowledge beyond image content itself.
Toward a Human-Level Video Understanding Intelligence
In this paper, we propose the Video Turing Test to provide effective and practical assessments of video understanding intelligence as well as human-likeness evaluation of AI agents.
CogME: A Cognition-Inspired Multi-Dimensional Evaluation Metric for Story Understanding
We introduce CogME, a cognition-inspired, multi-dimensional evaluation metric designed for AI models focusing on story understanding.
Toward General Scene Graph: Integration of Visual Semantic Knowledge with Entity Synset Alignment
In experiment, the integrated scene graph is applied to the image-caption retrieval task as a down-stream task.
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA
Despite recent progress on computer vision and natural language processing, developing a machine that can understand video story is still hard to achieve due to the intrinsic difficulty of video story.
Cut-Based Graph Learning Networks to Discover Compositional Structure of Sequential Video Data
Here, we propose Cut-Based Graph Learning Networks (CB-GLNs) for learning video data by discovering these complex structures of the video.
Compositional Structure Learning for Sequential Video Data
However, most of sequential data, as seen with videos, have complex temporal dependencies that imply variable-length semantic flows and their compositions, and those are hard to be captured by conventional methods.
Constructing Hierarchical Q&A Datasets for Video Story Understanding
Video understanding is emerging as a new paradigm for studying human-like AI.
Visualizing Semantic Structures of Sequential Data by Learning Temporal Dependencies
While conventional methods for sequential learning focus on interaction between consecutive inputs, we suggest a new method which captures composite semantic flows with variable-length dependencies.
Answerer in Questioner's Mind: Information Theoretic Approach to Goal-Oriented Visual Dialog
Goal-oriented dialogue tasks occur when a questioner asks an action-oriented question and an answerer responds with the intent of letting the questioner know a correct action to take.
