Search Results for author:
Found 18 papers, 6 papers with code
HOTR: End-to-End Human-Object Interaction Detection with Transformers
Human-Object Interaction (HOI) detection is a task of identifying "a set of interactions" in an image, which involves the i) localization of the subject (i. e., humans) and target (i. e., objects) of interaction, and ii) the classification of the interaction labels.
Ranked #16 on
Human-Object Interaction Detection
on V-COCO
Boundary-aware Self-supervised Learning for Video Scene Segmentation
Inspired from this, we tackle video scene segmentation, which is a task of temporally localizing scene boundaries in a video, with a self-supervised learning framework where we mainly focus on designing effective pretext tasks.
Hypergraph Transformer: Weakly-supervised Multi-hop Reasoning for Knowledge-based Visual Question Answering
Knowledge-based visual question answering (QA) aims to answer a question which requires visually-grounded external knowledge beyond image content itself.
Video-Text Representation Learning via Differentiable Weak Temporal Alignment
In this paper, we propose a novel multi-modal self-supervised framework Video-Text Temporally Weak Alignment-based Contrastive Learning (VT-TWINS) to capture significant information from noisy and weakly correlated data using a variant of Dynamic Time Warping (DTW).
Selective Token Generation for Few-shot Natural Language Generation
Natural language modeling with limited training data is a challenging problem, and many algorithms make use of large-scale pretrained language models (PLMs) for this due to its great generalization ability.
Image-to-Image Retrieval by Learning Similarity between Scene Graphs
Based on this idea, we propose a novel approach for image-to-image retrieval using scene graph similarity measured by graph neural networks.
Visualizing Semantic Structures of Sequential Data by Learning Temporal Dependencies
While conventional methods for sequential learning focus on interaction between consecutive inputs, we suggest a new method which captures composite semantic flows with variable-length dependencies.
Compositional Structure Learning for Sequential Video Data
However, most of sequential data, as seen with videos, have complex temporal dependencies that imply variable-length semantic flows and their compositions, and those are hard to be captured by conventional methods.
Cut-Based Graph Learning Networks to Discover Compositional Structure of Sequential Video Data
Here, we propose Cut-Based Graph Learning Networks (CB-GLNs) for learning video data by discovering these complex structures of the video.
Hypergraph Attention Networks for Multimodal Learning
HANs follow the process: constructing the common semantic space with symbolic graphs of each modality, matching the semantics between sub-structures of the symbolic graphs, constructing co-attention maps between the graphs in the semantic space, and integrating the multimodal inputs using the co-attention maps to get the final joint representation.
Spectrally Similar Graph Pooling
To further investigate the effectiveness of our proposed method, we evaluate our approach on a real-world problem, image retrieval with visual scene graphs.
Selective Token Generation for Few-shot Language Modeling
Therefore, in this work, we develop a novel additive learning algorithm based on reinforcement learning (RL) for few-shot natural language generation (NLG) tasks.
Boundary-aware Pre-training for Video Scene Segmentation
Inspired from this, we tackle video scene segmentation, which is a task of temporally localizing scene boundaries in a video, with a self-supervised learning framework where we mainly focus on designing effective pretext tasks.
Winning the ICCV'2021 VALUE Challenge: Task-aware Ensemble and Transfer Learning with Visual Concepts
The VALUE (Video-And-Language Understanding Evaluation) benchmark is newly introduced to evaluate and analyze multi-modal representation learning algorithms on three video-and-language tasks: Retrieval, QA, and Captioning.
Semantic Alignment with Calibrated Similarity for Multilingual Sentence Embedding
Predicting the similarity score consists of two sub-tasks, which are monolingual similarity evaluation and multilingual sentence retrieval.
MSTR: Multi-Scale Transformer for End-to-End Human-Object Interaction Detection
Human-Object Interaction (HOI) detection is the task of identifying a set of <human, object, interaction> triplets from an image.
Dense but Efficient VideoQA for Intricate Compositional Reasoning
It is well known that most of the conventional video question answering (VideoQA) datasets consist of easy questions requiring simple reasoning processes.
Clustering-based Image-Text Graph Matching for Domain Generalization
However, they use pivot embedding in global manner (i. e., aligning an image embedding with sentence-level text embedding), not fully utilizing the semantic cues of given text description.
