Search Results for author:
Found 20 papers, 8 papers with code
MMGRec: Multimodal Generative Recommendation with Transformer Model
Multimodal recommendation aims to recommend user-preferred candidates based on her/his historically interacted items and associated multimodal information.
Interactive Garment Recommendation with User in the Loop
Recommending fashion items often leverages rich user profiles and makes targeted suggestions based on past history and previous purchases.
Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue
Although existing studies have achieved great success based on the generative pretrained language model BART, they overlook exploiting the sentiments residing in the utterance, video and audio, which are vital clues for sarcasm explanation.
Prompt-based Multi-interest Learning Method for Sequential Recommendation
Multi-interest learning method for sequential recommendation aims to predict the next item according to user multi-faceted interests given the user historical interactions.
VK-G2T: Vision and Context Knowledge enhanced Gloss2Text
Existing sign language translation methods follow a two-stage pipeline: first converting the sign language video to a gloss sequence (i. e. Sign2Gloss) and then translating the generated gloss sequence into a spoken language sentence (i. e. Gloss2Text).
Multi-source Semantic Graph-based Multimodal Sarcasm Explanation Generation
Multimodal Sarcasm Explanation (MuSE) is a new yet challenging task, which aims to generate a natural language sentence for a multimodal social post (an image as well as its caption) to explain why it contains sarcasm.
Dual Semantic Knowledge Composed Multimodal Dialog Systems
Thereafter, considering that the attribute knowledge and relation knowledge can benefit the responding to different levels of questions, we design a multi-level knowledge composition module in MDS-S2 to obtain the latent composed response representation.
Stylized Data-to-Text Generation: A Case Study in the E-Commerce Domain
This task is non-trivial, due to three challenges: the logic of the generated text, unstructured style reference, and biased training samples.
Multimodal Matching-aware Co-attention Networks with Mutual Knowledge Distillation for Fake News Detection
Additionally, based on the designed image-text matching-aware co-attention mechanism, we propose to build two co-attention networks respectively centered on text and image for mutual knowledge distillation to improve fake news detection.
Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis
Inspired by this, we devise a model-agnostic counterfactual framework for multimodal sentiment analysis, which captures the direct effect of textual modality via an extra text model and estimates the indirect one by a multimodal model.
Multimodal Dialog Systems with Dual Knowledge-enhanced Generative Pretrained Language Model
To address these limitations, we propose a novel dual knowledge-enhanced generative pretrained language model for multimodal task-oriented dialog systems (DKMD), consisting of three key components: dual knowledge selection, dual knowledge-enhanced context learning, and knowledge-enhanced response generation.
Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation
Scene Graph Generation, which generally follows a regular encoder-decoder pipeline, aims to first encode the visual contents within the given image and then parse them into a compact summary graph.
Ranked #1 on
Unbiased Scene Graph Generation
on Visual Genome
(mR@20 metric)
MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning
Logical reasoning is of vital importance to natural language understanding.
Ranked #3 on
Reading Comprehension
on ReClor
Dual Preference Distribution Learning for Item Recommendation
Moreover, we can summarize a preferred attribute profile for each user, depicting his/her preferred item attributes.
Hierarchical Deep Residual Reasoning for Temporal Moment Localization
Temporal Moment Localization (TML) in untrimmed videos is a challenging task in the field of multimedia, which aims at localizing the start and end points of the activity in the video, described by a sentence query.
Multi-Modal Interaction Graph Convolutional Network for Temporal Language Localization in Videos
Towards this end, in this work, we propose a Multi-modal Interaction Graph Convolutional Network (MIGCN), which jointly explores the complex intra-modal relations and inter-modal interactions residing in the video and sentence query to facilitate the understanding and semantic correspondence capture of the video and sentence query.
Answer Questions with Right Image Regions: A Visual Attention Regularization Approach
However, recent studies have pointed out that the highlighted image regions from the visual attention are often irrelevant to the given question and answer, leading to model confusion for correct visual reasoning.
Market2Dish: Health-aware Food Recommendation
With the rising incidence of some diseases, such as obesity and diabetes, a healthy diet is arousing increasing attention.
Adaptive Collaborative Similarity Learning for Unsupervised Multi-view Feature Selection
We propose to dynamically learn the collaborative similarity structure, and further integrate it with the ultimate feature selection into a unified framework.
Neural Compatibility Modeling with Attentive Knowledge Distillation
Nevertheless, existing studies overlook the rich valuable knowledge (rules) accumulated in fashion domain, especially the rules regarding clothing matching.
