Search Results for author: Tat-Seng Chua

Found 187 papers, 119 papers with code

Re-examining the Role of Schema Linking in Text-to-SQL

no code implementations EMNLP 2020 Wenqiang Lei, Weixin Wang, Zhixin Ma, Tian Gan, Wei Lu, Min-Yen Kan, Tat-Seng Chua

By providing a schema linking corpus based on the Spider text-to-SQL dataset, we systematically study the role of schema linking.


Progressive Text-to-3D Generation for Automatic 3D Prototyping

no code implementations26 Sep 2023 Han Yi, Zhedong Zheng, Xiangyu Xu, Tat-Seng Chua

We aspire for our work to pave the way for automatic 3D prototyping via natural language descriptions.

Text to 3D

NExT-GPT: Any-to-Any Multimodal LLM

1 code implementation11 Sep 2023 Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, Tat-Seng Chua

While recently Multimodal Large Language Models (MM-LLMs) have made exciting strides, they mostly fall prey to the limitation of only input-side multimodal understanding, without the ability to produce content in multiple modalities.

Empowering Dynamics-aware Text-to-Video Diffusion with Large Language Models

no code implementations26 Aug 2023 Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Tat-Seng Chua

In this work, we investigate strengthening the awareness of video dynamics for DMs, for high-quality T2V generation.

Video Generation

Diffusion Variational Autoencoder for Tackling Stochasticity in Multi-Step Regression Stock Price Prediction

1 code implementation18 Aug 2023 Kelvin J. L. Koa, Yunshan Ma, Ritchie Ng, Tat-Seng Chua

The hierarchical VAE allows us to learn the complex and low-level latent variables for stock prediction, while the diffusion probabilistic model trains the predictor to handle stock price stochasticity by progressively adding random noise to the stock data.

regression Stock Prediction +1

Context-aware Event Forecasting via Graph Disentanglement

1 code implementation12 Aug 2023 Yunshan Ma, Chenchen Ye, Zijian Wu, Xiang Wang, Yixin Cao, Tat-Seng Chua

The task of event forecasting aims to model the relational and temporal patterns based on historical events and makes forecasting to what will happen in the future.

Disentanglement Link Prediction

Constructing Holistic Spatio-Temporal Scene Graph for Video Semantic Role Labeling

no code implementations9 Aug 2023 Yu Zhao, Hao Fei, Yixin Cao, Bobo Li, Meishan Zhang, Jianguo Wei, Min Zhang, Tat-Seng Chua

A scene-event mapping mechanism is first designed to bridge the gap between the underlying scene structure and the high-level event semantic structure, resulting in an overall hierarchical scene-event (termed ICE) graph structure.

Semantic Role Labeling

LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation

no code implementations9 Aug 2023 Leigang Qu, Shengqiong Wu, Hao Fei, Liqiang Nie, Tat-Seng Chua

Afterward, we propose a fine-grained object-interaction diffusion method to synthesize high-faithfulness images conditioned on the prompt and the automatically generated layout.

Revisiting Disentanglement and Fusion on Modality and Context in Conversational Multimodal Emotion Recognition

no code implementations8 Aug 2023 Bobo Li, Hao Fei, Lizi Liao, Yu Zhao, Chong Teng, Tat-Seng Chua, Donghong Ji, Fei Li

On the other hand, during the feature fusion stage, we propose a Contribution-aware Fusion Mechanism (CFM) and a Context Refusion Mechanism (CRM) for multimodal and context integration, respectively.

Contrastive Learning Disentanglement +1

Empowering Vision-Language Models to Follow Interleaved Vision-Language Instructions

1 code implementation8 Aug 2023 Juncheng Li, Kaihang Pan, Zhiqi Ge, Minghe Gao, Hanwang Zhang, Wei Ji, Wenqiao Zhang, Tat-Seng Chua, Siliang Tang, Yueting Zhuang

To address this issue, we propose a generic and lightweight controllable knowledge re-injection module, which utilizes the sophisticated reasoning ability of LLMs to control the VPG to conditionally extract instruction-specific visual information and re-inject it into the LLM.

Image Captioning Instruction Following

SimTeG: A Frustratingly Simple Approach Improves Textual Graph Learning

2 code implementations3 Aug 2023 Keyu Duan, Qian Liu, Tat-Seng Chua, Shuicheng Yan, Wei Tsang Ooi, Qizhe Xie, Junxian He

More recently, with the rapid development of language models (LMs), researchers have focused on leveraging LMs to facilitate the learning of TGs, either by jointly training them in a computationally intensive framework (merging the two stages), or designing complex self-supervised training tasks for feature extraction (enhancing the first stage).

Feature Engineering Graph Learning +3

XNLP: An Interactive Demonstration System for Universal Structured NLP

no code implementations3 Aug 2023 Hao Fei, Meishan Zhang, Min Zhang, Tat-Seng Chua

Structured Natural Language Processing (XNLP) is an important subset of NLP that entails understanding the underlying semantic or syntactic structure of texts, which serves as a foundational component for many downstream applications.

Revisiting Conversation Discourse for Dialogue Disentanglement

no code implementations6 Jun 2023 Bobo Li, Hao Fei, Fei Li, Shengqiong Wu, Lizi Liao, Yinwei Wei, Tat-Seng Chua, Donghong Ji

Conversation utterances are essentially organized and described by the underlying discourse, and thus dialogue disentanglement requires the full understanding and harnessing of the intrinsic discourse attribute.


Discovering Dynamic Causal Space for DAG Structure Learning

1 code implementation5 Jun 2023 Fangfu Liu, Wenchang Ma, An Zhang, Xiang Wang, Yueqi Duan, Tat-Seng Chua

Discovering causal structure from purely observational data (i. e., causal discovery), aiming to identify causal relationships among variables, is a fundamental task in machine learning.

Causal Discovery Combinatorial Optimization

LLMDet: A Large Language Models Detection Tool

1 code implementation24 May 2023 Kangxi Wu, Liang Pang, HuaWei Shen, Xueqi Cheng, Tat-Seng Chua

With the advancement of generative language models, the generated text has come remarkably close to high-quality human-authored text in terms of fluency and diversity.

Language Modelling Large Language Model

Robust Instruction Optimization for Large Language Models with Distribution Shifts

no code implementations23 May 2023 Moxin Li, Wenjie Wang, Fuli Feng, Jizhi Zhang, Tat-Seng Chua

Large Language Models have demonstrated significant ability in accomplishing a wide range of Natural Language Processing (NLP) tasks.

Prompting and Evaluating Large Language Models for Proactive Dialogues: Clarification, Target-guided, and Non-collaboration

no code implementations23 May 2023 Yang Deng, Wenqiang Lei, Lizi Liao, Tat-Seng Chua

Conversational systems based on Large Language Models (LLMs), such as ChatGPT, show exceptional proficiency in context understanding and response generation.

Descriptive Response Generation

MacLaSa: Multi-Aspect Controllable Text Generation via Efficient Sampling from Compact Latent Space

no code implementations22 May 2023 Hanxing Ding, Liang Pang, Zihao Wei, HuaWei Shen, Xueqi Cheng, Tat-Seng Chua

Multi-aspect controllable text generation aims to generate fluent sentences that possess multiple desired attributes simultaneously.

Text Generation

InstructVid2Vid: Controllable Video Editing with Natural Language Instructions

no code implementations21 May 2023 Bosheng Qin, Juncheng Li, Siliang Tang, Tat-Seng Chua, Yueting Zhuang

To improve the consistency between adjacent frames of generated videos, we propose the Frame Difference Loss, which is incorporated during the training process.

Image Generation Style Transfer +1

Scene Graph as Pivoting: Inference-time Image-free Unsupervised Multimodal Machine Translation with Visual Scene Hallucination

1 code implementation20 May 2023 Hao Fei, Qian Liu, Meishan Zhang, Min Zhang, Tat-Seng Chua

In this work, we investigate a more realistic unsupervised multimodal machine translation (UMMT) setup, inference-time image-free UMMT, where the model is trained with source-text image pairs, and tested with only source-text inputs.

Multimodal Machine Translation Translation

Constructing Code-mixed Universal Dependency Forest for Unbiased Cross-lingual Relation Extraction

no code implementations20 May 2023 Hao Fei, Meishan Zhang, Min Zhang, Tat-Seng Chua

Latest efforts on cross-lingual relation extraction (XRE) aggressively leverage the language-consistent structural features from the universal dependency (UD) resource, while they may largely suffer from biased transfer (e. g., either target-biased or source-biased) due to the inevitable linguistic disparity between languages.

Relation Extraction

Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment

no code implementations20 May 2023 Shengqiong Wu, Hao Fei, Wei Ji, Tat-Seng Chua

Unpaired cross-lingual image captioning has long suffered from irrelevancy and disfluency issues, due to the inconsistencies of the semantic scene and syntax attributes during transfer.

Image Captioning Translation

Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling

1 code implementation19 May 2023 Shengqiong Wu, Hao Fei, Yixin Cao, Lidong Bing, Tat-Seng Chua

First, we represent the fine-grained semantic structures of the input image and text with the visual and textual scene graphs, which are further fused into a unified cross-modal graph (CMG).

Denoising Relation Extraction

Generating Visual Spatial Description via Holistic 3D Scene Understanding

1 code implementation19 May 2023 Yu Zhao, Hao Fei, Wei Ji, Jianguo Wei, Meishan Zhang, Min Zhang, Tat-Seng Chua

With an external 3D scene extractor, we obtain the 3D objects and scene features for input images, based on which we construct a target object-centered 3D spatial scene graph (Go3D-S2G), such that we model the spatial semantics of target objects within the holistic 3D scenes.

Scene Understanding Text Generation

Reasoning Implicit Sentiment with Chain-of-Thought Prompting

1 code implementation18 May 2023 Hao Fei, Bobo Li, Qian Liu, Lidong Bing, Fei Li, Tat-Seng Chua

While sentiment analysis systems try to determine the sentiment polarities of given targets based on the key opinion expressions in input texts, in implicit sentiment analysis (ISA) the opinion cues come in an implicit and obscure manner.

Common Sense Reasoning Sentiment Analysis

Dual Semantic Knowledge Composed Multimodal Dialog Systems

no code implementations17 May 2023 Xiaolin Chen, Xuemeng Song, Yinwei Wei, Liqiang Nie, Tat-Seng Chua

Thereafter, considering that the attribute knowledge and relation knowledge can benefit the responding to different levels of questions, we design a multi-level knowledge composition module in MDS-S2 to obtain the latent composed response representation.

Response Generation

A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects

no code implementations4 May 2023 Yang Deng, Wenqiang Lei, Wai Lam, Tat-Seng Chua

Proactive dialogue systems, related to a wide range of real-world conversational applications, equip the conversational agent with the capability of leading the conversation direction towards achieving pre-defined targets or fulfilling certain goals from the system side.

Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents with Semantic-Oriented Hierarchical Graphs

no code implementations3 May 2023 Fengbin Zhu, Chao Wang, Fuli Feng, Zifeng Ren, Moxin Li, Tat-Seng Chua

Discrete reasoning over table-text documents (e. g., financial reports) gains increasing attention in recent two years.

Transfer Visual Prompt Generator across LLMs

1 code implementation2 May 2023 Ao Zhang, Hao Fei, Yuan YAO, Wei Ji, Li Li, Zhiyuan Liu, Tat-Seng Chua

While developing a new vision-language LLM (VL-LLM) by pre-training on tremendous image-text pairs from scratch can be exceedingly resource-consuming, connecting an existing LLM with a comparatively lightweight visual prompt generator (VPG) becomes a feasible paradigm.

Transfer Learning

Search-in-the-Chain: Towards Accurate, Credible and Traceable Large Language Models for Knowledge-intensive Tasks

1 code implementation28 Apr 2023 Shicheng Xu, Liang Pang, HuaWei Shen, Xueqi Cheng, Tat-Seng Chua

Second, IR verifies the answer of each node of CoQ, it corrects the answer that is not consistent with the retrieved information when IR gives high confidence, which improves the credibility.

Fact Checking Information Retrieval +6

Learnable Pillar-based Re-ranking for Image-Text Retrieval

1 code implementation25 Apr 2023 Leigang Qu, Meng Liu, Wenjie Wang, Zhedong Zheng, Liqiang Nie, Tat-Seng Chua

Image-text retrieval aims to bridge the modality gap and retrieve cross-modal content based on semantic similarities.

Re-Ranking Retrieval +1

On the Robustness of Aspect-based Sentiment Analysis: Rethinking Model, Data, and Training

no code implementations19 Apr 2023 Hao Fei, Tat-Seng Chua, Chenliang Li, Donghong Ji, Meishan Zhang, Yafeng Ren

In this study, we propose to enhance the ABSA robustness by systematically rethinking the bottlenecks from all possible angles, including model, data, and training.

Aspect-Based Sentiment Analysis (ABSA) Contrastive Learning +1

LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model

1 code implementation13 Apr 2023 Hao Fei, Shengqiong Wu, Jingye Li, Bobo Li, Fei Li, Libo Qin, Meishan Zhang, Min Zhang, Tat-Seng Chua

Universally modeling all typical information extraction tasks (UIE) with one generative language model (GLM) has revealed great potential by the latest study, where various IE predictions are unified into a linearized hierarchical expression under a GLM.

Language Modelling UIE

Diffusion Recommender Model

1 code implementation11 Apr 2023 Wenjie Wang, Yiyan Xu, Fuli Feng, Xinyu Lin, Xiangnan He, Tat-Seng Chua

In light of the impressive advantages of Diffusion Models (DMs) over traditional generative models in image synthesis, we propose a novel Diffusion Recommender Model (named DiffRec) to learn the generative process in a denoising manner.

Denoising Image Generation +1

Generative Recommendation: Towards Next-generation Recommender Paradigm

1 code implementation7 Apr 2023 Wenjie Wang, Xinyu Lin, Fuli Feng, Xiangnan He, Tat-Seng Chua

However, such a retrieval-based recommender paradigm faces two limitations: 1) the human-generated items in the corpus might fail to satisfy the users' diverse information needs, and 2) users usually adjust the recommendations via passive and inefficient feedback such as clicks.

Recommendation Systems Retrieval +1

Causal Disentangled Recommendation Against User Preference Shifts

1 code implementation28 Mar 2023 Wenjie Wang, Xinyu Lin, Liuhui Wang, Fuli Feng, Yunshan Ma, Tat-Seng Chua

Inspired by the causal graph, our key considerations to handle preference shifts lie in modeling the interaction generation procedure by: 1) capturing the preference shifts across environments for accurate preference prediction, and 2) disentangling the sparse influence from user preference to interactions for accurate effect estimation of preference.

Recommendation Systems

Gradient-Regulated Meta-Prompt Learning for Generalizable Vision-Language Models

no code implementations ICCV 2023 Juncheng Li, Minghe Gao, Longhui Wei, Siliang Tang, Wenqiao Zhang, Mengze Li, Wei Ji, Qi Tian, Tat-Seng Chua, Yueting Zhuang

Prompt tuning, a recently emerging paradigm, enables the powerful vision-language pre-training models to adapt to downstream tasks in a parameter -- and data -- efficient way, by learning the ``soft prompts'' to condition frozen pre-training models.

Domain Generalization Few-Shot Learning

Boosting Differentiable Causal Discovery via Adaptive Sample Reweighting

1 code implementation6 Mar 2023 An Zhang, Fangfu Liu, Wenchang Ma, Zhibo Cai, Xiang Wang, Tat-Seng Chua

Despite great success in low-dimensional linear systems, it has been observed that these approaches overly exploit easier-to-fit samples, thus inevitably learning spurious edges.

Bilevel Optimization Causal Discovery

Contrastive Video Question Answering via Video Graph Transformer

1 code implementation27 Feb 2023 Junbin Xiao, Pan Zhou, Angela Yao, Yicong Li, Richang Hong, Shuicheng Yan, Tat-Seng Chua

CoVGT's uniqueness and superiority are three-fold: 1) It proposes a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations and dynamics, for complex spatio-temporal reasoning.

Ranked #2 on Video Question Answering on NExT-QA (using extra training data)

Contrastive Learning Question Answering +1

Invariant Collaborative Filtering to Popularity Distribution Shift

1 code implementation10 Feb 2023 An Zhang, Jingnan Zheng, Xiang Wang, Yancheng Yuan, Tat-Seng Chua

Collaborative Filtering (CF) models, despite their great success, suffer from severe performance drops due to popularity distribution shifts, where these changes are ubiquitous and inevitable in real-world scenarios.

Collaborative Filtering Representation Learning

Variational Cross-Graph Reasoning and Adaptive Structured Semantics Learning for Compositional Temporal Grounding

no code implementations22 Jan 2023 Juncheng Li, Siliang Tang, Linchao Zhu, Wenqiao Zhang, Yi Yang, Tat-Seng Chua, Fei Wu, Yueting Zhuang

To systematically benchmark the compositional generalizability of temporal grounding models, we introduce a new Compositional Temporal Grounding task and construct two new dataset splits, i. e., Charades-CG and ActivityNet-CG.

Semantic correspondence

Are Binary Annotations Sufficient? Video Moment Retrieval via Hierarchical Uncertainty-Based Active Learning

1 code implementation CVPR 2023 Wei Ji, Renjie Liang, Zhedong Zheng, Wenqiao Zhang, Shengyu Zhang, Juncheng Li, Mengze Li, Tat-Seng Chua

Moreover, we treat the uncertainty score of frames in a video as a whole, and estimate the difficulty of each video, which can further relieve the burden of video selection.

Active Learning Moment Retrieval +1

MRTNet: Multi-Resolution Temporal Network for Video Sentence Grounding

no code implementations26 Dec 2022 Wei Ji, Long Chen, Yinwei Wei, Yiming Wu, Tat-Seng Chua

In this work, we propose a novel multi-resolution temporal video sentence grounding network: MRTNet, which consists of a multi-modal feature encoder, a Multi-Resolution Temporal (MRT) module, and a predictor module.


Causal Inference for Knowledge Graph based Recommendation

1 code implementation20 Dec 2022 Yinwei Wei, Xiang Wang, Liqiang Nie, Shaoyu Li, Dingxian Wang, Tat-Seng Chua

Knowledge Graph (KG), as a side-information, tends to be utilized to supplement the collaborative filtering (CF) based recommendation model.

Collaborative Filtering Counterfactual Inference

Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark

1 code implementation19 Dec 2022 Jianwu Fang, Lei-Lei Li, Kuan Yang, Zhedong Zheng, Jianru Xue, Tat-Seng Chua

In particular, the text description provides a dense semantic description guidance for the primary context of the traffic scene, while the driver attention provides a traction to focus on the critical region closely correlating with safe driving.

Decision Making

Visually Grounded Commonsense Knowledge Acquisition

1 code implementation22 Nov 2022 Yuan YAO, Tianyu Yu, Ao Zhang, Mengdi Li, Ruobing Xie, Cornelius Weber, Zhiyuan Liu, Hai-Tao Zheng, Stefan Wermter, Tat-Seng Chua, Maosong Sun

In this work, we present CLEVER, which formulates CKE as a distantly supervised multi-instance learning problem, where models learn to summarize commonsense relations from a bag of images about an entity pair without any human annotation on image instances.

Language Modelling

Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization

1 code implementation14 Nov 2022 Yiyang Chen, Zhedong Zheng, Wei Ji, Leigang Qu, Tat-Seng Chua

The key idea underpinning the proposed method is to integrate fine- and coarse-grained retrieval as matching data points with small and large fluctuations, respectively.

Composed Image Retrieval (CoIR) Image Retrieval with Multi-Modal Query +1

PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation

1 code implementation14 Nov 2022 Mu Chen, Zhedong Zheng, Yi Yang, Tat-Seng Chua

In an attempt to fill this gap, we propose a unified pixel- and patch-wise self-supervised learning framework, called PiPa, for domain adaptive semantic segmentation that facilitates intra-image pixel-wise correlations and patch-wise semantic consistency against different contexts.

Self-Supervised Learning Semantic Segmentation +2

Behavioral Intention Prediction in Driving Scenes: A Survey

no code implementations1 Nov 2022 Jianwu Fang, Fan Wang, Peining Shen, Zhedong Zheng, Jianru Xue, Tat-Seng Chua

In the driving scene, the road participants usually show frequent interaction and intention understanding with the surrounding.

Trajectory Prediction

Incorporating Bias-aware Margins into Contrastive Loss for Collaborative Filtering

1 code implementation20 Oct 2022 An Zhang, Wenchang Ma, Xiang Wang, Tat-Seng Chua

Collaborative filtering (CF) models easily suffer from popularity bias, which makes recommendation deviate from users' actual preferences.

Collaborative Filtering

ConReader: Exploring Implicit Relations in Contracts for Contract Clause Extraction

1 code implementation17 Oct 2022 Weiwen Xu, Yang Deng, Wenqiang Lei, Wenlong Zhao, Tat-Seng Chua, Wai Lam

We study automatic Contract Clause Extraction (CCE) by modeling implicit relations in legal contracts.

Implicit Relations

Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendation

1 code implementation17 Aug 2022 Shengyu Zhang, Lingxiao Yang, Dong Yao, Yujie Lu, Fuli Feng, Zhou Zhao, Tat-Seng Chua, Fei Wu

Specifically, Re4 encapsulates three backward flows, i. e., 1) Re-contrast, which drives each interest embedding to be distinct from other interests using contrastive learning; 2) Re-attend, which ensures the interest-item correlation estimation in the forward flow to be consistent with the criterion used in final recommendation; and 3) Re-construct, which ensures that each interest embedding can semantically reflect the information of representative items that relate to the corresponding interest.

Contrastive Learning Recommendation Systems

CCL4Rec: Contrast over Contrastive Learning for Micro-video Recommendation

no code implementations17 Aug 2022 Shengyu Zhang, Bofang Li, Dong Yao, Fuli Feng, Jieming Zhu, Wenyan Fan, Zhou Zhao, Xiaofei He, Tat-Seng Chua, Fei Wu

Micro-video recommender systems suffer from the ubiquitous noises in users' behaviors, which might render the learned user representation indiscriminating, and lead to trivial recommendations (e. g., popular items) or even weird ones that are far beyond users' interests.

Contrastive Learning Recommendation Systems

Equivariant and Invariant Grounding for Video Question Answering

1 code implementation26 Jul 2022 Yicong Li, Xiang Wang, Junbin Xiao, Tat-Seng Chua

Specifically, the equivariant grounding encourages the answering to be sensitive to the semantic changes in the causal scene and question; in contrast, the invariant grounding enforces the answering to be insensitive to the changes in the environment scene.

Question Answering Video Question Answering

Towards Complex Document Understanding By Discrete Reasoning

no code implementations25 Jul 2022 Fengbin Zhu, Wenqiang Lei, Fuli Feng, Chao Wang, Haozhou Zhang, Tat-Seng Chua

Document Visual Question Answering (VQA) aims to understand visually-rich documents to answer questions in natural language, which is an emerging research topic for both Natural Language Processing and Computer Vision.

document understanding Question Answering +1

Video Graph Transformer for Video Question Answering

1 code implementation12 Jul 2022 Junbin Xiao, Pan Zhou, Tat-Seng Chua, Shuicheng Yan

VGT's uniqueness are two-fold: 1) it designs a dynamic graph transformer module which encodes video by explicitly capturing the visual objects, their relations, and dynamics for complex spatio-temporal reasoning; and 2) it exploits disentangled video and text Transformers for relevance comparison between the video and text to perform QA, instead of entangled cross-modal Transformer for answer classification.

Ranked #8 on Video Question Answering on NExT-QA (using extra training data)

Question Answering Video Question Answering +1

Structured and Natural Responses Co-generation for Conversational Search

1 code implementation ACM SIGIR Conference on Research and Development in Information Retrieval 2022 Chenchen Ye, Lizi Liao, Fuli Feng, Wei Ji, Tat-Seng Chua

Existing approaches either 1) predict structured dialog acts first and then generate natural response; or 2) map conversation context to natural responses directly in an end-to-end manner.

Conversational Search

Intelligent Request Strategy Design in Recommender System

no code implementations23 Jun 2022 Xufeng Qian, Yue Xu, Fuyu Lv, Shengyu Zhang, Ziwen Jiang, Qingwen Liu, Xiaoyi Zeng, Tat-Seng Chua, Fei Wu

RSs typically put a large number of items into one page to reduce excessive resource consumption from numerous paging requests, which, however, would diminish the RSs' ability to timely renew the recommendations according to users' real-time interest and lead to a poor user experience.

Causal Inference Recommendation Systems

Let Invariant Rationale Discovery Inspire Graph Contrastive Learning

1 code implementation16 Jun 2022 Sihang Li, Xiang Wang, An Zhang, Yingxin Wu, Xiangnan He, Tat-Seng Chua

Specifically, without supervision signals, RGCL uses a rationale generator to reveal salient features about graph instance-discrimination as the rationale, and then creates rationale-aware views for contrastive learning.

Contrastive Learning

Invariant Grounding for Video Question Answering

1 code implementation CVPR 2022 Yicong Li, Xiang Wang, Junbin Xiao, Wei Ji, Tat-Seng Chua

At its core is understanding the alignments between visual scenes in video and linguistic semantics in question to yield the answer.

Question Answering Video Question Answering

CrossCBR: Cross-view Contrastive Learning for Bundle Recommendation

1 code implementation1 Jun 2022 Yunshan Ma, Yingzhi He, An Zhang, Xiang Wang, Tat-Seng Chua

Recent methods usually take advantage of both user-bundle and user-item interactions information to obtain informative representations for users and bundles, corresponding to bundle view and item view, respectively.

Contrastive Learning Graph Learning

Differentiable Invariant Causal Discovery

no code implementations31 May 2022 Yu Wang, An Zhang, Xiang Wang, Yancheng Yuan, Xiangnan He, Tat-Seng Chua

This paper proposes Differentiable Invariant Causal Discovery (DICD), utilizing the multi-environment information based on a differentiable framework to avoid learning spurious edges and wrong causal directions.

Causal Discovery

PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models

1 code implementation23 May 2022 Yuan YAO, Qianyu Chen, Ao Zhang, Wei Ji, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun

We show that PEVL enables state-of-the-art performance of detector-free VLP models on position-sensitive tasks such as referring expression comprehension and phrase grounding, and also improves the performance on position-insensitive tasks with grounded inputs.

Language Modelling Phrase Grounding +5

3D Magic Mirror: Clothing Reconstruction from a Single Image via a Causal Perspective

1 code implementation27 Apr 2022 Zhedong Zheng, Jiayin Zhu, Wei Ji, Yi Yang, Tat-Seng Chua

This research aims to study a self-supervised 3D clothing reconstruction method, which recovers the geometry shape and texture of human clothing from a single image.

3D Reconstruction Person Re-Identification +2

Reinforced Causal Explainer for Graph Neural Networks

1 code implementation23 Apr 2022 Xiang Wang, Yingxin Wu, An Zhang, Fuli Feng, Xiangnan He, Tat-Seng Chua

Such reward accounts for the dependency of the newly-added edge and the previously-added edges, thus reflecting whether they collaborate together and form a coalition to pursue better explanations.

Graph Classification

Interacting with Non-Cooperative User: A New Paradigm for Proactive Dialogue Policy

no code implementations7 Apr 2022 Wenqiang Lei, Yao Zhang, Feifan Song, Hongru Liang, Jiaxin Mao, Jiancheng Lv, Zhenglu Yang, Tat-Seng Chua

To this end, we contribute to advance the study of the proactive dialogue policy to a more natural and challenging setting, i. e., interacting dynamically with users.

Video Question Answering: Datasets, Algorithms and Challenges

1 code implementation2 Mar 2022 Yaoyao Zhong, Junbin Xiao, Wei Ji, Yicong Li, Weihong Deng, Tat-Seng Chua

Video Question Answering (VideoQA) aims to answer natural language questions according to the given videos.

Question Answering Video Question Answering

KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems

3 code implementations22 Feb 2022 Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, Tat-Seng Chua

The progress of recommender systems is hampered mainly by evaluation as it requires real-time interactions between humans and systems, which is too laborious and expensive.

Recommendation Systems User Simulation

Discovering Invariant Rationales for Graph Neural Networks

1 code implementation ICLR 2022 Ying-Xin Wu, Xiang Wang, An Zhang, Xiangnan He, Tat-Seng Chua

Intrinsic interpretability of graph neural networks (GNNs) is to find a small subset of the input graph's features -- rationale -- which guides the model prediction.

Graph Classification

Deconfounding to Explanation Evaluation in Graph Neural Networks

no code implementations21 Jan 2022 Ying-Xin Wu, Xiang Wang, An Zhang, Xia Hu, Fuli Feng, Xiangnan He, Tat-Seng Chua

In this work, we propose Deconfounded Subgraph Evaluation (DSE) which assesses the causal effect of an explanatory subgraph on the model prediction.

A Dual Prompt Learning Framework for Few-Shot Dialogue State Tracking

no code implementations15 Jan 2022 Yuting Yang, Wenqiang Lei, Pei Huang, Juan Cao, Jintao Li, Tat-Seng Chua

In this paper, we focus on how to utilize the language understanding and generation ability of pre-trained language models for DST.

Dialogue State Tracking Language Modelling

Training Free Graph Neural Networks for Graph Matching

1 code implementation14 Jan 2022 Zhiyuan Liu, Yixin Cao, Fuli Feng, Xiang Wang, Jie Tang, Kenji Kawaguchi, Tat-Seng Chua

We present a framework of Training Free Graph Matching (TFGM) to boost the performance of Graph Neural Networks (GNNs) based graph matching, providing a fast promising solution without training (training-free).

Entity Alignment Graph Matching +1

Causal Attention for Interpretable and Generalizable Graph Classification

1 code implementation30 Dec 2021 Yongduo Sui, Xiang Wang, Jiancan Wu, Min Lin, Xiangnan He, Tat-Seng Chua

To endow the classifier with better interpretation and generalization, we propose the Causal Attention Learning (CAL) strategy, which discovers the causal patterns and mitigates the confounding effect of shortcuts.

Graph Attention Graph Classification

Video as Conditional Graph Hierarchy for Multi-Granular Question Answering

1 code implementation12 Dec 2021 Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, Tat-Seng Chua

To align with the multi-granular essence of linguistic concepts in language queries, we propose to model video as a conditional graph hierarchy which weaves together visual facts of different granularity in a level-wise manner, with the guidance of corresponding textual cues.

Question Answering Video Question Answering +1

Rethinking the Two-Stage Framework for Grounded Situation Recognition

1 code implementation10 Dec 2021 Meng Wei, Long Chen, Wei Ji, Xiaoyu Yue, Tat-Seng Chua

Since each verb is associated with a specific set of semantic roles, all existing GSR methods resort to a two-stage framework: predicting the verb in the first stage and detecting the semantic roles in the second stage.

Grounded Situation Recognition Object Recognition +1

Learning Robust Recommender from Noisy Implicit Feedback

1 code implementation2 Dec 2021 Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, Tat-Seng Chua

Inspired by this observation, we propose a new training strategy named Adaptive Denoising Training (ADT), which adaptively prunes the noisy interactions by two paradigms (i. e., Truncated Loss and Reweighted Loss).

Denoising Recommendation Systems

Towards Multi-Grained Explainability for Graph Neural Networks

1 code implementation NeurIPS 2021 Xiang Wang, Yingxin Wu, An Zhang, Xiangnan He, Tat-Seng Chua

A performant paradigm towards multi-grained explainability is until-now lacking and thus a focus of our work.

Inductive Lottery Ticket Learning for Graph Neural Networks

no code implementations29 Sep 2021 Yongduo Sui, Xiang Wang, Tianlong Chen, Xiangnan He, Tat-Seng Chua

In this work, we propose a simple and effective learning paradigm, Inductive Co-Pruning of GNNs (ICPG), to endow graph lottery tickets with inductive pruning capacity.

Graph Classification Node Classification +1

Decoupling Strategy and Surface Realization for Task-oriented Dialogues

no code implementations29 Sep 2021 Chenchen Ye, Lizi Liao, Fuli Feng, Wei Ji, Tat-Seng Chua

The core is to construct a latent content space for strategy optimization and disentangle the surface style from it.

Reinforcement Learning (RL) Style Transfer +1

Why Do We Click: Visual Impression-aware News Recommendation

1 code implementation26 Sep 2021 Jiahao Xun, Shengyu Zhang, Zhou Zhao, Jieming Zhu, Qi Zhang, Jingjie Li, Xiuqiang He, Xiaofei He, Tat-Seng Chua, Fei Wu

In this work, inspired by the fact that users make their click decisions mostly based on the visual impression they perceive when browsing news, we propose to capture such visual impression information with visual-semantic modeling for news recommendation.

Decision Making News Recommendation

CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models

1 code implementation24 Sep 2021 Yuan YAO, Ao Zhang, Zhengyan Zhang, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun

Pre-Trained Vision-Language Models (VL-PTMs) have shown promising capabilities in grounding natural language in image data, facilitating a broad variety of cross-modal tasks.

Visual Grounding

CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendation

no code implementations11 Sep 2021 Shengyu Zhang, Dong Yao, Zhou Zhao, Tat-Seng Chua, Fei Wu

In this paper, we propose to learn accurate and robust user representations, which are required to be less sensitive to (attack on) noisy behaviors and trust more on the indispensable ones, by modeling counterfactual data distribution.

Representation Learning Sequential Recommendation

How Knowledge Graph and Attention Help? A Qualitative Analysis into Bag-level Relation Extraction

1 code implementation ACL 2021 Zikun Hu, Yixin Cao, Lifu Huang, Tat-Seng Chua

In this paper, we contribute a dataset and propose a paradigm to quantitatively evaluate the effect of attention and KG on bag-level relation extraction (RE).

Relation Extraction

How Knowledge Graph and Attention Help? A Quantitative Analysis into Bag-level Relation Extraction

1 code implementation26 Jul 2021 Zikun Hu, Yixin Cao, Lifu Huang, Tat-Seng Chua

In this paper, we contribute a dataset and propose a paradigm to quantitatively evaluate the effect of attention and KG on bag-level relation extraction (RE).

Relation Extraction

MConv: An Environment for Multimodal Conversational Search across Multiple Domains

1 code implementation SIGIR 2021 Lizi Liao, Le Hong Long, Zheng Zhang, Minlie Huang, Tat-Seng Chua

Second, a set of benchmark results for dialogue state tracking, conversational recommendation, response generation as well as a unified model for multiple tasks are reported.

Conversational Search Dialogue State Tracking +1

NExT-QA: Next Phase of Question-Answering to Explaining Temporal Actions

1 code implementation CVPR 2021 Junbin Xiao, Xindi Shang, Angela Yao, Tat-Seng Chua

We introduce NExT-QA, a rigorously designed video question answering (VideoQA) benchmark to advance video understanding from describing to explaining the temporal actions.

Question Answering Video Question Answering +2

Empowering Language Understanding with Counterfactual Reasoning

1 code implementation Findings (ACL) 2021 Fuli Feng, Jizhi Zhang, Xiangnan He, Hanwang Zhang, Tat-Seng Chua

Present language understanding methods have demonstrated extraordinary ability of recognizing patterns in texts via machine learning.

Natural Language Inference Sentiment Analysis

Deconfounded Video Moment Retrieval with Causal Intervention

1 code implementation3 Jun 2021 Xun Yang, Fuli Feng, Wei Ji, Meng Wang, Tat-Seng Chua

To fill the research gap, we propose a causality-inspired VMR framework that builds structural causal model to capture the true effect of query and video content on the prediction.

Moment Retrieval Retrieval

Reproducibility Companion Paper: Knowledge Enhanced Neural Fashion Trend Forecasting

1 code implementation25 May 2021 Yunshan Ma, Yujuan Ding, Xun Yang, Lizi Liao, Wai Keung Wong, Tat-Seng Chua, Jinyoung Moon, Hong-Han Shuai

This companion paper supports the replication of the fashion trend forecasting experiments with the KERN (Knowledge Enhanced Recurrent Network) method that we presented in the ICMR 2020.

Deconfounded Recommendation for Alleviating Bias Amplification

1 code implementation22 May 2021 Wenjie Wang, Fuli Feng, Xiangnan He, Xiang Wang, Tat-Seng Chua

In this work, we scrutinize the cause-effect factors for bias amplification, identifying the main reason lies in the confounder effect of imbalanced item distribution on user representation and prediction score.

Fairness Recommendation Systems

NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actions

2 code implementations18 May 2021 Junbin Xiao, Xindi Shang, Angela Yao, Tat-Seng Chua

We introduce NExT-QA, a rigorously designed video question answering (VideoQA) benchmark to advance video understanding from describing to explaining the temporal actions.

Question Answering Video Question Answering +2

Leveraging Two Types of Global Graph for Sequential Fashion Recommendation

no code implementations17 May 2021 Yujuan Ding, Yunshan Ma, Wai Keung Wong, Tat-Seng Chua

Sequential fashion recommendation is of great significance in online fashion shopping, which accounts for an increasing portion of either fashion retailing or online e-commerce.

Vocal Bursts Valence Prediction

TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance

1 code implementation ACL 2021 Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, Tat-Seng Chua

In this work, we extract samples from real financial reports to build a new large-scale QA dataset containing both Tabular And Textual data, named TAT-QA, where numerical reasoning is usually required to infer the answer, such as addition, subtraction, multiplication, division, counting, comparison/sorting, and the compositions.

Question Answering

Leveraging Multiple Relations for Fashion Trend Forecasting Based on Social Media

no code implementations7 May 2021 Yujuan Ding, Yunshan Ma, Lizi Liao, Wai Keung Wong, Tat-Seng Chua

Towards insightful fashion trend forecasting, previous work [1] proposed to analyze more fine-grained fashion elements which can informatively reveal fashion trends.

Time Series Analysis

A-FMI: Learning Attributions from Deep Networks via Feature Map Importance

no code implementations12 Apr 2021 An Zhang, Xiang Wang, Chengfang Fang, Jie Shi, Tat-Seng Chua, Zehua Chen

Gradient-based attribution methods can aid in the understanding of convolutional neural networks (CNNs).

Conditional Hyper-Network for Blind Super-Resolution with Multiple Degradations

1 code implementation8 Apr 2021 Guanghao Yin, Wei Wang, Zehuan Yuan, Wei Ji, Dongdong Yu, Shouqian Sun, Tat-Seng Chua, Changhu Wang

We extract degradation prior at task-level with the proposed ConditionNet, which will be used to adapt the parameters of the basic SR network (BaseNet).

Blind Super-Resolution Image Super-Resolution

Learning Intents behind Interactions with Knowledge Graph for Recommendation

2 code implementations14 Feb 2021 Xiang Wang, Tinglin Huang, Dingxian Wang, Yancheng Yuan, Zhenguang Liu, Xiangnan He, Tat-Seng Chua

In this study, we explore intents behind a user-item interaction by using auxiliary item knowledge, and propose a new model, Knowledge Graph-based Intent Network (KGIN).

Recommendation Systems

Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering

no code implementations4 Jan 2021 Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, Tat-Seng Chua

Open-domain Question Answering (OpenQA) is an important task in Natural Language Processing (NLP), which aims to answer a question in the form of natural language based on large-scale unstructured documents.

Machine Reading Comprehension Open-Domain Question Answering

Causal Screening to Interpret Graph Neural Networks

no code implementations1 Jan 2021 Xiang Wang, Yingxin Wu, An Zhang, Xiangnan He, Tat-Seng Chua

In this work, we focus on the causal interpretability in GNNs and propose a method, Causal Screening, from the perspective of cause-effect.

Explanation Generation

Learning Relation Prototype from Unlabeled Texts for Long-tail Relation Extraction

1 code implementation27 Nov 2020 Yixin Cao, Jun Kuang, Ming Gao, Aoying Zhou, Yonggang Wen, Tat-Seng Chua

In this paper, we propose a general approach to learn relation prototypesfrom unlabeled texts, to facilitate the long-tail relation extraction by transferring knowledge from the relation types with sufficient trainingdata.

Relation Extraction Transfer Learning

Should Graph Convolution Trust Neighbors? A Simple Causal Inference Method

1 code implementation22 Oct 2020 Fuli Feng, Weiran Huang, Xiangnan He, Xin Xin, Qifan Wang, Tat-Seng Chua

To this end, we analyze the working mechanism of GCN with causal graph, estimating the causal effect of a node's local structure for the prediction.

Blocking Causal Inference +4

Clicks can be Cheating: Counterfactual Recommendation for Mitigating Clickbait Issue

1 code implementation21 Sep 2020 Wenjie Wang, Fuli Feng, Xiangnan He, Hanwang Zhang, Tat-Seng Chua

However, we argue that there is a significant gap between clicks and user satisfaction -- it is common that a user is "cheated" to click an item by the attractive title/cover of the item.

Click-Through Rate Prediction Counterfactual Inference

Multi-modal Cooking Workflow Construction for Food Recipes

no code implementations20 Aug 2020 Liangming Pan, Jingjing Chen, Jianlong Wu, Shaoteng Liu, Chong-Wah Ngo, Min-Yen Kan, Yu-Gang Jiang, Tat-Seng Chua

Understanding food recipe requires anticipating the implicit causal effects of cooking actions, such that the recipe can be converted into a graph describing the temporal workflow of the recipe.

Common Sense Reasoning

Neural Sparse Voxel Fields

1 code implementation NeurIPS 2020 Lingjie Liu, Jiatao Gu, Kyaw Zaw Lin, Tat-Seng Chua, Christian Theobalt

We also demonstrate several challenging tasks, including multi-scene learning, free-viewpoint rendering of a moving human, and large-scale scene rendering.

Tree-Augmented Cross-Modal Encoding for Complex-Query Video Retrieval

no code implementations6 Jul 2020 Xun Yang, Jianfeng Dong, Yixin Cao, Xun Wang, Meng Wang, Tat-Seng Chua

To facilitate video retrieval with complex queries, we propose a Tree-augmented Cross-modal Encoding method by jointly learning the linguistic structure of queries and the temporal representation of videos.

Retrieval Video Retrieval

Disentangled Graph Collaborative Filtering

2 code implementations3 Jul 2020 Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, Tat-Seng Chua

Such uniform approach to model user interests easily results in suboptimal representations, failing to model diverse relationships and disentangle user intents in representations.

Collaborative Filtering Disentanglement

Interactive Path Reasoning on Graph for Conversational Recommendation

no code implementations1 Jul 2020 Wenqiang Lei, Gangyi Zhang, Xiangnan He, Yisong Miao, Xiang Wang, Liang Chen, Tat-Seng Chua

Traditional recommendation systems estimate user preference on items from past interaction history, thus suffering from the limitations of obtaining fine-grained and dynamic user preference.

Recommendation Systems

Few-shot 3D Point Cloud Semantic Segmentation

1 code implementation CVPR 2021 Na Zhao, Tat-Seng Chua, Gim Hee Lee

These fully supervised approaches heavily rely on large amounts of labeled training data that are difficult to obtain and cannot segment new classes after training.

Few-shot 3D Point Cloud Semantic Segmentation Semantic Segmentation

Denoising Implicit Feedback for Recommendation

1 code implementation7 Jun 2020 Wenjie Wang, Fuli Feng, Xiangnan He, Liqiang Nie, Tat-Seng Chua

In this work, we explore the central theme of denoising implicit feedback for recommender training.

Denoising Recommendation Systems

Rethinking Dialogue State Tracking with Reasoning

no code implementations27 May 2020 Lizi Liao, Yunshan Ma, Wenqiang Lei, Tat-Seng Chua

Tracking dialogue states to better interpret user goals and feed downstream policy learning is a bottleneck in dialogue management.

Dialogue Management Dialogue State Tracking +1

Hierarchical Fashion Graph Network for Personalized Outfit Recommendation

1 code implementation26 May 2020 Xingchen Li, Xiang Wang, Xiangnan He, Long Chen, Jun Xiao, Tat-Seng Chua

Fashion outfit recommendation has attracted increasing attentions from online shopping services and fashion communities. Distinct from other scenarios (e. g., social networking or content sharing) which recommend a single item (e. g., a friend or picture) to a user, outfit recommendation predicts user preference on a set of well-matched fashion items. Hence, performing high-quality personalized outfit recommendation should satisfy two requirements -- 1) the nice compatibility of fashion items and 2) the consistence with user preference.

Seamlessly Unifying Attributes and Items: Conversational Recommendation for Cold-Start Users

1 code implementation23 May 2020 Shijun Li, Wenqiang Lei, Qingyun Wu, Xiangnan He, Peng Jiang, Tat-Seng Chua

In this work, we consider the conversational recommendation for cold-start users, where a system can both ask the attributes from and recommend items to a user interactively.

Collaborative Filtering Thompson Sampling

Knowledge Enhanced Neural Fashion Trend Forecasting

1 code implementation7 May 2020 Yunshan Ma, Yujuan Ding, Xun Yang, Lizi Liao, Wai Keung Wong, Tat-Seng Chua

Further-more, to effectively model the time series data of fashion elements with rather complex patterns, we propose a Knowledge EnhancedRecurrent Network model (KERN) which takes advantage of the capability of deep recurrent neural networks in modeling time-series data.

Time Series Time Series Analysis

Semantic Graphs for Generating Deep Questions

1 code implementation ACL 2020 Liangming Pan, Yuxi Xie, Yansong Feng, Tat-Seng Chua, Min-Yen Kan

This paper proposes the problem of Deep Question Generation (DQG), which aims to generate complex questions that require reasoning over multiple pieces of information of the input passage.

Question Generation Question-Generation

Learning Goal-oriented Dialogue Policy with Opposite Agent Awareness

no code implementations Asian Chapter of the Association for Computational Linguistics 2020 Zheng Zhang, Lizi Liao, Xiaoyan Zhu, Tat-Seng Chua, Zitao Liu, Yan Huang, Minlie Huang

Most existing approaches for goal-oriented dialogue policy learning used reinforcement learning, which focuses on the target agent policy and simply treat the opposite agent policy as part of the environment.

Decision Making

Reinforced Negative Sampling over Knowledge Graph for Recommendation

1 code implementation12 Mar 2020 Xiang Wang, Yaokun Xu, Xiangnan He, Yixin Cao, Meng Wang, Tat-Seng Chua

Properly handling missing data is a fundamental challenge in recommendation.

Cross-GCN: Enhancing Graph Convolutional Network with $k$-Order Feature Interactions

no code implementations5 Mar 2020 Fuli Feng, Xiangnan He, Hanwang Zhang, Tat-Seng Chua

Graph Convolutional Network (GCN) is an emerging technique that performs learning and reasoning on graph data.

Document Classification

Estimation-Action-Reflection: Towards Deep Interaction Between Conversational and Recommender Systems

no code implementations21 Feb 2020 Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, Tat-Seng Chua

Recommender systems are embracing conversational technologies to obtain user preferences dynamically, and to overcome inherent limitations of their static models.

Recommendation Systems

Multi-source Domain Adaptation for Visual Sentiment Classification

no code implementations12 Jan 2020 Chuang Lin, Sicheng Zhao, Lei Meng, Tat-Seng Chua

Existing domain adaptation methods on visual sentiment classification typically are investigated under the single-source scenario, where the knowledge learned from a source domain of sufficient labeled data is transferred to the target domain of loosely labeled or unlabeled data.

Classification Domain Adaptation +3

SESS: Self-Ensembling Semi-Supervised 3D Object Detection

1 code implementation CVPR 2020 Na Zhao, Tat-Seng Chua, Gim Hee Lee

The performance of existing point cloud-based 3D object detection methods heavily relies on large-scale high-quality 3D annotations.

3D Object Detection object-detection +1

Heuristic Black-box Adversarial Attacks on Video Recognition Models

1 code implementation21 Nov 2019 Zhipeng Wei, Jingjing Chen, Xingxing Wei, Linxi Jiang, Tat-Seng Chua, Fengfeng Zhou, Yu-Gang Jiang

To overcome this challenge, we propose a heuristic black-box attack model that generates adversarial perturbations only on the selected frames and regions.

Adversarial Attack Video Recognition

Mining Unfollow Behavior in Large-Scale Online Social Networks via Spatial-Temporal Interaction

1 code implementation17 Nov 2019 Haozhe Wu, Zhiyuan Hu, Jia Jia, Yaohua Bu, Xiangnan He, Tat-Seng Chua

Next, we define user's attributes as two categories: spatial attributes (e. g., social role of user) and temporal attributes (e. g., post content of user).


Semi-supervised Entity Alignment via Joint Knowledge Embedding Model and Cross-graph Model

1 code implementation IJCNLP 2019 Chengjiang Li, Yixin Cao, Lei Hou, Jiaxin Shi, Juanzi Li, Tat-Seng Chua

Specifically, as for the knowledge embedding model, we utilize TransE to implicitly complete two KGs towards consistency and learn relational constraints between entities.

Entity Alignment Graph Attention +1

Meta-Transfer Learning through Hard Tasks

1 code implementation7 Oct 2019 Qianru Sun, Yaoyao Liu, Zhaozheng Chen, Tat-Seng Chua, Bernt Schiele

In this paper, we propose a novel approach called meta-transfer learning (MTL) which learns to transfer the weights of a deep NN for few-shot learning tasks.

Few-Shot Learning Transfer Learning

Low-Resource Name Tagging Learned with Weakly Labeled Data

1 code implementation IJCNLP 2019 Yixin Cao, Zikun Hu, Tat-Seng Chua, Zhiyuan Liu, Heng Ji

Name tagging in low-resource languages or domains suffers from inadequate training data.


PS^2-Net: A Locally and Globally Aware Network for Point-Based Semantic Segmentation

1 code implementation15 Aug 2019 Na Zhao, Tat-Seng Chua, Gim Hee Lee

In this paper, we present the PS^2-Net -- a locally and globally aware deep learning framework for semantic segmentation on 3D scene-level point clouds.

Scene Segmentation

Who, Where, and What to Wear? Extracting Fashion Knowledge from Social Media

no code implementations12 Aug 2019 Yunshan Ma, Xun Yang, Lizi Liao, Yixin Cao, Tat-Seng Chua

We unify three tasks of occasion, person and clothing discovery from multiple modalities of images, texts and metadata.

Human Detection

Automatic Fashion Knowledge Extraction from Social Media

no code implementations12 Aug 2019 Yunshan Ma, Lizi Liao, Tat-Seng Chua

Fashion knowledge plays a pivotal role in helping people in their dressing.

Modeling Embedding Dimension Correlations via Convolutional Neural Collaborative Filtering

1 code implementation26 Jun 2019 Xiaoyu Du, Xiangnan He, Fajie Yuan, Jinhui Tang, Zhiguang Qin, Tat-Seng Chua

In this work, we emphasize on modeling the correlations among embedding dimensions in neural networks to pursue higher effectiveness for CF.

Collaborative Filtering Recommendation Systems

Deep Conversational Recommender in Travel

no code implementations25 Jun 2019 Lizi Liao, Ryuichi Takanobu, Yunshan Ma, Xun Yang, Minlie Huang, Tat-Seng Chua

When traveling to a foreign country, we are often in dire need of an intelligent conversational agent to provide instant and informative responses to our various queries.

Response Generation

Learning to Self-Train for Semi-Supervised Few-Shot Classification

1 code implementation NeurIPS 2019 Xinzhe Li, Qianru Sun, Yaoyao Liu, Shibao Zheng, Qin Zhou, Tat-Seng Chua, Bernt Schiele

On each task, we train a few-shot model to predict pseudo labels for unlabeled data, and then iterate the self-training steps on labeled and pseudo-labeled data with each step followed by fine-tuning.

Classification General Classification +1

Recent Advances in Neural Question Generation

no code implementations22 May 2019 Liangming Pan, Wenqiang Lei, Tat-Seng Chua, Min-Yen Kan

Emerging research in Neural Question Generation (NQG) has started to integrate a larger variety of inputs, and generating questions requiring higher levels of cognition.

Question Generation Question-Generation

KGAT: Knowledge Graph Attention Network for Recommendation

7 code implementations20 May 2019 Xiang Wang, Xiangnan He, Yixin Cao, Meng Liu, Tat-Seng Chua

To provide more accurate, diverse, and explainable recommendation, it is compulsory to go beyond modeling user-item interactions and take side information into account.

Explainable Recommendation Knowledge Graphs +1

Neural Graph Collaborative Filtering

18 code implementations20 May 2019 Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, Tat-Seng Chua

Further analysis verifies the importance of embedding propagation for learning better user and item representations, justifying the rationality and effectiveness of NGCF.

Collaborative Filtering Link Prediction +1

Graph Adversarial Training: Dynamically Regularizing Based on Graph Structure

1 code implementation20 Feb 2019 Fuli Feng, Xiangnan He, Jie Tang, Tat-Seng Chua

Adversarial Training (AT), a dynamic regularization technique, can resist the worst-case perturbations on input features and is a promising choice to improve model robustness and generalization.

General Classification Node Classification

Learning a Disentangled Embedding for Monocular 3D Shape Retrieval and Pose Estimation

no code implementations24 Dec 2018 Kyaw Zaw Lin, Weipeng Xu, Qianru Sun, Christian Theobalt, Tat-Seng Chua

We propose a novel approach to jointly perform 3D shape retrieval and pose estimation from monocular images. In order to make the method robust to real-world image variations, e. g. complex textures and backgrounds, we learn an embedding space from 3D data that only includes the relevant information, namely the shape and pose.

3D Object Retrieval 3D Shape Classification +3

Meta-Transfer Learning for Few-Shot Learning

2 code implementations CVPR 2019 Qianru Sun, Yaoyao Liu, Tat-Seng Chua, Bernt Schiele

In this paper we propose a novel few-shot learning method called meta-transfer learning (MTL) which learns to adapt a deep NN for few shot learning tasks.

Few-Shot Image Classification Few-Shot Learning +1