1 code implementation • EMNLP 2020 • Zujie Liang, Weitao Jiang, Haifeng Hu, Jiaying Zhu
In the task of Visual Question Answering (VQA), most state-of-the-art models tend to learn spurious correlations in the training set and achieve poor performance in out-of-distribution test data.
no code implementations • 20 May 2023 • Yi Zhong, Chen Zhang, Xule Liu, Chenxi Sun, Weishan Deng, Haifeng Hu, Zhongqian Sun
EE-TTS contains an emphasis predictor that can identify appropriate emphasis positions from text and a conditioned acoustic model to synthesize expressive speech with emphasis and linguistic information.
no code implementations • 15 Dec 2022 • Sijie Mai, Ya Sun, Haifeng Hu
To assist the correlation learning, we feed the training pairs to the model according to difficulty by the proposed curriculum learning, which consists of elaborately designed scoring and feeding functions.
1 code implementation • 6 Dec 2022 • Yueming Yin, Haifeng Hu, Zhen Yang, Jitao Yang, Chun Ye, JianSheng Wu, Wilson Wen Bin Goh
However, this is non-ideal, as clumsy integration of incompatible models increases research overheads, and may even reduce success rates in drug discovery.
no code implementations • 22 Nov 2022 • Jianfeng Wu, Sijie Mai, Haifeng Hu
In this paper, we introduce Relation-dependent Contrastive Learning (ReCoLe) for inductive relation prediction, which adapts contrastive learning with a novel sampling method based on clustering algorithm to enhance the role of relation and improve the generalization ability to unseen relations.
1 code implementation • 31 Oct 2022 • Sijie Mai, Ying Zeng, Haifeng Hu
To this end, we introduce the multimodal information bottleneck (MIB), aiming to learn a powerful and sufficient multimodal representation that is free of redundancy and to filter out noisy information in unimodal representations.
Multimodal Emotion Recognition Multimodal Sentiment Analysis
no code implementations • 26 Oct 2022 • Ronghao Lin, Haifeng Hu
The former is like encoding robust uni-modal representation while the later is like integrating interactive information among different modalities, both of which are critical to learning an effective multimodal representation.
no code implementations • 20 Oct 2022 • Zhicong Huang, Jingwen Zhao, Zhijie Zheng, Dihu Chena, Haifeng Hu
In this paper, we propose a pillar set abstraction (PSA) and foreground point compensation (FPC) and design a point-based detection network, PSA-Det3D, to improve the detection performance for small object.
no code implementations • Findings (EMNLP) 2021 • Ying Zeng, Sijie Mai, Haifeng Hu
On the other hand, noisy information hidden in each modality interferes the learning of correct cross-modal dynamics.
no code implementations • 4 Sep 2021 • Sijie Mai, Ying Zeng, Shuangjia Zheng, Haifeng Hu
Specifically, we simultaneously perform intra-/inter-modal contrastive learning and semi-contrastive learning (that is why we call it hybrid contrastive learning), with which the model can fully explore cross-modal interactions, preserve inter-class relationships and reduce the modality gap.
1 code implementation • 17 Aug 2021 • Jianfeng Wu, Sijie Mai, Haifeng Hu
In this paper, we introduce Graph Capsule Aggregation (GraphCAGE) to model unaligned multimodal sequences with graph-based neural model and Capsule Network.
no code implementations • 26 Jul 2021 • Shuangjia Zheng, Sijie Mai, Ya Sun, Haifeng Hu, Yuedong Yang
In this way, we find the model can quickly adapt to few-shot relationships using only a handful of known facts with inductive settings.
1 code implementation • 29 May 2021 • Zujie Liang, Haifeng Hu, Jiaying Zhu
Most existing Visual Question Answering (VQA) systems tend to overly rely on language bias and hence fail to reason from the visual clue.
1 code implementation • 16 Dec 2020 • Sijie Mai, Shuangjia Zheng, Yuedong Yang, Haifeng Hu
Relation prediction for knowledge graphs aims at predicting missing relationships between entities.
no code implementations • 27 Nov 2020 • Sijie Mai, Songlong Xing, Jiaxuan He, Ying Zeng, Haifeng Hu
A majority of existing works generally focus on aligned fusion, mostly at word level, of the three modalities to accomplish this task, which is impractical in real-world scenarios.
no code implementations • 5 Nov 2020 • Yueming Yin, Zhen Yang, Haifeng Hu, Xiaofu Wu
Recent study reveals that knowledge can be transferred from one source domain to another unknown target domain, called Universal Domain Adaptation (UDA).
no code implementations • 10 Oct 2020 • Yueming Yin, Zhen Yang, Xiaofu Wu, Haifeng Hu
As a more practical setting for unsupervised domain adaptation, Universal Domain Adaptation (UDA) is recently introduced, where the target label set is unknown.
1 code implementation • CVPR 2020 • Haoxin Li, Wei-Shi Zheng, Yu Tao, Haifeng Hu, Jian-Huang Lai
We propose to search the network structures with differentiable architecture search mechanism, which learns to construct adaptive structures for different videos to facilitate adaptive interaction modeling.
no code implementations • 23 Apr 2020 • Yueming Yin, Zhen Yang, Haifeng Hu, Xiaofu Wu
Domain alignment (DA) has been widely used in unsupervised domain adaptation.
1 code implementation • 18 Nov 2019 • Sijie Mai, Haifeng Hu, Songlong Xing
Visualization of the learned embeddings suggests that the joint embedding space learned by our method is discriminative.
no code implementations • ACL 2019 • Sijie Mai, Haifeng Hu, Songlong Xing
We propose a general strategy named {`}divide, conquer and combine{'} for multimodal fusion.