no code implementations • ECCV 2020 • Sinan Tan, Weilai Xiang, Huaping Liu, Di Guo, Fuchun Sun
We investigate a new AI task --- Multi-Agent Interactive Question Answering --- where several agents explore the scene jointly in interactive environments to answer a question.
no code implementations • 20 Jan 2023 • Hang Gao, Jiangmeng Li, Wenwen Qiang, Lingyu Si, Xingzhe Su, Fengge Wu, Changwen Zheng, Fuchun Sun
Following the theoretical guidance, we innovatively introduce the auxiliary causal logic learning paradigm to improve the model to learn the expertise logic causally related to the graph representation learning task.
no code implementations • 22 Dec 2022 • Dazhao Du, Enhan Li, Lingyu Si, Fanjiang Xu, Fuchun Sun
We find the frames near the boundaries of action segments are in the transition region between two consecutive actions and have unclear semantics, which we call ambiguous intervals.
1 code implementation • 12 Dec 2022 • Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, Fuchun Sun
The early works in this domain mainly focus on static KGR and tend to directly apply general knowledge graph embedding models to the reasoning task.
no code implementations • 4 Dec 2022 • Xiao Li, Ziqi Wang, Bo Zhang, Fuchun Sun, Xiaolin Hu
The first stage of ROCK corresponds to the process of decomposing objects into parts in human vision.
no code implementations • 15 Oct 2022 • Tianying Ji, Yu Luo, Fuchun Sun, Mingxuan Jing, Fengxiang He, Wenbing Huang
Our follow-up derived bounds reveal the relationship between model shifts and performance improvement.
Model-based Reinforcement Learning
reinforcement-learning
+1
no code implementations • 4 Oct 2022 • Yinfeng Yu, Lele Cao, Fuchun Sun, Xiaohong Liu, Liejun Wang
Audio-visual embodied navigation, as a hot research topic, aims training a robot to reach an audio target using egocentric visual (from the sensors mounted on the robot) and audio (emitted from the target) input.
no code implementations • CVPR 2022 • Yikai Wang, TengQi Ye, Lele Cao, Wenbing Huang, Fuchun Sun, Fengxiang He, DaCheng Tao
Recently, there is a trend of leveraging multiple sources of input data, such as complementing the 3D point cloud with 2D images that often have richer color and fewer noises.
no code implementations • 26 Aug 2022 • Jiangmeng Li, Yanan Zhang, Wenwen Qiang, Lingyu Si, Chengbo Jiao, Xiaohui Hu, Changwen Zheng, Fuchun Sun
To understand the reasons behind this phenomenon, we revisit the learning paradigm of knowledge distillation on the few-shot object detection task from the causal theoretic standpoint, and accordingly, develop a Structural Causal Model.
no code implementations • 18 Aug 2022 • Hang Gao, Jiangmeng Li, Wenwen Qiang, Lingyu Si, Bing Xu, Changwen Zheng, Fuchun Sun
This observation reveals that there exist confounders in graphs, which may interfere with the model learning semantic information, and current graph representation learning methods have not eliminated their influence.
no code implementations • NeurIPS 2021 • Xueyi Liu, Yu Rong, Tingyang Xu, Fuchun Sun, Wenbing Huang, Junzhou Huang
To remedy this issue, we propose to select positive graph instances directly from existing graphs in the training set, which ultimately maintains the legality and similarity to the target graphs.
1 code implementation • 3 Jun 2022 • Chengliang Zhong, Peixing You, Xiaoxue Chen, Hao Zhao, Fuchun Sun, Guyue Zhou, Xiaodong Mu, Chuang Gan, Wenbing Huang
Detecting 3D keypoints from point clouds is important for shape reconstruction, while this work investigates the dual question: can shape reconstruction benefit 3D keypoint detection?
4 code implementations • CVPR 2022 • Yikai Wang, Xinghao Chen, Lele Cao, Wenbing Huang, Fuchun Sun, Yunhe Wang
Many adaptations of transformers have emerged to address the single-modal vision tasks, where self-attention modules are stacked to handle input sources like images.
Ranked #1 on
Semantic Segmentation
on SUN-RGBD
1 code implementation • 15 Mar 2022 • Runfa Chen, Yu Rong, Shangmin Guo, Jiaqi Han, Fuchun Sun, Tingyang Xu, Wenbing Huang
After the great success of Vision Transformer variants (ViTs) in computer vision, it has also demonstrated great potential in domain adaptive semantic segmentation.
Ranked #7 on
Domain Adaptation
on GTA5 to Cityscapes
1 code implementation • 12 Mar 2022 • Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, Junzhou Huang
The core of GMN is that it represents, by generalized coordinates, the forward kinematics information (positions and velocities) of a structural object.
1 code implementation • CVPR 2022 • Zhanhao Hu, Siyuan Huang, Xiaopei Zhu, Fuchun Sun, Bo Zhang, Xiaolin Hu
Experiments showed that these clothes could fool person detectors in the physical world.
1 code implementation • ICLR 2022 • Yinfeng Yu, Wenbing Huang, Fuchun Sun, Changan Chen, Yikai Wang, Xiaohong Liu
In this work, we design an acoustically complex environment in which, besides the target sound, there exists a sound attacker playing a zero-sum game with the agent.
no code implementations • 1 Feb 2022 • Chengliang Zhong, Chao Yang, Jinshan Qi, Fuchun Sun, Huaping Liu, Xiaodong Mu, Wenbing Huang
Keypoint detection and description play a central role in computer vision.
no code implementations • 26 Jan 2022 • Sinan Tan, Mengmeng Ge, Di Guo, Huaping Liu, Fuchun Sun
In the Vision-and-Language Navigation task, the embodied agent follows linguistic instructions and navigates to a specific goal.
1 code implementation • 11 Jan 2022 • Hang Gao, Jiangmeng Li, Wenwen Qiang, Lingyu Si, Fuchun Sun, Changwen Zheng
To this end, we propose a novel approach to learning a graph augmenter that can generate an augmentation with uniformity and informativeness.
1 code implementation • 4 Dec 2021 • Yikai Wang, Fuchun Sun, Wenbing Huang, Fengxiang He, DaCheng Tao
For the application of dense image prediction, the validity of CEN is tested by four different scenarios: multimodal fusion, cycle multimodal fusion, multitask learning, and multimodal multitask learning.
Ranked #13 on
Semantic Segmentation
on NYU Depth v2
1 code implementation • ICCV 2021 • Yikai Wang, Yi Yang, Fuchun Sun, Anbang Yao
In the low-bit quantization field, training Binary Neural Networks (BNNs) is the extreme solution to ease the deployment of deep models on resource-constrained devices, having the lowest storage cost and significantly cheaper bit-wise operations compared to 32-bit floating-point counterparts.
no code implementations • ICLR 2022 • Wenbing Huang, Jiaqi Han, Yu Rong, Tingyang Xu, Fuchun Sun, Junzhou Huang
In this manner, the geometrical constraints are implicitly and naturally encoded in the forward kinematics.
no code implementations • 20 Sep 2021 • Xinzhu Liu, Di Guo, Huaping Liu, Fuchun Sun
In this paper, we propose the multi-agent visual semantic navigation, in which multiple agents collaborate with others to find multiple target objects.
no code implementations • 11 Aug 2021 • Yikai Wang, Fuchun Sun, Ming Lu, Anbang Yao
We propose a compact and effective framework to fuse multimodal features at multiple layers in a single network.
Ranked #20 on
Semantic Segmentation
on NYU Depth v2
1 code implementation • 11 Aug 2021 • Yikai Wang, Wenbing Huang, Bin Fang, Fuchun Sun, Chang Li
By contrast, EIP models the tactile sensor as a group of coordinated particles, and the elastic property is applied to regulate the deformation of particles during contact.
1 code implementation • 10 Jun 2021 • Mingxuan Jing, Wenbing Huang, Fuchun Sun, Xiaojian Ma, Tao Kong, Chuang Gan, Lei LI
In particular, we propose an Expectation-Maximization(EM)-style algorithm: an E-step that samples the options of expert conditioned on the current learned policy, and an M-step that updates the low- and high-level policies of agent simultaneously to minimize the newly proposed option-occupancy measurement between the expert and the agent.
no code implementations • 23 Nov 2020 • Yikai Wang, Wenbing Huang, Bin Fang, Fuchun Sun
At its core, EIP models the tactile sensor as a group of coordinated particles, and the elastic theory is applied to regulate the deformation of particles during the contact process.
no code implementations • 17 Nov 2020 • Fan Yang, Chao Yang, Di Guo, Huaping Liu, Fuchun Sun
Robots have limited adaptation ability compared to humans and animals in the case of damage.
1 code implementation • NeurIPS 2020 • Yikai Wang, Wenbing Huang, Fuchun Sun, Tingyang Xu, Yu Rong, Junzhou Huang
Deep multimodal fusion by using multiple sources of data for classification or regression has exhibited a clear advantage over the unimodal counterpart on various applications.
1 code implementation • 7 Oct 2020 • Feng Wang, Huaping Liu, Di Guo, Fuchun Sun
In this paper, we propose Invariance Propagation to focus on learning representations invariant to category-level variations, which are provided by different instances from the same category.
no code implementations • 22 Aug 2020 • Wenbing Huang, Yu Rong, Tingyang Xu, Fuchun Sun, Junzhou Huang
Increasing the depth of GCN, which is expected to permit more expressivity, is shown to incur performance detriment especially on node classification.
2 code implementations • ECCV 2020 • Xingyu Chen, Xuguang Lan, Fuchun Sun, Nanning Zheng
Using a gating mechanism that discriminates the unseen samples from the seen samples can decompose the GZSL problem to a conventional Zero-Shot Learning (ZSL) problem and a supervised classification problem.
1 code implementation • ECCV 2020 • Yikai Wang, Fuchun Sun, Duo Li, Anbang Yao
We propose a general method to train a single convolutional neural network which is capable of switching image resolutions at inference.
no code implementations • 30 Apr 2020 • Sinan Tan, Huaping Liu, Di Guo, Xin-Yu Zhang, Fuchun Sun
Embodiment is an important characteristic for all intelligent agents (creatures and robots), while existing scene description tasks mainly focus on analyzing images passively and the semantic understanding of the scenario is separated from the interaction between the agent and the environment.
1 code implementation • 11 Mar 2020 • Shuang Li, Jiaxi Jiang, Philipp Ruppel, Hongzhuo Liang, Xiaojian Ma, Norman Hendrich, Fuchun Sun, Jianwei Zhang
In this paper, we present a multimodal mobile teleoperation system that consists of a novel vision-based hand pose regression network (Transteleop) and an IMU-based arm tracking method.
1 code implementation • 10 Mar 2020 • Yuhong Deng, Di Guo, Xiaofeng Guo, Naifu Zhang, Huaping Liu, Fuchun Sun
In this paper, we propose a novel task, Manipulation Question Answering (MQA), where the robot performs manipulation actions to change the environment in order to answer a given question.
1 code implementation • 29 Feb 2020 • Hongzhuo Liang, Chuangchuang Zhou, Shuang Li, Xiaojian Ma, Norman Hendrich, Timo Gerkmann, Fuchun Sun, Marcus Stoffel, Jianwei Zhang
Both network training results and robot experiments demonstrate that MP-Net is robust against noise and changes to the task and environment.
2 code implementations • CVPR 2020 • Runfa Chen, Wenbing Huang, Binghui Huang, Fuchun Sun, Bin Fang
The proposed architecture, termed as NICE-GAN, exhibits two advantageous patterns over previous approaches: First, it is more compact since no independent encoding component is required; Second, this plug-in encoder is directly trained by the adversary loss, making it more informative and trained more effectively if a multi-scale discriminator is applied.
no code implementations • ICLR 2020 • Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Lei LI, Jianbo Shi
While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors.
no code implementations • 16 Nov 2019 • Mingxuan Jing, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Chao Yang, Bin Fang, Huaping Liu
In this paper, we study Reinforcement Learning from Demonstrations (RLfD) that improves the exploration efficiency of Reinforcement Learning (RL) by providing expert demonstrations.
no code implementations • 3 Nov 2019 • Yikai Wang, Liang Zhang, Quanyu Dai, Fuchun Sun, Bo Zhang, Yang He, Weipeng Yan, Yongjun Bao
In deep CTR models, exploiting users' historical data is essential for learning users' behaviors and interests.
no code implementations • NeurIPS 2019 • Chao Yang, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Huaping Liu, Junzhou Huang, Chuang Gan
This paper studies Learning from Observations (LfO) for imitation learning with access to state-only demonstrations.
1 code implementation • 17 Sep 2019 • Luxuan Li, Tao Kong, Fuchun Sun, Huaping Liu
Detecting actions in videos is an important yet challenging task.
no code implementations • 25 Apr 2019 • Chuanqi Tan, Fuchun Sun, Tao Kong, Bin Fang, Wenchang Zhang
Different functional areas of the human brain play different roles in brain activity, which has not been paid sufficient research attention in the brain-computer interface (BCI) field.
6 code implementations • 8 Apr 2019 • Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Lei LI, Jianbo Shi
In FoveaBox, an instance is assigned to adjacent feature levels to make the model more accurate. We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis.
Ranked #93 on
Object Detection
on COCO test-dev
(APM metric)
no code implementations • 19 Jan 2019 • Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Jianbo Shi
We present consistent optimization for single stage object detection.
4 code implementations • 17 Sep 2018 • Shuang Li, Xiaojian Ma, Hongzhuo Liang, Michael Görner, Philipp Ruppel, Bing Fang, Fuchun Sun, Jianwei Zhang
In this paper, we present TeachNet, a novel neural network architecture for intuitive and markerless vision-based teleoperation of dexterous robotic hands.
Robotics
4 code implementations • 17 Sep 2018 • Hongzhuo Liang, Xiaojian Ma, Shuang Li, Michael Görner, Song Tang, Bin Fang, Fuchun Sun, Jianwei Zhang
In this paper, we propose an end-to-end grasp evaluation model to address the challenging problem of localizing robot grasp configurations directly from the point cloud.
Robotics
no code implementations • ECCV 2018 • Tao Kong, Fuchun Sun, Wenbing Huang, Huaping Liu
In this paper, we begin by investigating current feature pyramids solutions, and then reformulate the feature pyramid construction as the feature reconfiguration process.
no code implementations • 6 Aug 2018 • Chuanqi Tan, Fuchun Sun, Wenchang Zhang
First, we model cognitive events based on EEG data by characterizing the data using EEG optical flow, which is designed to preserve multimodal EEG information in a uniform representation.
no code implementations • 6 Aug 2018 • Chuanqi Tan, Fuchun Sun, Tao Kong, Wenchang Zhang, Chao Yang, Chunfang Liu
As a new classification platform, deep learning has recently received increasing attention from researchers and has been successfully applied to many domains.
no code implementations • 24 Jul 2018 • Chuanqi Tan, Fuchun Sun, Wenchang Zhang, Jianhua Chen, Chunfang Liu
Herein, we propose a novel approach to modeling cognitive events from EEG data by reducing it to a video classification problem, which is designed to preserve the multimodal information of EEG.
no code implementations • 18 May 2018 • Mingxuan Jing, Xiaojian Ma, Fuchun Sun, Huaping Liu
Learning and inference movement is a very challenging problem due to its high dimensionality and dependency to varied environments or tasks.
no code implementations • 12 May 2018 • Mingxuan Jing, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Huaping Liu
The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task.
no code implementations • NeurIPS 2017 • Wenbing Huang, Mehrtash Harandi, Tong Zhang, Lijie Fan, Fuchun Sun, Junzhou Huang
Linear Dynamical Systems (LDSs) are fundamental tools for modeling spatio-temporal data in various disciplines.
1 code implementation • CVPR 2017 • Tao Kong, Fuchun Sun, Anbang Yao, Huaping Liu, Ming Lu, Yurong Chen
To address (a), we design the reverse connection, which enables the network to detect objects on multi-levels of CNNs.
no code implementations • 18 Feb 2017 • Chang Liu, Fuchun Sun, Changhu Wang, Feng Wang, Alan Yuille
In this way, the sequential representation of an image can be naturally translated to a sequence of words, as the target sequence of the RNN model.
1 code implementation • 3 Aug 2016 • Wenbing Huang, Fuchun Sun, Lele Cao, Mehrtash Harandi
We then devise efficient algorithms to perform sparse coding and dictionary learning on the space of infinite-dimensional subspaces.
no code implementations • CVPR 2016 • Wenbing Huang, Fuchun Sun, Lele Cao, Deli Zhao, Huaping Liu, Mehrtash Harandi
To enhance the performance of LDSs, in this paper, we address the challenging issue of performing sparse coding on the space of LDSs, where both data and dictionary atoms are LDSs.
no code implementations • CVPR 2016 • Tao Kong, Anbang Yao, Yurong Chen, Fuchun Sun
Almost all of the current top-performing object detection networks employ region proposals to guide the search for object instances.