Search Results for author: Huaping Liu

Found 49 papers, 16 papers with code

Multi-Agent Embodied Question Answering in Interactive Environments

no code implementations ECCV 2020 Sinan Tan, Weilai Xiang, Huaping Liu, Di Guo, Fuchun Sun

We investigate a new AI task --- Multi-Agent Interactive Question Answering --- where several agents explore the scene jointly in interactive environments to answer a question.

3D Reconstruction Embodied Question Answering +1

Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting

no code implementations22 Mar 2024 Jun Guo, Xiaojian Ma, Yue Fan, Huaping Liu, Qing Li

Open-vocabulary 3D scene understanding presents a significant challenge in computer vision, withwide-ranging applications in embodied agents and augmented reality systems.

Scene Understanding Segmentation +2

Stimulate the Potential of Robots via Competition

no code implementations15 Mar 2024 Kangyao Huang, Di Guo, Xinyu Zhang, Xiangyang Ji, Huaping Liu

It is common for us to feel pressure in a competition environment, which arises from the desire to obtain success comparing with other individuals or opponents.

V3D: Video Diffusion Models are Effective 3D Generators

2 code implementations11 Mar 2024 Zilong Chen, Yikai Wang, Feng Wang, Zhengyi Wang, Huaping Liu

To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator.

Novel View Synthesis

Efficient Multi-scale Network with Learnable Discrete Wavelet Transform for Blind Motion Deblurring

no code implementations29 Dec 2023 Xin Gao, Tianheng Qiu, Xinyu Zhang, Hanlin Bai, Kang Liu, Xuan Huang, Hu Wei, Guoying Zhang, Huaping Liu

Coarse-to-fine schemes are widely used in traditional single-image motion deblur; however, in the context of deep learning, existing multi-scale algorithms not only require the use of complex modules for feature fusion of low-scale RGB images and deep semantics, but also manually generate low-resolution pairs of images that do not have sufficient confidence.

Computational Efficiency Deblurring

Rethinking Causal Relationships Learning in Graph Neural Networks

1 code implementation15 Dec 2023 Hang Gao, Chengyu Yao, Jiangmeng Li, Lingyu Si, Yifan Jin, Fengge Wu, Changwen Zheng, Huaping Liu

In order to comprehensively analyze various GNN models from a causal learning perspective, we constructed an artificially synthesized dataset with known and controllable causal relationships between data and labels.

EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models

1 code implementation27 Nov 2023 Sijie Cheng, Zhicheng Guo, Jingwen Wu, Kechen Fang, Peng Li, Huaping Liu, Yang Liu

However, the capability of VLMs to "think" from a first-person perspective, a crucial attribute for advancing autonomous agents and robotics, remains largely unexplored.

Attribute Question Answering +1

Vision-Language Foundation Models as Effective Robot Imitators

no code implementations2 Nov 2023 Xinghang Li, Minghuan Liu, Hanbo Zhang, Cunjun Yu, Jie Xu, Hongtao Wu, Chilam Cheang, Ya Jing, Weinan Zhang, Huaping Liu, Hang Li, Tao Kong

We believe RoboFlamingo has the potential to be a cost-effective and easy-to-use solution for robotics manipulation, empowering everyone with the ability to fine-tune their own robotics policy.

Imitation Learning

Fuzzy-NMS: Improving 3D Object Detection with Fuzzy Classification in NMS

no code implementations21 Oct 2023 Li Wang, Xinyu Zhang, Fachuan Zhao, Chuze Wu, Yichen Wang, Ziying Song, Lei Yang, Jun Li, Huaping Liu

The proposed Fuzzy-NMS module combines the volume and clustering density of candidate bounding boxes, refining them with a fuzzy classification method and optimizing the appropriate suppression thresholds to reduce uncertainty in the NMS process.

3D Object Detection object-detection

Text-to-3D using Gaussian Splatting

1 code implementation28 Sep 2023 Zilong Chen, Feng Wang, Huaping Liu

In this stage, we increase the number of Gaussians by compactness-based densification to enhance continuity and improve fidelity.

Text to 3D

SkipcrossNets: Adaptive Skip-cross Fusion for Road Detection

no code implementations24 Aug 2023 Xinyu Zhang, Yan Gong, Zhiwei Li, Xin Gao, Dafeng Jin, Jun Li, Huaping Liu

Multi-modal fusion is increasingly being used for autonomous driving tasks, as images from different modalities provide unique information for feature extraction.

Autonomous Driving

TrOMR:Transformer-Based Polyphonic Optical Music Recognition

1 code implementation18 Aug 2023 Yixuan Li, Huaping Liu, Qiang Jin, Miaomiao Cai, Peng Li

Optical Music Recognition (OMR) is an important technology in music and has been researched for a long time.

Heterogeneous Embodied Multi-Agent Collaboration

no code implementations26 Jul 2023 Xinzhu Liu, Di Guo, Huaping Liu

To study collaboration among heterogeneous agents, we propose the heterogeneous multi-agent tidying-up task, in which multiple heterogeneous agents with different capabilities collaborate with each other to detect misplaced objects and place them in reasonable locations.

object-detection Object Detection

TG-Critic: A Timbre-Guided Model for Reference-Independent Singing Evaluation

1 code implementation16 May 2023 Xiaoheng Sun, Yuejie Gao, Hanyao Lin, Huaping Liu

In this paper, a data-driven model TG-Critic is proposed to introduce timbre embeddings as one of the model inputs to guide the evaluation of singing quality.

Attribute

Path Planning for Air-Ground Robot Considering Modal Switching Point Optimization

no code implementations14 May 2023 Xiaoyu Wang, Kangyao Huang, Xinyu Zhang, Honglin Sun, Wenzhuo LIU, Huaping Liu, Jun Li, Pingping Lu

A robot for the field application environment was proposed, and a lightweight global spatial planning technique for the robot based on the graph-search algorithm taking mode switching point optimization into account, with an emphasis on energy efficiency, searching speed, and the viability of real deployment.

Informative Data Selection with Uncertainty for Multi-modal Object Detection

no code implementations23 Apr 2023 Xinyu Zhang, Zhiwei Li, Zhenhong Zou, Xin Gao, Yijin Xiong, Dafeng Jin, Jun Li, Huaping Liu

To quantify the correlation in multi-modal information, we model the uncertainty, as the inverse of data information, in different modalities and embed it in the bounding box generation.

Informativeness object-detection +1

Mixed Neural Voxels for Fast Multi-view Video Synthesis

1 code implementation ICCV 2023 Feng Wang, Sinan Tan, Xinghang Li, Zeyue Tian, Yafei Song, Huaping Liu

In this paper, we present a novel method named MixVoxels to better represent the dynamic scenes with fast training speed and competitive rendering qualities.

CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion

no code implementations6 Sep 2022 Li Wang, Xinyu Zhang, Wenyuan Qin, Xiaoyu Li, Lei Yang, Zhiwei Li, Lei Zhu, Hong Wang, Jun Li, Huaping Liu

As such, we propose a novel camera-LiDAR fusion 3D MOT framework based on the Combined Appearance-Motion Optimization (CAMO-MOT), which uses both camera and LiDAR data and significantly reduces tracking failures caused by occlusion and false detection.

3D Multi-Object Tracking Autonomous Driving +2

Continual Learning with Recursive Gradient Optimization

no code implementations ICLR 2022 Hao liu, Huaping Liu

Learning multiple tasks sequentially without forgetting previous knowledge, called Continual Learning(CL), remains a long-standing challenge for neural networks.

Continual Learning

An Automated Question-Answering Framework Based on Evolution Algorithm

no code implementations26 Jan 2022 Sinan Tan, Hui Xue, Qiyu Ren, Huaping Liu, Jing Bai

Our framework is based on an innovative evolution algorithm, which is stable and suitable for multiple dataset scenario.

Question Answering

Self-supervised 3D Semantic Representation Learning for Vision-and-Language Navigation

no code implementations26 Jan 2022 Sinan Tan, Mengmeng Ge, Di Guo, Huaping Liu, Fuchun Sun

In the Vision-and-Language Navigation task, the embodied agent follows linguistic instructions and navigates to a specific goal.

Representation Learning Test unseen +1

Rich Action-semantic Consistent Knowledge for Early Action Prediction

1 code implementation23 Jan 2022 Xiaoli Liu, Jianqin Yin, Di Guo, Huaping Liu

Next, we build a bi-directional semantic graph for the teacher network and a single-directional semantic graph for the student network to model rich ASCK among partial videos.

Early Action Prediction

Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge

no code implementations20 Sep 2021 Xinzhu Liu, Di Guo, Huaping Liu, Fuchun Sun

In this paper, we propose the multi-agent visual semantic navigation, in which multiple agents collaborate with others to find multiple target objects.

Efficient Exploration

A novel multimodal fusion network based on a joint coding model for lane line segmentation

no code implementations20 Mar 2021 Zhenhong Zou, Xinyu Zhang, Huaping Liu, Zhiwei Li, Amir Hussain, Jun Li

There has recently been growing interest in utilizing multimodal sensors to achieve robust lane line segmentation.

Understanding the Behaviour of Contrastive Loss

no code implementations CVPR 2021 Feng Wang, Huaping Liu

We will show that the contrastive loss is a hardness-aware loss function, and the temperature {\tau} controls the strength of penalties on hard negative samples.

Contrastive Learning

Unsupervised Representation Learning by Invariance Propagation

no code implementations NeurIPS 2020 Feng Wang, Huaping Liu, Di Guo, Sun Fuchun

In this paper, we propose Invariance Propagation to focus on learning representations invariant to category-level variations, which are provided by different instances from the same category.

Contrastive Learning Representation Learning +1

Unsupervised Representation Learning by InvariancePropagation

1 code implementation7 Oct 2020 Feng Wang, Huaping Liu, Di Guo, Fuchun Sun

In this paper, we propose Invariance Propagation to focus on learning representations invariant to category-level variations, which are provided by different instances from the same category.

Contrastive Learning Representation Learning +1

DeepSSM: Deep State-Space Model for 3D Human Motion Prediction

1 code implementation25 May 2020 Xiaoli Liu, Jianqin Yin, Huaping Liu, Jun Liu

In contrast to prior works, we improve the multi-order modeling ability of human motion systems for more accurate predictions by building a deep state-space model (DeepSSM).

Human motion prediction motion prediction

Towards Embodied Scene Description

no code implementations30 Apr 2020 Sinan Tan, Huaping Liu, Di Guo, Xin-Yu Zhang, Fuchun Sun

Embodiment is an important characteristic for all intelligent agents (creatures and robots), while existing scene description tasks mainly focus on analyzing images passively and the semantic understanding of the scenario is separated from the interaction between the agent and the environment.

Imitation Learning reinforcement-learning +1

Energy-based Periodicity Mining with Deep Features for Action Repetition Counting in Unconstrained Videos

no code implementations15 Mar 2020 Jianqin Yin, Yanchun Wu, Huaping Liu, Yonghao Dang, Zhiyi Liu, Jun Liu

Our work features two-fold: 1) An important insight that deep features extracted for action recognition can well model the self-similarity periodicity of the repetitive action is presented.

Action Recognition

MQA: Answering the Question via Robotic Manipulation

1 code implementation10 Mar 2020 Yuhong Deng, Di Guo, Xiaofeng Guo, Naifu Zhang, Huaping Liu, Fuchun Sun

In this paper, we propose a novel task, Manipulation Question Answering (MQA), where the robot performs manipulation actions to change the environment in order to answer a given question.

Imitation Learning Question Answering +1

FoveaBox: Beyound Anchor-based Object Detection

no code implementations ICLR 2020 Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Lei LI, Jianbo Shi

While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors.

Object object-detection +1

Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance

no code implementations16 Nov 2019 Mingxuan Jing, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Chao Yang, Bin Fang, Huaping Liu

In this paper, we study Reinforcement Learning from Demonstrations (RLfD) that improves the exploration efficiency of Reinforcement Learning (RL) by providing expert demonstrations.

reinforcement-learning Reinforcement Learning (RL)

TrajectoryNet: a new spatio-temporal feature learning network for human motion prediction

no code implementations15 Oct 2019 Xiaoli Liu, Jianqin Yin, Jin Liu, Pengxiang Ding, Jun Liu, Huaping Liu

And the global temporal co-occurrence features represent the co-occurrence relationship that different subsequences in a complex motion sequence are appeared simultaneously, which can be obtained automatically with our proposed TrajectoryNet by reorganizing the temporal information as the depth dimension of the input tensor.

Human motion prediction motion prediction +1

PISEP^2: Pseudo Image Sequence Evolution based 3D Pose Prediction

no code implementations arXiv:1909.01818 2019 Xiaoli Liu, Jianqin Yin, Huaping Liu, Yilong Yin

Specifically, a skeletal representation is proposed by transforming the joint coordinate sequence into an image sequence, which can model the different correlations of different joints.

Computational Efficiency Pose Prediction

FoveaBox: Beyond Anchor-based Object Detector

7 code implementations8 Apr 2019 Tao Kong, Fuchun Sun, Huaping Liu, Yuning Jiang, Lei LI, Jianbo Shi

In FoveaBox, an instance is assigned to adjacent feature levels to make the model more accurate. We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis.

Ranked #89 on Object Detection on COCO test-dev (APM metric)

Object object-detection +1

Deep Feature Pyramid Reconfiguration for Object Detection

no code implementations ECCV 2018 Tao Kong, Fuchun Sun, Wenbing Huang, Huaping Liu

In this paper, we begin by investigating current feature pyramids solutions, and then reformulate the feature pyramid construction as the feature reconfiguration process.

Object object-detection +1

Learning and Inferring Movement with Deep Generative Model

no code implementations18 May 2018 Mingxuan Jing, Xiaojian Ma, Fuchun Sun, Huaping Liu

Learning and inference movement is a very challenging problem due to its high dimensionality and dependency to varied environments or tasks.

Motion Planning

Task Transfer by Preference-Based Cost Learning

no code implementations12 May 2018 Mingxuan Jing, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Huaping Liu

The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task.

Sparse Coding and Dictionary Learning With Linear Dynamical Systems

no code implementations CVPR 2016 Wenbing Huang, Fuchun Sun, Lele Cao, Deli Zhao, Huaping Liu, Mehrtash Harandi

To enhance the performance of LDSs, in this paper, we address the challenging issue of performing sparse coding on the space of LDSs, where both data and dictionary atoms are LDSs.

Dictionary Learning Video Classification

Cannot find the paper you are looking for? You can Submit a new open access paper.