Search Results for author:
Found 61 papers, 20 papers with code
Multi-Agent Embodied Question Answering in Interactive Environments
We investigate a new AI task --- Multi-Agent Interactive Question Answering --- where several agents explore the scene jointly in interactive environments to answer a question.
Conditional Diffusion Model for Electrical Impedance Tomography
The method consists of a pre-imaging module, a conditional diffusion model for reconstruction, a forward voltage constraint network and a scheme of voltage consistency constraint during sampling process.
Towards Generalist Robot Policies: What Matters in Building Vision-Language-Action Models
The obtained results convince us firmly to explain why we need VLA and develop a new family of VLAs, RoboVLMs, which require very few manual designs and achieve a new state-of-the-art performance in three simulation tasks and real-world experiments.
Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach
To address these limitations, we propose a novel method that leverages the strengths of both LLM and GNN, allowing for the processing of graph data with any format and type of nodes and edges without the need for type information or special preprocessing.
Multimodal Information Bottleneck for Deep Reinforcement Learning with Multiple Sensors
We argue that compressing information in the learned joint representations about raw multimodal observations is helpful, and propose a multimodal information bottleneck model to learn task-relevant joint representations from egocentric images and proprioception.
AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environment
The increasing demand for intelligent assistants in human-populated environments has motivated significant research in autonomous robotic systems.
Observe Then Act: Asynchronous Active Vision-Action Model for Robotic Manipulation
In real-world scenarios, many robotic manipulation tasks are hindered by occlusions and limited fields of view, posing significant challenges for passive observation-based models that rely on fixed or wrist-mounted cameras.
Small Scale Data-Free Knowledge Distillation
In this line of research, existing methods typically follow an inversion-and-distillation paradigm in which a generative adversarial network on-the-fly trained with the guidance of the pre-trained teacher network is used to synthesize a large-scale sample set for knowledge distillation.
Data-free Knowledge Distillation
Generative Adversarial Network
+2
CompetEvo: Towards Morphological Evolution from Competition
Training an agent to adapt to specific tasks through co-optimization of morphology and control has widely attracted attention.
Sparse Spectral Training and Inference on Euclidean and Hyperbolic Neural Networks
The growing computational demands posed by increasingly number of neural network's parameters necessitate low-memory-consumption training approaches.
Unified End-to-End V2X Cooperative Autonomous Driving
V2X cooperation, through the integration of sensor data from both vehicles and infrastructure, is considered a pivotal approach to advancing autonomous driving technology.
Towards Objectively Benchmarking Social Intelligence for Language Agents at Action Level
To gauge the significance of agent architecture, we implement a target-driven planning (TDP) module as an adjunct to the existing agent.
Semantic Gaussians: Open-Vocabulary Scene Understanding with 3D Gaussian Splatting
Unlike existing methods, we design a versatile projection approach that maps various 2D semantic features from pre-trained image encoders into a novel semantic component of 3D Gaussians, which is based on spatial relationship and need no additional training.
Graph Partial Label Learning with Potential Cause Discovering
To address this issue, we have introduced Partial Label Learning (PLL) into graph representation learning.
Stimulate the Potential of Robots via Competition
It is common for us to feel pressure in a competition environment, which arises from the desire to obtain success comparing with other individuals or opponents.
V3D: Video Diffusion Models are Effective 3D Generators
To fully unleash the potential of video diffusion to perceive the 3D world, we further introduce geometrical consistency prior and extend the video diffusion model to a multi-view consistent 3D generator.
Efficient Multi-scale Network with Learnable Discrete Wavelet Transform for Blind Motion Deblurring
Coarse-to-fine schemes are widely used in traditional single-image motion deblur; however, in the context of deep learning, existing multi-scale algorithms not only require the use of complex modules for feature fusion of low-scale RGB images and deep semantics, but also manually generate low-resolution pairs of images that do not have sufficient confidence.
Ranked #1 on
Deblurring
on RSBlur
Rethinking Causal Relationships Learning in Graph Neural Networks
In order to comprehensively analyze various GNN models from a causal learning perspective, we constructed an artificially synthesized dataset with known and controllable causal relationships between data and labels.
EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models
However, the capability of VLMs to "think" from a first-person perspective, a crucial attribute for advancing autonomous agents and robotics, remains largely unexplored.
GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting
3D editing plays a crucial role in many areas such as gaming and virtual reality.
Vision-Language Foundation Models as Effective Robot Imitators
We believe RoboFlamingo has the potential to be a cost-effective and easy-to-use solution for robotics manipulation, empowering everyone with the ability to fine-tune their own robotics policy.
Fuzzy-NMS: Improving 3D Object Detection with Fuzzy Classification in NMS
The proposed Fuzzy-NMS module combines the volume and clustering density of candidate bounding boxes, refining them with a fuzzy classification method and optimizing the appropriate suppression thresholds to reduce uncertainty in the NMS process.
Text-to-3D using Gaussian Splatting
Specifically, our method adopts a progressive optimization strategy, which includes a geometry optimization stage and an appearance refinement stage.
SkipcrossNets: Adaptive Skip-cross Fusion for Road Detection
Specifically, skip-cross fusion strategy connects each layer to each layer in a feed-forward manner, and for each layer, the feature maps of all previous layers are used as input and its own feature maps are used as input to all subsequent layers for the other modality, enhancing feature propagation and multi-modal features fusion.
TrOMR:Transformer-Based Polyphonic Optical Music Recognition
Optical Music Recognition (OMR) is an important technology in music and has been researched for a long time.
Heterogeneous Embodied Multi-Agent Collaboration
To study collaboration among heterogeneous agents, we propose the heterogeneous multi-agent tidying-up task, in which multiple heterogeneous agents with different capabilities collaborate with each other to detect misplaced objects and place them in reasonable locations.
TG-Critic: A Timbre-Guided Model for Reference-Independent Singing Evaluation
In this paper, a data-driven model TG-Critic is proposed to introduce timbre embeddings as one of the model inputs to guide the evaluation of singing quality.
Path Planning for Air-Ground Robot Considering Modal Switching Point Optimization
A robot for the field application environment was proposed, and a lightweight global spatial planning technique for the robot based on the graph-search algorithm taking mode switching point optimization into account, with an emphasis on energy efficiency, searching speed, and the viability of real deployment.
Informative Data Selection with Uncertainty for Multi-modal Object Detection
To quantify the correlation in multi-modal information, we model the uncertainty, as the inverse of data information, in different modalities and embed it in the bounding box generation.
Deep Reinforcement Learning for Robotic Pushing and Picking in Cluttered Environment
A composite robotic hand composed of a suction cup and a gripper is designed for grasping the object stably.
Mixed Neural Voxels for Fast Multi-view Video Synthesis
In this paper, we present a novel method named MixVoxels to better represent the dynamic scenes with fast training speed and competitive rendering qualities.
CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusion
As such, we propose a novel camera-LiDAR fusion 3D MOT framework based on the Combined Appearance-Motion Optimization (CAMO-MOT), which uses both camera and LiDAR data and significantly reduces tracking failures caused by occlusion and false detection.
Sim2Real Object-Centric Keypoint Detection and Description
Keypoint detection and description play a central role in computer vision.
Continual Learning with Recursive Gradient Optimization
Learning multiple tasks sequentially without forgetting previous knowledge, called Continual Learning(CL), remains a long-standing challenge for neural networks.
An Automated Question-Answering Framework Based on Evolution Algorithm
Our framework is based on an innovative evolution algorithm, which is stable and suitable for multiple dataset scenario.
Self-supervised 3D Semantic Representation Learning for Vision-and-Language Navigation
In the Vision-and-Language Navigation task, the embodied agent follows linguistic instructions and navigates to a specific goal.
Rich Action-semantic Consistent Knowledge for Early Action Prediction
Next, we build a bi-directional semantic graph for the teacher network and a single-directional semantic graph for the student network to model rich ASCK among partial videos.
Self-Supervised Learning by Estimating Twin Class Distributions
To solve this problem, we propose to maximize the mutual information between the input and the class predictions.
Ranked #1 on
Image Classification
on Oxford-IIIT Pet Dataset
Fine-Grained Image Classification
Representation Learning
+5
Multi-Agent Embodied Visual Semantic Navigation with Scene Prior Knowledge
In this paper, we propose the multi-agent visual semantic navigation, in which multiple agents collaborate with others to find multiple target objects.
A novel multimodal fusion network based on a joint coding model for lane line segmentation
There has recently been growing interest in utilizing multimodal sensors to achieve robust lane line segmentation.
Understanding the Behaviour of Contrastive Loss
We will show that the contrastive loss is a hardness-aware loss function, and the temperature {\tau} controls the strength of penalties on hard negative samples.
Unsupervised Representation Learning by Invariance Propagation
In this paper, we propose Invariance Propagation to focus on learning representations invariant to category-level variations, which are provided by different instances from the same category.
Fault-Aware Robust Control via Adversarial Reinforcement Learning
Robots have limited adaptation ability compared to humans and animals in the case of damage.
Unsupervised Representation Learning by InvariancePropagation
In this paper, we propose Invariance Propagation to focus on learning representations invariant to category-level variations, which are provided by different instances from the same category.
DeepSSM: Deep State-Space Model for 3D Human Motion Prediction
In contrast to prior works, we improve the multi-order modeling ability of human motion systems for more accurate predictions by building a deep state-space model (DeepSSM).
Towards Embodied Scene Description
Embodiment is an important characteristic for all intelligent agents (creatures and robots), while existing scene description tasks mainly focus on analyzing images passively and the semantic understanding of the scenario is separated from the interaction between the agent and the environment.
Energy-based Periodicity Mining with Deep Features for Action Repetition Counting in Unconstrained Videos
Our work features two-fold: 1) An important insight that deep features extracted for action recognition can well model the self-similarity periodicity of the repetitive action is presented.
MQA: Answering the Question via Robotic Manipulation
In this paper, we propose a novel task, Manipulation Question Answering (MQA), where the robot performs manipulation actions to change the environment in order to answer a given question.
FoveaBox: Beyound Anchor-based Object Detection
While almost all state-of-the-art object detectors utilize predefined anchors to enumerate possible locations, scales and aspect ratios for the search of the objects, their performance and generalization ability are also limited to the design of anchors.
Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance
In this paper, we study Reinforcement Learning from Demonstrations (RLfD) that improves the exploration efficiency of Reinforcement Learning (RL) by providing expert demonstrations.
TrajectoryNet: a new spatio-temporal feature learning network for human motion prediction
And the global temporal co-occurrence features represent the co-occurrence relationship that different subsequences in a complex motion sequence are appeared simultaneously, which can be obtained automatically with our proposed TrajectoryNet by reorganizing the temporal information as the depth dimension of the input tensor.
Imitation Learning from Observations by Minimizing Inverse Dynamics Disagreement
This paper studies Learning from Observations (LfO) for imitation learning with access to state-only demonstrations.
Deep Point-wise Prediction for Action Temporal Proposal
Detecting actions in videos is an important yet challenging task.
PISEP^2: Pseudo Image Sequence Evolution based 3D Pose Prediction
Specifically, a skeletal representation is proposed by transforming the joint coordinate sequence into an image sequence, which can model the different correlations of different joints.
Ranked #1 on
Pose Prediction
on Filtered NTU RGB+D
FoveaBox: Beyond Anchor-based Object Detector
In FoveaBox, an instance is assigned to adjacent feature levels to make the model more accurate. We demonstrate its effectiveness on standard benchmarks and report extensive experimental analysis.
Ranked #91 on
Object Detection
on COCO test-dev
(APM metric)
Consistent Optimization for Single-Shot Object Detection
We present consistent optimization for single stage object detection.
Deep Feature Pyramid Reconfiguration for Object Detection
In this paper, we begin by investigating current feature pyramids solutions, and then reformulate the feature pyramid construction as the feature reconfiguration process.
Learning and Inferring Movement with Deep Generative Model
Learning and inference movement is a very challenging problem due to its high dimensionality and dependency to varied environments or tasks.
Task Transfer by Preference-Based Cost Learning
The goal of task transfer in reinforcement learning is migrating the action policy of an agent to the target task from the source task.
RON: Reverse Connection with Objectness Prior Networks for Object Detection
To address (a), we design the reverse connection, which enables the network to detect objects on multi-levels of CNNs.
Sparse Coding and Dictionary Learning With Linear Dynamical Systems
To enhance the performance of LDSs, in this paper, we address the challenging issue of performing sparse coding on the space of LDSs, where both data and dictionary atoms are LDSs.
