Search Results for author:
Found 23 papers, 7 papers with code
DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge
However, existing methods are limited to challenging image-based forecasting, which suffers from redundant information and lacks comprehensive and critical world knowledge, including dynamic, spatial and semantic information.
Ranked #1 on
Robot Manipulation
on CALVIN
OctoNav: Towards Generalist Embodied Navigation
Thus, we aim to investigate how to achieve thinking-before-action in the embodied navigation field, to improve model's reasoning ability toward generalists.
TrackVLA: Embodied Visual Tracking in the Wild
Embodied visual tracking is a fundamental skill in Embodied AI, enabling an agent to follow a specific target in dynamic environments using only egocentric vision.
LaDi-WM: A Latent Diffusion-based World Model for Predictive Manipulation
To this end, we propose LaDi-WM, a world model that predicts the latent space of future states using diffusion modeling.
OnlineAnySeg: Online Zero-Shot 3D Segmentation by Visual Foundation Model Guided 2D Mask Merging
Online zero-shot 3D instance segmentation of a progressively reconstructed scene is both a critical and challenging task for embodied applications.
3D Instance Segmentation
open vocabulary 3d instance segmentation
+2
SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation
Spatial intelligence is a critical component of embodied AI, promoting robots to understand and interact with their environments.
Ranked #1 on
Spatial Reasoning
on EmbSpatial-Bench
Neural Observation Field Guided Hybrid Optimization of Camera Placement
Consequently, most existing methods tackle this challenge by leveraging non-gradient-based optimization methods. In this work, we present a hybrid camera placement optimization approach that incorporates both gradient-based and non-gradient-based optimization methods.
CogNav: Cognitive Process Modeling for Object Goal Navigation with LLMs
Object goal navigation (ObjectNav) is a fundamental task in embodied AI, requiring an agent to locate a target object in previously unseen environments.
Uni-NaVid: A Video-based Vision-Language-Action Model for Unifying Embodied Navigation Tasks
A practical navigation agent must be capable of handling a wide range of interaction demands, such as following instructions, searching objects, answering questions, tracking people, and more.
GAPartManip: A Large-scale Part-centric Dataset for Material-Agnostic Articulated Object Manipulation
To address these challenges, we introduced a large-scale part-centric dataset for articulated object manipulation that features both photo-realistic material randomization and detailed annotations of part-oriented, scene-level actionable interaction poses.
Open6DOR: Benchmarking Open-instruction 6-DoF Object Rearrangement and A VLM-based Approach
In this work, we propel the pioneer construction of the benchmark and approach for table-top Open-instruction 6-DoF Object Rearrangement (Open6DOR).
Ranked #2 on
Object Rearrangement
on Open6DOR V2
NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation
Vision-and-language navigation (VLN) stands as a key research problem of Embodied AI, aiming at enabling agents to navigate in unseen environments following linguistic instructions.
A Unified Gaussian Process for Branching and Nested Hyperparameter Optimization
To capture the conditional dependence between branching and nested parameters, a unified Bayesian optimization framework is proposed.
MaskClustering: View Consensus based Mask Graph Clustering for Open-Vocabulary 3D Instance Segmentation
Through iterative clustering of masks showing high view consensus, we generate a series of clusters, each representing a distinct 3D instance.
3D Instance Segmentation
3D Open-Vocabulary Instance Segmentation
+6
FG-NeRF: Flow-GAN based Probabilistic Neural Radiance Field for Independence-Assumption-Free Uncertainty Estimation
Existing works rely on the independence assumption of points in the radiance field or the pixels in input views to obtain tractable forms of the probability density function.
GAMMA: Graspability-Aware Mobile MAnipulation Policy Learning based on Online Grasping Pose Fusion
Mobile manipulation constitutes a fundamental task for robotic assistants and garners significant attention within the robotics community.
MIPS-Fusion: Multi-Implicit-Submaps for Scalable and Robust Online Neural RGB-D Reconstruction
For the first time, randomized optimization is made possible in neural tracking with several key designs to the learning process, enabling efficient and robust tracking even under fast camera motions.
3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification
However, leveraging 3D scene representation can be prohibitively unpractical for policy learning in this floor-level task, due to low sample efficiency and expensive computational cost.
GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF
In this work, we tackle 6-DoF grasp detection for transparent and specular objects, which is an important yet challenging problem in vision-based robotic systems, due to the failure of depth cameras in sensing their geometry.
Tracking and Reconstructing Hand Object Interactions from Point Cloud Sequences in the Wild
We for the first time propose a point cloud based hand joint tracking network, HandTrackNet, to estimate the inter-frame hand joint motion.
ROSEFusion: Random Optimization for Online Dense Reconstruction under Fast Camera Motion
We propose to tackle the difficulties of fast-motion camera tracking in the absence of inertial measurements using random optimization, in particular, the Particle Filter Optimization (PFO).
Fusion-Aware Point Convolution for Online Semantic 3D Scene Segmentation
Online semantic 3D segmentation in company with real-time RGB-D reconstruction poses special challenges such as how to perform 3D convolution directly over the progressively fused 3D geometric data, and how to smartly fuse information from frame to frame.
Active Scene Understanding via Online Semantic Reconstruction
In our method, the exploratory robot scanning is both driven by and targeting at the recognition and segmentation of semantic objects from the scene.
