Search Results for author:
Found 7 papers, 0 papers with code
PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs
In each iteration, the image is annotated with a visual representation of proposals that the VLM can refer to (e. g., candidate robot actions, localizations, or trajectories).
AutoRT: Embodied Foundation Models for Large Scale Orchestration of Robotic Agents
We experimentally show that such "in-the-wild" data collected by AutoRT is significantly more diverse, and that AutoRT's use of LLMs allows for instruction following data collection robots that can align to human preferences.
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities
By training a VLM on such data, we significantly enhance its ability on both qualitative and quantitative spatial VQA.
RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches
Generalization remains one of the most important desiderata for robust robot learning systems.
Language to Rewards for Robotic Skill Synthesis
However, since low-level robot actions are hardware-dependent and underrepresented in LLM training corpora, existing efforts in applying LLMs to robotics have largely treated LLMs as semantic planners or relied on human-engineered control primitives to interface with the robot.
Open-World Object Manipulation using Pre-trained Vision-Language Models
This brings up a notably difficult challenge for robots: while robot learning approaches allow robots to learn many different behaviors from first-hand experience, it is impractical for robots to have first-hand experiences that span all of this semantic information.
Practical Imitation Learning in the Real World via Task Consistency Loss
To the best of our knowledge, this is the first work to tackle latched door opening from a purely end-to-end learning approach, where the task of navigation and manipulation are jointly modeled by a single neural network.
