Search Results for author:
Found 38 papers, 18 papers with code
The Lighthouse of Language: Enhancing LLM Agents via Critique-Guided Improvement
In this work, we introduce Critique-Guided Improvement (CGI), a novel two-player framework, comprising an actor model that explores an environment and a critic model that generates detailed nature language feedback.
WildLMa: Long Horizon Loco-Manipulation in the Wild
'In-the-wild' mobile manipulation aims to deploy robots in diverse real-world environments, which requires the robot to (1) have skills that generalize across object configurations; (2) be capable of long-horizon task execution in diverse environments; and (3) perform complex manipulation beyond pick-and-place.
LoGU: Long-form Generation with Uncertainty Expressions
To tackle these challenges, we propose a refinement-based data collection framework and a two-stage training pipeline.
Atomic Calibration of LLMs in Long-Form Generations
Existing research on LLM calibration has primarily focused on short-form tasks, providing a single confidence score at the response level (macro calibration).
ACE: A Cross-Platform Visual-Exoskeletons System for Low-Cost Dexterous Teleoperation
Compared to previous systems, which often require hardware customization according to different robots, our single system can generalize to humanoid hands, arm-hands, arm-gripper, and quadruped-gripper systems with high-precision teleoperation.
Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning
Our system's ability to handle bimanual manipulations while prioritizing safety and real-time performance makes it a powerful tool for advancing dexterous manipulation and imitation learning.
SelfGoal: Your Language Agents Already Know How to Achieve High-level Goals
Language agents powered by large language models (LLMs) are increasingly valuable as decision-making tools in domains such as gaming and programming.
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models
Vision Language Models (VLMs) have demonstrated remarkable performance in 2D vision and language tasks.
Fast Samplers for Inverse Problems in Iterative Refinement Models
Constructing fast samplers for unconditional diffusion and flow-matching models has received much attention recently; however, existing methods for solving inverse problems, such as super-resolution, inpainting, or deblurring, still require hundreds to thousands of iterative steps to obtain high-quality results.
From Persona to Personalization: A Survey on Role-Playing Language Agents
Through this work, we aim to establish a clear taxonomy of RPLA research and applications, and facilitate future research in this critical and ever-evolving field, and pave the way for a future where humans and RPLAs coexist in harmony.
Visual Whole-Body Control for Legged Loco-Manipulation
We propose a framework that can conduct the whole-body control autonomously with visual observations.
Learning Generalizable Feature Fields for Mobile Manipulation
An open problem in mobile manipulation is how to represent objects and scenes in a unified manner so that robots can use both for navigation and manipulation.
Expressive Whole-Body Control for Humanoid Robots
Can we enable humanoid robots to generate rich, diverse, and expressive motions in the real world?
GumbelSoft: Diversified Language Model Watermarking via the GumbelMax-trick
Large language models (LLMs) excellently generate human-like text, but also raise concerns about misuse in fake news and academic dishonesty.
Harmonic Mobile Manipulation
Recent advancements in robotics have enabled robots to navigate complex scenes or manipulate diverse objects independently.
Precipitation Downscaling with Spatiotemporal Video Diffusion
In climate science and meteorology, high-resolution local precipitation (rain and snowfall) predictions are limited by the computational costs of simulation-based methods.
CMMD: Contrastive Multi-Modal Diffusion for Video-Audio Conditional Modeling
We introduce a multi-modal diffusion model tailored for the bi-directional conditional generation of video and audio.
Generalized Animal Imitator: Agile Locomotion with Versatile Motion Prior
This paper introduces the Versatile Instructable Motion prior (VIM) - a Reinforcement Learning framework designed to incorporate a range of agile locomotion tasks suitable for advanced robotic applications.
Neural Volumetric Memory for Visual Locomotion Control
To solve this problem, we follow the paradigm in computer vision that explicitly models the 3D geometry of the scene and propose Neural Volumetric Memory (NVM), a geometric memory architecture that explicitly accounts for the SE(3) equivariance of the 3D world.
Lossy Image Compression with Conditional Diffusion Models
This paper outlines an end-to-end optimized lossy image compression framework using diffusion generative models.
Diffusion Probabilistic Modeling for Video Generation
Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation.
SC2 Benchmark: Supervised Compression for Split Computing
With the increasing demand for deep learning models on mobile devices, splitting neural network computation between the device and a more powerful edge server has become an attractive solution.
Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization
While Reinforcement Learning (RL) provides a promising paradigm for agile locomotion skills with vision inputs in simulation, it is still very challenging to deploy the RL policy in the real world.
Supervised Compression for Resource-Constrained Edge Computing Systems
There has been much interest in deploying deep learning algorithms on low-powered devices, including smartphones, drones, and medical sensors.
DexMV: Imitation Learning for Dexterous Manipulation from Human Videos
While significant progress has been made on understanding hand-object interactions in computer vision, it is still very challenging for robots to perform complex dexterous manipulation.
Insights from Generative Modeling for Neural Video Compression
While recent machine learning research has revealed connections between deep generative models such as VAEs and rate-distortion losses used in learned compression, most of this work has focused on images.
Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers
Our key insight is that proprioceptive states only offer contact measurements for immediate reaction, whereas an agent equipped with visual sensory observations can learn to proactively maneuver environments with obstacles and uneven terrain by anticipating changes in the environment many steps ahead.
SCALE SPACE FLOW WITH AUTOREGRESSIVE PRIORS
There has been a recent surge of interest in neural video compression models that combines data-driven dimensionality reduction with learned entropy coding.
Generative Video Compression as Hierarchical Variational Inference
Recent work by Marino et al. (2020) showed improved performance in sequential density estimation by combining masked autoregressive flows with hierarchical latent variable models.
Hierarchical Autoregressive Modeling for Neural Video Compression
Recent work by Marino et al. (2020) showed improved performance in sequential density estimation by combining masked autoregressive flows with hierarchical latent variable models.
PIANOTREE VAE: Structured Representation Learning for Polyphonic Music
The dominant approach for music representation learning involves the deep unsupervised model family variational autoencoder (VAE).
Suphx: Mastering Mahjong with Deep Reinforcement Learning
Artificial Intelligence (AI) has achieved great success in many domains, and game AI is widely regarded as its beachhead since the dawn of AI.
Multi-Task Reinforcement Learning with Soft Modularization
While training multiple tasks jointly allow the policies to share parameters across different tasks, the optimization problem becomes non-trivial: It remains unclear what parameters in the network should be reused across tasks, and how the gradients from different tasks may interfere with each other.
Ranked #1 on
Meta-Learning
on MT50
Deep Music Analogy Via Latent Representation Disentanglement
Analogy-making is a key method for computer algorithms to generate both natural and creative music pieces.
Learning Efficient and Effective Exploration Policies with Counterfactual Meta Policy
Based on that, We proposed an end-to-end algorithm to learn exploration policy by meta-learning.
Inspecting and Interacting with Meaningful Music Representations using VAE
Variational Autoencoders(VAEs) have already achieved great results on image generation and recently made promising progress on music generation.
Artificial Intelligence for Prosthetics - challenge solutions
In the NeurIPS 2018 Artificial Intelligence for Prosthetics challenge, participants were tasked with building a controller for a musculoskeletal model with a goal of matching a given time-varying velocity vector.
MatchBench: An Evaluation of Feature Matchers
This leads to a critical absence in this field that there is no standard datasets and evaluation metrics to evaluate different feature matchers fairly.
