Search Results for author:
Found 56 papers, 19 papers with code
SAFE: Multitask Failure Detection for Vision-Language-Action Models
SAFE is trained on both successful and failed rollouts, and is evaluated on unseen tasks.
Pseudo-Simulation for Autonomous Driving
Our method then assigns a higher importance to synthetic observations that best match the AV's likely behavior using a novel proximity-based weighting scheme.
Calibrated Value-Aware Model Learning with Stochastic Environment Models
The idea of value-aware model learning, that models should produce accurate value estimates, has gained prominence in model-based reinforcement learning.
MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoning
To address these limitations, we propose MORE, a novel approach for enhancing the capabilities of language models to solve zero-shot mobile manipulation planning for rearrangement tasks.
Pippo: High-Resolution Multi-View Humans from a Single Image
Finally, we also introduce an improved metric to evaluate 3D consistency of multi-view generations, and show that Pippo outperforms existing works on multi-view human generation from a single image.
EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering
We introduce a method for using event camera data in novel view synthesis via Gaussian Splatting.
Mind the Time: Temporally-Controlled Multi-Event Video Generation
Generating such sequences with precise temporal control is infeasible with existing video generators that rely on a single paragraph of text as input.
Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos
Recent advancements in static feed-forward scene reconstruction have demonstrated significant progress in high-quality novel view synthesis.
CTRL-D: Controllable Dynamic 3D Scene Editing with Personalized 2D Diffusion
Our fine-tuning enables the model to "learn" the editing ability from a single edited reference image, transforming the complex task of dynamic scene editing into a simple 2D image editing process.
GaussianCut: Interactive segmentation via graph cut for 3D Gaussian Splatting
Using 3D Gaussian Splatting (3DGS) as the underlying scene representation simplifies the extraction of objects of interest which are considered to be a subset of the scene's Gaussians.
SG-I2V: Self-Guided Trajectory Control in Image-to-Video Generation
Methods for image-to-video generation have achieved impressive, photo-realistic quality.
MAD-TD: Model-Augmented Data stabilizes High Update Ratio RL
Our method, Model-Augmented Data for Temporal Difference learning (MAD-TD) uses small amounts of generated data to stabilize high UTD training and achieve competitive performance on the most challenging tasks in the DeepMind control suite.
Towards Unsupervised Blind Face Restoration using Diffusion Prior
The models trained on such synthetic degradations, however, cannot deal with inputs of unseen degradations.
Realistic Evaluation of Model Merging for Compositional Generalization
Merging has become a widespread way to cheaply combine individual models into a single model that inherits their capabilities and attains better performance.
Accelerating Online Mapping and Behavior Prediction via Direct BEV Feature Attention
Understanding road geometry is a critical component of the autonomous vehicle (AV) stack.
When does Self-Prediction help? Understanding Auxiliary Tasks in Reinforcement Learning
We provide a theoretical analysis of the learning dynamics of observation reconstruction, latent self-prediction, and TD learning in the presence of distractions and observation functions under linear model assumptions.
NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking
On a large set of challenging scenarios, we observe that simple methods with moderate compute requirements such as TransFuser can match recent large-scale end-to-end driving architectures such as UniAD.
Ranked #26 on
NavSim
on OpenScene
Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models
By fine-tuning a pre-trained text-to-image diffusion model with this information, our approach enables fine-grained 3D pose and placement control of individual objects in a scene.
RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpainting
In this work, we propose an approach for 3D scene inpainting -- the task of coherently replacing parts of the reconstructed scene with desired content.
Producing and Leveraging Online Map Uncertainty in Trajectory Prediction
High-definition (HD) maps have played an integral role in the development of modern autonomous vehicle (AV) stacks, albeit with high associated labeling and maintenance costs.
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers
Vid2Robot uses cross-attention transformer layers between video features and the current robot state to produce the actions and perform the same task as shown in the video.
Dissecting Deep RL with High Update Ratios: Combatting Value Divergence
We show that deep reinforcement learning algorithms can retain their ability to learn without resetting network parameters in settings where the number of gradient updates greatly exceeds the number of environment samples by combatting value function divergence.
SPAD : Spatially Aware Multiview Diffusers
We present SPAD, a novel approach for creating consistent multi-view images from text prompts or single images.
AToM: Amortized Text-to-Mesh using 2D Diffusion
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously.
SPAD: Spatially Aware Multi-View Diffusers
We present SPAD a novel approach for creating consistent multi-view images from text prompts or single images.
LEOD: Label-Efficient Object Detection for Event Cameras
On 1Mpx, RVT-S with 10% labels even surpasses its fully-supervised counterpart using 100% labels.
Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations
The resulting latent-space NeRF can produce novel views with higher quality than standard colour-space NeRFs, as the AE can correct certain visual artifacts, while rendering over three times faster.
iNVS: Repurposing Diffusion Inpainters for Novel View Synthesis
Our approach focuses on maximizing the reuse of visible pixels from the source image.
Watch Your Steps: Local Image and Scene Editing by Text Instructions
A field is trained on relevance maps of training views, denoted as the relevance field, defining the 3D region within which modifications should be made.
trajdata: A Unified Interface to Multiple Human Trajectory Datasets
The field of trajectory forecasting has grown significantly in recent years, partially owing to the release of numerous large-scale, real-world human trajectory datasets for autonomous vehicles (AVs) and pedestrian motion tracking.
$λ$-models: Effective Decision-Aware Reinforcement Learning with Latent Models
The idea of decision-aware model learning, that models should be accurate where it matters for decision-making, has gained prominence in model-based reinforcement learning.
EventCLIP: Adapting CLIP for Event-based Object Recognition
Recent advances in zero-shot and few-shot classification heavily rely on the success of pre-trained vision-language models (VLMs) such as CLIP.
SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models
Finally, we demonstrate the scalability of SlotDiffusion to unconstrained real-world datasets such as PASCAL VOC and COCO, when integrated with self-supervised pre-trained image encoders.
Reference-guided Controllable Inpainting of Neural Radiance Fields
The popularity of Neural Radiance Fields (NeRFs) for view synthesis has led to a desire for NeRF editing tools.
CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videos
Animating an object in 3D often requires an articulated structure, e. g. a kinematic chain or skeleton of the manipulated object with proper skinning weights, to obtain smooth movements and surface deformations.
Invertible Neural Skinning
Next, we combine PIN with a differentiable LBS module to build an expressive and end-to-end Invertible Neural Skinning (INS) pipeline.
Building Scalable Video Understanding Benchmarks through Sports
Our human studies indicate that ASAP can align videos and annotations with high fidelity, precision, and speed.
SparsePose: Sparse-View Camera Pose Regression and Refinement
Camera pose estimation is a key step in standard 3D reconstruction pipelines that operate on a dense set of images of a single object or scene.
SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields
We refer to this task as 3D inpainting.
Solving Continuous Control via Q-learning
While there has been substantial success for solving continuous control with actor-critic methods, simpler critic-only methods such as Q-learning find limited application in the associated high-dimensional action spaces.
See, Plan, Predict: Language-guided Cognitive Planning with Video Prediction
Cognitive planning is the structural decomposition of complex tasks into a sequence of future behaviors.
LaTeRF: Label and Text Driven Object Radiance Fields
In this paper we introduce LaTeRF, a method for extracting an object of interest from a scene given 2D images of the entire scene, known camera poses, a natural language description of the object, and a set of point-labels of object and non-object points in the input images.
Housekeep: Tidying Virtual Households using Commonsense Reasoning
Instead, the agent must learn from and is evaluated against human preferences of which objects belong where in a tidy house.
Neighborhood Mixup Experience Replay: Local Convex Interpolation for Improved Sample Efficiency in Continuous Control Tasks
Experience replay plays a crucial role in improving the sample efficiency of deep reinforcement learning agents.
Learning Interactive Driving Policies via Data-driven Simulation
Data-driven simulators promise high data-efficiency for driving policy learning.
VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios.
Is Bang-Bang Control All You Need? Solving Continuous Control with Bernoulli Policies
Reinforcement learning (RL) for continuous control typically employs distributions whose support covers the entire action space.
HYPER: Learned Hybrid Trajectory Prediction via Factored Inference and Adaptive Sampling
Modeling multi-modal high-level intent is important for ensuring diversity in trajectory prediction.
Deep Latent Competition: Learning to Race Using Visual Control Policies in Latent Space
We demonstrate the effectiveness of our algorithm in learning competitive behaviors on a novel multi-agent racing benchmark that requires planning from image observations.
Deep Orientation Uncertainty Learning based on a Bingham Loss
Reasoning about uncertain orientations is one of the core problems in many perception tasks such as object pose estimation or motion estimation.
Deep Context Maps: Agent Trajectory Prediction using Location-specific Latent Maps
Our main contribution is the concept of learning context maps to improve the prediction task.
SiPPing Neural Networks: Sensitivity-informed Provable Pruning of Neural Networks
We introduce a pruning algorithm that provably sparsifies the parameters of a trained model in a way that approximately preserves the model's predictive accuracy.
LandmarkBoost: Efficient Visual Context Classifiers for Robust Localization
We propose LandmarkBoost - an approach that, in contrast to the conventional 2D-3D matching methods, casts the search problem as a landmark classification task.
Data-Dependent Coresets for Compressing Neural Networks with Applications to Generalization Bounds
We present an efficient coresets-based neural network compression algorithm that sparsifies the parameters of a trained fully-connected neural network in a manner that provably approximates the network's output.
Directional Statistics and Filtering Using libDirectional
In this paper, we present libDirectional, a MATLAB library for directional statistics and directional estimation.
maplab: An Open Framework for Research in Visual-inertial Mapping and Localization
On the other hand, maplab provides the research community with a collection of multisession mapping tools that include map merging, visual-inertial batch optimization, and loop closure.
Robotics
