Search Results for author:
Found 46 papers, 23 papers with code
RefFusion: Reference Adapted Diffusion Models for 3D Scene Inpainting
In this work, we propose an approach for 3D scene inpainting -- the task of coherently replacing parts of the reconstructed scene with desired content.
PhotoBot: Reference-Guided Interactive Photography via Natural Language
We leverage a visual language model (VLM) and an object detector to characterize the reference images via textual descriptions and then use a large language model (LLM) to retrieve relevant reference images based on a user's language query through text-based reasoning.
Working Backwards: Learning to Place by Picking
By doing so, the policy can generalize to object placement scenarios outside of the training environment without privileged information (e. g., placing a plate picked up from a table).
Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensor
An STS sensor can be switched between visual and tactile modes by leveraging a semi-transparent surface and controllable lighting, allowing for both pre-contact visual sensing and during-contact tactile sensing with a single sensor.
Reconstructive Latent-Space Neural Radiance Fields for Efficient 3D Scene Representations
The resulting latent-space NeRF can produce novel views with higher quality than standard colour-space NeRFs, as the AE can correct certain visual artifacts, while rendering over three times faster.
Watch Your Steps: Local Image and Scene Editing by Text Instructions
A field is trained on relevance maps of training views, denoted as the relevance field, defining the 3D region within which modifications should be made.
aUToLights: A Robust Multi-Camera Traffic Light Detection and Tracking System
Following four successful years in the SAE AutoDrive Challenge Series I, the University of Toronto is participating in the Series II competition to develop a Level 4 autonomous passenger vehicle capable of handling various urban driving scenarios by 2025.
Living in a Material World: Learning Material Properties from Full-Waveform Flash Lidar Data for Semantic Segmentation
The shape of the return waveform is affected by many factors, including the distance that the light pulse travels and the angle of incidence with a surface.
Reference-guided Controllable Inpainting of Neural Radiance Fields
The popularity of Neural Radiance Fields (NeRFs) for view synthesis has led to a desire for NeRF editing tools.
The Sum of Its Parts: Visual Part Segmentation for Inertial Parameter Identification of Manipulated Objects
To operate safely and efficiently alongside human workers, collaborative robots (cobots) require the ability to quickly understand the dynamics of manipulated objects.
Contrastive Learning for Self-Supervised Pre-Training of Point Cloud Segmentation Networks With Image Data
Reducing the quantity of annotations required for supervised training is vital when labels are scarce and costly.
Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks
In this work, we show that the standard, naive approach to exploration can manifest as a suboptimal local maximum if a policy learned with AIL sufficiently matches the expert distribution without fully learning the desired task.
SPIn-NeRF: Multiview Segmentation and Perceptual Inpainting with Neural Radiance Fields
We refer to this task as 3D inpainting.
Self-Supervised Pre-training of 3D Point Cloud Networks with Image Data
Reducing the quantity of annotations required for supervised training is vital when labels are scarce and costly.
LaTeRF: Label and Text Driven Object Radiance Fields
In this paper we introduce LaTeRF, a method for extracting an object of interest from a scene given 2D images of the entire scene, known camera poses, a natural language description of the object, and a set of point-labels of object and non-object points in the input images.
Learning Sequential Latent Variable Models from Multimodal Time Series Data
In this work, we present a self-supervised generative modelling framework to jointly learn a probabilistic latent state representation of multimodal data and the respective dynamics.
Learning to Detect Slip with Barometric Tactile Sensors and a Temporal Convolutional Neural Network
The ability to perceive object slip via tactile feedback enables humans to accomplish complex manipulation tasks including maintaining a stable grasp.
Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning
We present Learning from Guided Play (LfGP), a framework in which we leverage expert demonstrations of, in addition to a main task, multiple auxiliary tasks.
On the Coupling of Depth and Egomotion Networks for Self-Supervised Structure from Motion
Structure from motion (SfM) has recently been formulated as a self-supervised learning problem, where neural network models of depth and egomotion are learned jointly through view synthesis.
A Question of Time: Revisiting the Use of Recursive Filtering for Temporal Calibration of Multisensor Systems
We examine the problem of time delay estimation, or temporal calibration, in the context of multisensor data fusion.
Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations
In this work, we demonstrate that a multiview policy can be found through imitation learning by collecting data from a variety of viewpoints.
Under Pressure: Learning to Detect Slip with Barometric Tactile Sensors
Despite the utility of tactile information, tactile sensors have yet to be widely deployed in industrial robotics settings.
Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching
Successful visual navigation depends upon capturing images that contain sufficient useful information.
Towards a Policy-as-a-Service Framework to Enable Compliant, Trustworthy AI and HRI Systems in the Wild
Building trustworthy autonomous systems is challenging for many reasons beyond simply trying to engineer agents that 'always do the right thing.'
Computers and Society Human-Computer Interaction Robotics Software Engineering
BRON -- Linking Attack Tactics, Techniques, and Patterns with Defensive Weaknesses, Vulnerabilities and Affected Platform Configurations
Many public sources of cyber threat and vulnerability information exist to serve the defense of cyber systems.
Cryptography and Security
Self-Supervised Scale Recovery for Monocular Depth and Egomotion Estimation
The self-supervised loss formulation for jointly training depth and egomotion neural networks with monocular images is well studied and has demonstrated state-of-the-art accuracy.
Heteroscedastic Uncertainty for Robust Generative Latent Dynamics
Learning or identifying dynamics from a sequence of high-dimensional observations is a difficult challenge in many domains, including reinforcement learning and control.
A Smooth Representation of Belief over SO(3) for Deep Rotation Learning with Uncertainty
Accurate rotation estimation is at the heart of robot perception tasks such as visual odometry and object pose estimation.
Certifiably Optimal Monocular Hand-Eye Calibration
Correct fusion of data from two sensors is not possible without an accurate estimate of their relative pose, which can be determined through the process of extrinsic calibration.
Robotics
Adversarial Genetic Programming for Cyber Security: A Rising Application Domain Where GP Matters
We delineate Adversarial Genetic Programming for Cyber Security, a research topic that, by means of genetic programming (GP), replicates and studies the behavior of cyber adversaries and the dynamics of their engagements.
Self-Supervised Deep Pose Corrections for Robust Visual Odometry
We present a self-supervised deep pose correction (DPC) network that applies pose corrections to a visual odometry estimator to improve its accuracy.
Robotics
Robust Data-Driven Zero-Velocity Detection for Foot-Mounted Inertial Navigation
We present two novel techniques for detecting zero-velocity events to improve foot-mounted inertial navigation.
Robotics Signal Processing
Inverse Kinematics for Serial Kinematic Chains via Sum of Squares Optimization
This is accomplished through a novel formulation of inverse kinematics as a nearest point problem, and with a fast sum of squares solver that exploits the sparsity of kinematic constraints for serial manipulators.
Robotics Optimization and Control
Sparse Bounded Degree Sum of Squares Optimization for Certifiably Globally Optimal Rotation Averaging
Estimating unknown rotations from noisy measurements is an important step in SfM and other 3D vision tasks.
Learning Matchable Image Transformations for Long-term Metric Visual Localization
Long-term metric self-localization is an essential capability of autonomous mobile robots, but remains challenging for vision-based systems due to appearance changes caused by lighting, weather, or seasonal variations.
Probabilistic Regression of Rotations using Quaternion Averaging and a Deep Multi-Headed Network
Accurate estimates of rotation are crucial to vision-based motion estimation in augmented reality and robotics.
The Phoenix Drone: An Open-Source Dual-Rotor Tail-Sitter Platform for Research and Education
In particular, the details in this paper and the available open-source materials should enable learners to gain an understanding of aerodynamics, flight control, state estimation, software design, and simulation, while experimenting with a unique aerial robot.
Robotics
Certifiably Globally Optimal Extrinsic Calibration from Per-Sensor Egomotion
We present a certifiably globally optimal algorithm for determining the extrinsic calibration between two sensors that are capable of producing independent egomotion estimates.
Robotics
LSTM-Based Zero-Velocity Detection for Robust Inertial Navigation
While existing threshold-based zero-velocity detectors are not robust to varying motion types, our learned model accurately detects stationary periods of the inertial measurement unit (IMU) despite changes in the motion of the user.
Robotics Human-Computer Interaction
DPC-Net: Deep Pose Correction for Visual Localization
We use this loss to train a Deep Pose Correction network (DPC-Net) that predicts corrections for a particular estimator, sensor and environment.
How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change
Direct visual localization has recently enjoyed a resurgence in popularity with the increasing availability of cheap mobile computing power.
PROBE-GK: Predictive Robust Estimation using Generalized Kernels
Many algorithms in computer vision and robotics make strong assumptions about uncertainty, and rely on the validity of these assumptions to produce accurate and consistent state estimates.
PROBE: Predictive Robust Estimation for Visual-Inertial Navigation
Navigation in unknown, chaotic environments continues to present a significant challenge for the robotics community.
Improving Foot-Mounted Inertial Navigation Through Real-Time Motion Classification
We present a method to improve the accuracy of a foot-mounted, zero-velocity-aided inertial navigation system (INS) by varying estimator parameters based on a real-time classification of motion type.
Robotics Human-Computer Interaction
Reducing Drift in Visual Odometry by Inferring Sun Direction Using a Bayesian Convolutional Neural Network
We present a method to incorporate global orientation information from the sun into a visual odometry pipeline using only the existing image stream, where the sun is typically not visible.
Improving the Accuracy of Stereo Visual Odometry Using Visual Illumination Estimation
In the absence of reliable and accurate GPS, visual odometry (VO) has emerged as an effective means of estimating the egomotion of robotic vehicles.
