Search Results for author:
Found 70 papers, 27 papers with code
AsyInst: Asymmetric Affinity with DepthGrad and Color for Box-Supervised Instance Segmentation
Through systematic analysis, we found that the commonly used pairwise affinity loss has two limitations: (1) it works with color affinity but leads to inferior performance with other modalities such as depth gradient, (2)the original affinity loss does not prevent trivial predictions as intended but actually accelerates this process due to the affinity loss term being symmetric.
Box-supervised Instance Segmentation
Semantic Segmentation
+1
Learning Physically Realizable Skills for Online Packing of General 3D Shapes
We study the problem of learning online packing skills for irregular 3D shapes, which is arguably the most challenging setting of bin packing problems.
PVT++: A Simple End-to-End Latency-Aware Visual Tracking Framework
Visual object tracking is essential to intelligent robots.
P4P: Conflict-Aware Motion Prediction for Planning in Autonomous Driving
Motion prediction is crucial in enabling safe motion planning for autonomous vehicles in interactive scenarios.
VectorFlow: Combining Images and Vectors for Traffic Occupancy and Flow Prediction
Recently, the occupancy flow fields representation was proposed to represent joint future states of road agents through a combination of occupancy grid and flow, which supports efficient and consistent joint predictions.
ViP3D: End-to-end Visual Trajectory Prediction via 3D Agent Queries
In the existing autonomous driving systems, perception and prediction are two separate modules.
Beyond Visual Field of View: Perceiving 3D Environment with Echoes and Vision
This paper focuses on perceiving and navigating 3D environments using echoes and RGB image.
Augmented Imagefication: A Data-driven Fault Detection Method for Aircraft Air Data Sensors
In this paper, a novel data-driven approach named Augmented Imagefication for Fault detection (FD) of aircraft air data sensors (ADS) is proposed.
VectorMapNet: End-to-end Vectorized HD Map Learning
To the best of our knowledge, VectorMapNet is the first work designed towards end-to-end vectorized map learning from onboard observations.
Ranked #1 on
3D Lane Detection
on OpenLane-V2
The Modality Focusing Hypothesis: Towards Understanding Crossmodal Knowledge Distillation
Crossmodal knowledge distillation (KD) extends traditional knowledge distillation to the area of multimodal learning and demonstrates great success in various applications.
R4D: Utilizing Reference Objects for Long-Range Distance Estimation
An edge in the graph encodes the relative distance information between a pair of target and reference objects.
Depth Estimation Matters Most: Improving Per-Object Depth Estimation for Monocular 3D Detection and Tracking
Monocular image-based 3D perception has become an active research area in recent years owing to its applications in autonomous driving.
Learning Visual Styles from Audio-Visual Associations
Our model learns to manipulate the texture of a scene to match a sound, a problem we term audio-driven image stylization.
Sound2Synth: Interpreting Sound via FM Synthesizer Parameters Estimation
Synthesizer is a type of electronic musical instrument that is now widely used in modern music production and sound design.
MUTR3D: A Multi-camera Tracking Framework via 3D-to-2D Queries
In contrast to prior works, MUTR3D does not explicitly rely on the spatial and appearance similarity of objects.
Training-Free Robust Multimodal Learning via Sample-Wise Jacobian Regularization
Multimodal fusion emerges as an appealing technique to improve model performances on many tasks.
Egocentric Prediction of Action Target in 3D
We are interested in anticipating as early as possible the target location of a person's object manipulation action in a 3D workspace from egocentric vision.
Self-supervision through Random Segments with Autoregressive Coding (RandSAC)
We illustrate that randomized serialization of the segments significantly improves the performance and results in distribution over spatially-long (across-segments) and -short (within-segment) predictions which are effective for feature learning.
FUTR3D: A Unified Sensor Fusion Framework for 3D Detection
Sensor fusion is an essential topic in many perception systems, such as autonomous driving and robotics.
CYBORGS: Contrastively Bootstrapping Object Representations by Grounding in Segmentation
Many recent approaches in contrastive learning have worked to close the gap between pretraining on iconic images like ImageNet and pretraining on complex scenes like COCO.
M2I: From Factored Marginal Trajectory Prediction to Interactive Prediction
Predicting future motions of road participants is an important task for driving autonomously in urban scenes.
S3T: Self-Supervised Pre-training with Swin Transformer for Music Classification
To our knowledge, S3T is the first method combining the Swin Transformer with a self-supervised learning method for music classification.
AutoAlign: Pixel-Instance Feature Aggregation for Multi-Modal 3D Object Detection
This map enables our model to automate the alignment of non-homogenous features in a dynamic and data-driven manner.
Embracing Single Stride 3D Object Detector with Sparse Transformer
In LiDAR-based 3D object detection for autonomous driving, the ratio of the object size to input scene size is significantly smaller compared to 2D detection cases.
Ranked #3 on
3D Object Detection
on waymo pedestrian
IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenes
While most works focus on single-object or agent-object visual functionality and affordances, our work proposes to study a new kind of visual relationship that is also important to perceive and model -- inter-object functional relationships (e. g., a switch on the wall turns on or off the light, a remote control operates the TV).
SimIPU: Simple 2D Image and 3D Point Cloud Unsupervised Pre-Training for Spatial-Aware Visual Representations
To bridge this gap, we aim to learn a spatial-aware visual representation that can describe the three-dimensional space and is more suitable and effective for these tasks.
Neural Dubber: Dubbing for Videos According to Scripts
Neural Dubber is a multi-modal text-to-speech (TTS) model that utilizes the lip movement in the video to control the prosody of the generated speech.
DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries
This top-down approach outperforms its bottom-up counterpart in which object bounding box prediction follows per-pixel depth estimation, since it does not suffer from the compounding error introduced by a depth prediction model.
Learning Efficient Online 3D Bin Packing on Packing Configuration Trees
PCT is a full-fledged description of the state and action space of bin packing which can support packing policy learning based on deep reinforcement learning (DRL).
Modality Laziness: Everybody's Business is Nobody's Business
We name this problem of multi-modal training, \emph{Modality Laziness}.
Learning Practically Feasible Policies for Online 3D Bin Packing
In this problem, the items are delivered to the agent without informing the full sequence information.
DenseTNT: End-to-end Trajectory Prediction from Dense Goal Sets
In this work, we propose an anchor-free and end-to-end trajectory prediction model, named DenseTNT, that directly outputs a set of trajectories from dense goal candidates.
HDMapNet: An Online HD Map Construction and Evaluation Framework
By introducing the method and metrics, we invite the community to study this novel map learning problem.
HDMapGen: A Hierarchical Graph Generative Model of High Definition Maps
To address this issue, we introduce a new challenging task to generate HD maps.
DenseTNT: Waymo Open Dataset Motion Prediction Challenge 1st Place Solution
In autonomous driving, goal-based multi-trajectory prediction methods are proved to be effective recently, where they first score goal candidates, then select a final set of goals, and finally complete trajectories based on the selected goals.
Intrinsically Motivated Self-supervised Learning in Reinforcement Learning
In vision-based reinforcement learning (RL) tasks, it is prevalent to assign auxiliary tasks with a surrogate self-supervised loss so as to obtain more semantic representations and improve sample efficiency.
Co-advise: Cross Inductive Bias Distillation
Transformers recently are adapted from the community of natural language processing as a promising substitute of convolution-based neural networks for visual learning tasks.
Improving Multi-Modal Learning with Uni-Modal Teachers
We name this problem Modality Failure, and hypothesize that the imbalance of modalities and the implicit bias of common objectives in fusion method prevent encoders of each modality from sufficient feature learning.
Ranked #33 on
Semantic Segmentation
on NYU Depth v2
What Makes Multi-modal Learning Better than Single (Provably)
The world provides us with data of multiple modalities.
On Feature Decorrelation in Self-Supervised Learning
In self-supervised representation learning, a common idea behind most of the state-of-the-art approaches is to enforce the robustness of the representations to predefined augmentations.
Large Scale Interactive Motion Forecasting for Autonomous Driving : The Waymo Open Motion Dataset
Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
Multimodal Knowledge Expansion
The popularity of multimodal sensors and the accessibility of the Internet have brought us a massive amount of unlabeled multimodal data.
Ranked #35 on
Semantic Segmentation
on NYU Depth v2
Predictive Visual Tracking: A New Benchmark and Baseline Approach
As a crucial robotic perception capability, visual tracking has been intensively studied recently.
AETree: Areal Spatial Data Generation
Areal spatial data represent not only geographical locations but also sizes and shapes of physical objects such as buildings in a city.
Large Scale Interactive Motion Forecasting for Autonomous Driving: The Waymo Open Motion Dataset
Furthermore, we introduce a new set of metrics that provides a comprehensive evaluation of both single agent and joint agent interaction motion forecasting models.
UnModNet: Learning to Unwrap a Modulo Image for High Dynamic Range Imaging
A conventional camera often suffers from over- or under-exposure when recording a real-world scene with a very high dynamic range (HDR).
Unsupervised Monocular Depth Learning in Dynamic Scenes
We present a method for jointly training the estimation of depth, ego-motion, and a dense 3D translation field of objects relative to the scene, with monocular photometric consistency being the sole source of supervision.
CLOUD: Contrastive Learning of Unsupervised Dynamics
Developing agents that can perform complex control tasks from high dimensional observations such as pixels is challenging due to difficulties in learning dynamics efficiently.
LID 2020: The Learning from Imperfect Data Challenge Results
The purpose of the Learning from Imperfect Data (LID) workshop is to inspire and facilitate the research in developing novel approaches that would harness the imperfect data and improve the data-efficiency during training.
SEMI: Self-supervised Exploration via Multisensory Incongruity
The variance of actions is further used to measure action incongruity.
Multivariate Time-series Anomaly Detection via Graph Attention Network
Anomaly detection on multivariate time-series is of great importance in both data mining research and industrial applications.
TNT: Target-driveN Trajectory Prediction
Our key insight is that for prediction within a moderate time horizon, the future modes can be effectively captured by a set of target states.
Online 3D Bin Packing with Constrained Deep Reinforcement Learning
We solve a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP).
VectorNet: Encoding HD Maps and Agent Dynamics from Vectorized Representation
Behavior prediction in dynamic, multi-agent systems is an important problem in the context of self-driving cars, due to the complex representations and interactions of road components, including moving agents (e. g. pedestrians and vehicles) and road context information (e. g. lanes, traffic lights).
Music Gesture for Visual Sound Separation
Recent deep learning approaches have achieved impressive performance on visual sound separation tasks.
AlignNet: A Unifying Approach to Audio-Visual Alignment
We present AlignNet, a model that synchronizes videos with reference audios under non-uniform and irregular misalignments.
Neural network with data augmentation in multi-objective prediction of multi-stage pump
Finally, a neural network model based on data augmentation (NNDA) is proposed for the reason that simulation cost is too high and data is scarce in mechanical simulation field especially in CFD problems.
Scalability in Perception for Autonomous Driving: Waymo Open Dataset
In an effort to help align the research community's contributions with real-world self-driving problems, we introduce a new large scale, high quality, diverse dataset.
Self-supervised Moving Vehicle Tracking with Stereo Sound
At test time, the stereo-sound student network can work independently to perform object localization us-ing just stereo audio and camera meta-data, without any visual input.
Active Scene Understanding via Online Semantic Reconstruction
In our method, the exploratory robot scanning is both driven by and targeting at the recognition and segmentation of semantic objects from the scene.
Self-Supervised Audio-Visual Co-Segmentation
Segmenting objects in images and separating sound sources in audio are challenging tasks, in part because traditional approaches require large amounts of labeled data.
RF-based 3D skeletons
It maintains this accuracy even in the presence of multiple people, and in new environments that it has not seen in the training set.
Through-Wall Human Pose Estimation Using Radio Signals
Yet, unlike vision-based pose estimation, the radio-based system can estimate 2D poses through walls despite never trained on such scenarios.
The Sound of Pixels
We introduce PixelPlayer, a system that, by leveraging large amounts of unlabeled videos, learns to locate image regions which produce sounds and separate the input sounds into a set of components that represents the sound from each pixel.
HACS: Human Action Clips and Segments Dataset for Recognition and Temporal Localization
This paper presents a new large-scale dataset for recognition and temporal localization of human actions collected from Web videos.
Ranked #5 on
Temporal Action Localization
on HACS
Scene Parsing Through ADE20K Dataset
A novel network design called Cascade Segmentation Module is proposed to parse a scene into stuff, objects, and object parts in a cascade and improve over the baselines.
Open Vocabulary Scene Parsing
Recognizing arbitrary objects in the wild has been a challenging problem due to the limitations of existing classification models and datasets.
Semantic Understanding of Scenes through the ADE20K Dataset
Scene parsing, or recognizing and segmenting objects and stuff in an image, is one of the key problems in computer vision.
Loss Functions for Neural Networks for Image Processing
Neural networks are becoming central in several areas of computer vision and image processing and different architectures have been proposed to solve specific problems.
