Search Results for author:
Found 71 papers, 31 papers with code
VMCML: Video and Music Matching via Cross-Modality Lifting
We propose a content-based system for matching video and background music.
CC-3DT: Panoramic 3D Object Tracking via Cross-Camera Fusion
To track the 3D locations and trajectories of the other traffic participants at any given time, modern autonomous vehicles are equipped with multiple cameras that cover the vehicle's full surroundings.
Semiconductor Defect Pattern Classification by Self-Proliferation-and-Attention Neural Network
In this paper, we present a novel architecture that can perform defect classification in a more efficient way.
MixFairFace: Towards Ultimate Fairness via MixFair Adapter in Face Recognition
In this paper, we propose MixFairFace framework to improve the fairness in face recognition models.
QRF: Implicit Neural Representations with Quantum Radiance Fields
Photorealistic rendering of real-world scenes is a tremendous challenge with a wide range of applications, including mixed reality (MR), and virtual reality (VR).
360-MLC: Multi-view Layout Consistency for Self-training and Hyper-parameter Tuning
In addition, since ground truth annotations are not available during training nor in testing, we leverage the entropy information in multiple layout estimations as a quantitative metric to measure the geometry consistency of the scene, allowing us to evaluate any layout estimator for hyper-parameter tuning, including model selection without ground truth annotations.
BiFuse++: Self-supervised and Efficient Bi-projection Fusion for 360 Depth Estimation
Thus, state-of-the-art frameworks for monocular 360 depth estimation such as bi-projection fusion in BiFuse are proposed.
Ranked #12 on
Depth Estimation
on Stanford2D3D Panoramic
Semiconductor Defect Detection by Hybrid Classical-Quantum Deep Learning
However, the massive expansion of semiconductor manufacturing and the development of new technology will bring many defect wafers.
Autoregressive 3D Shape Generation via Canonical Mapping
With the capacity of modeling long-range dependencies in sequential data, transformers have shown remarkable performances in a variety of generative tasks such as image, audio, and text generation.
Data Efficient 3D Learner via Knowledge Transferred from 2D Model
In this work, we deal with the data scarcity challenge of 3D tasks by transferring knowledge from strong 2D models via RGB-D images.
CLA-NeRF: Category-Level Articulated Neural Radiance Field
We propose CLA-NeRF -- a Category-Level Articulated Neural Radiance Field that can perform view synthesis, part segmentation, and articulated pose estimation.
Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module
The number of agents can grow or an environment sometimes needs to interact with a changing number of agents in real-world scenarios.
360-DFPE: Leveraging Monocular 360-Layouts for Direct Floor Plan Estimation
We present 360-DFPE, a sequential floor plan estimation method that directly takes 360-images as input without relying on active sensors or 3D information.
Leveraging Sequence Embedding and Convolutional Neural Network for Protein Function Prediction
In contrast, most of the existing methods delete the rare protein functions to reduce the label space.
Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction
Finally, evaluation on five inward-facing benchmarks shows that our method matches, if not surpasses, NeRF's quality, yet it only takes about 15 minutes to train from scratch for a new scene.
Dense Prediction with Attentive Feature Aggregation
Our experiments show consistent and significant improvements on challenging semantic segmentation benchmarks, including Cityscapes, BDD100K, and Mapillary Vistas, at negligible computational and parameter overhead.
Specialize and Fuse: Pyramidal Output Representation for Semantic Segmentation
We present a novel pyramidal output representation to ensure parsimony with our "specialize and fuse" process for semantic segmentation.
Learning 3D Dense Correspondence via Canonical Point Autoencoder
We propose a canonical point autoencoder (CPAE) that predicts dense correspondences between 3D shapes of the same category.
Indoor Panorama Planar 3D Reconstruction via Divide and Conquer
Indoor panorama typically consists of human-made structures parallel or perpendicular to gravity.
LED2-Net: Monocular 360deg Layout Estimation via Differentiable Depth Rendering
Although significant progress has been made in room layout estimation, most methods aim to reduce the loss in the 2D pixel coordinate rather than exploiting the room structure in the 3D space.
Robust 360-8PA: Redesigning The Normalized 8-point Algorithm for 360-FoV Images
This paper presents a novel preconditioning strategy for the classic 8-point algorithm (8-PA) for estimating an essential matrix from 360-FoV images (i. e., equirectangular images) in spherical projection.
LED2-Net: Monocular 360 Layout Estimation via Differentiable Depth Rendering
Although significant progress has been made in room layout estimation, most methods aim to reduce the loss in the 2D pixel coordinate rather than exploiting the room structure in the 3D space.
3D Room Layouts From A Single RGB Panorama
Depth Estimation
+2
Monocular Quasi-Dense 3D Object Tracking
Experiments on our proposed simulation data and real-world benchmarks, including KITTI, nuScenes, and Waymo datasets, show that our tracking framework offers robust object association and tracking on urban-driving scenarios.
Ranked #6 on
Multiple Object Tracking
on KITTI Tracking test
Toward Robust Long Range Policy Transfer
We also design two regularization terms to improve the diversity and utilization rate of the primitives in the pre-training phase.
Interactive Radiotherapy Target Delineation with 3D-Fused Context Propagation
The proposed method is evaluated on two published radiotherapy target contouring datasets of nasopharyngeal and esophageal cancer.
Mitigating Forgetting in Online Continual Learning via Instance-Aware Parameterization
To preserve the knowledge we learn from previous instances, we proposed a method to protect the path by restricting the gradient updates of one instance from overriding past updates calculated from previous instances if these instances are not similar.
HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features
We present HoHoNet, a versatile and efficient framework for holistic understanding of an indoor 360-degree panorama using a Latent Horizontal Feature (LHFeat).
3D Room Layouts From A Single RGB Panorama
Depth Estimation
+1
Lymph Node Gross Tumor Volume Detection in Oncology Imaging via Relationship Learning Using Graph Neural Network
Specifically, we first utilize a 3D convolutional neural network with ROI-pooling to extract the GTV$_{LN}$'s instance-wise appearance features.
Controllable Image Synthesis via SegVAE
We also apply an off-the-shelf image-to-image translation model to generate realistic RGB images to better understand the quality of the synthesized semantic maps.
LayoutMP3D: Layout Annotation of Matterport3D
Inferring the information of 3D layout from a single equirectangular panorama is crucial for numerous applications of virtual reality or robotics (e. g., scene understanding and navigation).
Visual Question Answering on 360° Images
We then study two different VQA models on VQA 360, including one conventional model that takes an equirectangular image (with intrinsic distortion) as input and one dedicated model that first projects a 360 image onto cubemaps and subsequently aggregates the information from multiple spatial resolutions.
Bias-Aware Heapified Policy for Active Learning
Last but not least, we investigate the generalization of the HAL policy learned on MNIST dataset by directly applying it on MNIST-M. We show that the agent can generalize and outperform directly-learned policy under constrained labeled sets.
360SD-Net: 360° Stereo Depth Estimation with Learnable Cost Volume
Recently, end-to-end trainable deep neural networks have significantly improved stereo depth estimation for perspective images.
360-Indoor: Towards Learning Real-World Objects in 360° Indoor Equirectangular Images
In this paper, our goal is to provide a standard dataset to facilitate the vision and machine learning communities in 360{\deg} domain.
Flat2Layout: Flat Representation for Estimating Layout of General Room Types
This paper also constructs a benchmark for validating the performance on general layout topologies, where Flat2Layout achieves good performance on general room types.
3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization
The complementary characteristics of active and passive depth sensing techniques motivate the fusion of the Li-DAR sensor and stereo camera for improved depth perception.
Radiotherapy Target Contouring with Convolutional Gated Graph Neural Network
Instead, inspired by the treating methodology of considering meaningful information across slices, we used Gated Graph Neural Network to frame this problem more efficiently.
Point-to-Point Video Generation
We introduce point-to-point video generation that controls the generation process with two control points: the targeted start- and end-frames.
Learning a Multi-Modal Policy via Imitating Demonstrations with Mixed Behaviors
The encoder infers discrete latent factors corresponding to different behaviors from demonstrations.
HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation
We present a new approach to the problem of estimating the 3D room layout from a single panoramic image.
3D Room Layouts From A Single RGB Panorama
Data Augmentation
Plug-and-Play: Improve Depth Estimation via Sparse Data Propagation
We propose a novel plug-and-play (PnP) module for improving depth prediction with taking arbitrary patterns of sparse depths as input.
DuLa-Net: A Dual-Projection Network for Estimating Room Layouts from a Single RGB Panorama
We present a deep learning framework, called DuLa-Net, to predict Manhattan-world 3D room layouts from a single RGB panorama.
Joint Monocular 3D Vehicle Detection and Tracking
The framework can not only associate detections of vehicles in motion over time, but also estimate their complete 3D bounding box information from a sequence of 2D images captured on a moving platform.
Ranked #11 on
Multiple Object Tracking
on KITTI Tracking test
InstaNAS: Instance-aware Neural Architecture Search
Conventional Neural Architecture Search (NAS) aims at finding a single architecture that achieves the best performance, which usually optimizes task related learning objectives such as accuracy.
Self-Supervised Learning of Depth and Camera Motion from 360° Videos
We propose a novel self-supervised learning approach for predicting the omnidirectional depth and camera motion from a 360{\deg} video.
Unsupervised Stylish Image Description Generation via Domain Layer Norm
It can learn to generate stylish image descriptions that are more related to image content and can be trained with the arbitrary monolingual corpus without collecting new paired image and stylish descriptions.
Searching Toward Pareto-Optimal Device-Aware Neural Architectures
Recent breakthroughs in Neural Architectural Search (NAS) have achieved state-of-the-art performance in many tasks such as image classification and language understanding.
Liquid Pouring Monitoring via Rich Sensory Inputs
In the closed-loop system, the ability to monitor the state of the task via rich sensory information is important but often less studied.
Leveraging Motion Priors in Videos for Improving Human Segmentation
Recently, a few domain adaptation and active learning approaches have been proposed to mitigate the performance drop.
Efficient Uncertainty Estimation for Semantic Segmentation in Videos
Uncertainty estimation in deep learning becomes more important recently.
Ranked #15 on
Semantic Segmentation
on CamVid
DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures
We propose DPP-Net: Device-aware Progressive Search for Pareto-optimal Neural Architectures, optimizing for both device-related (e. g., inference time and memory usage) and device-agnostic (e. g., accuracy and model size) objectives.
Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos
Then, we concatenate all six faces while utilizing the connectivity between faces on the cube for image padding (i. e., Cube Padding) in convolution, pooling, convolutional LSTM layers.
Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos
Then, we concatenate all six faces while utilizing the connectivity between faces on the cube for image padding (i. e., Cube Padding) in convolution, pooling, convolutional LSTM layers.
A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss
On the one hand, a simple extractive model can obtain sentence-level attention with high ROUGE scores but less readable.
Ranked #36 on
Abstractive Text Summarization
on CNN / Daily Mail
Omnidirectional CNN for Visual Place Recognition and Navigation
Given a visual input, the task of the O-CNN is not to retrieve the matched place exemplar, but to retrieve the closest place exemplar and estimate the relative distance between the input and the closest place.
Compatibility Family Learning for Item Recommendation and Generation
In our learned space, we introduce a novel Projected Compatibility Distance (PCD) function which is differentiable and ensures diversity by aiming for at least one prototype to be close to a compatible item, whereas none of the prototypes are close to an incompatible item.
Self-view Grounding Given a Narrated 360° Video
The negative log reconstruction loss of the reverse sentence (referred to as "irrelevant loss") is jointly minimized to encourage the reverse sentence to be different from the given sentence.
Anticipating Daily Intention using On-Wrist Motion Triggered Sensing
The core of the system is a novel Recurrent Neural Network (RNN) and Policy Network (PN), where the RNN encodes visual and motion observation to anticipate intention, and the PN parsimoniously triggers the process of visual observation to reduce computation requirement.
Detecting Adversarial Attacks on Neural Network Policies with Visual Foresight
Our core idea is that the adversarial examples targeting at a neural network-based policy are not effective for the frame prediction model.
Visual Forecasting by Imitating Dynamics in Natural Sequences
This allows us to apply IRL at scale and directly imitate the dynamics in high-dimensional continuous visual sequences from the raw pixel values.
Deep 360 Pilot: Learning a Deep Agent for Piloting Through 360deg Sports Videos
Given the main object and previously selected viewing angles, our method regresses a shift in viewing angle to move to the next one.
Agent-Centric Risk Assessment: Accident Anticipation and Risky Region Localization
For survival, a living agent must have the ability to assess risk (1) by temporally anticipating accidents before they occur, and (2) by spatially localizing risky regions in the environment to move away from threats.
Deep 360 Pilot: Learning a Deep Agent for Piloting through 360° Sports Video
Watching a 360{\deg} sports video requires a viewer to continuously select a viewing angle, either through a sequence of mouse clicks or head movements.
Show, Adapt and Tell: Adversarial Training of Cross-domain Image Captioner
The domain critic assesses whether the generated sentences are indistinguishable from sentences in the target domain.
No More Discrimination: Cross City Adaptation of Road Scene Segmenters
Despite the recent success of deep-learning based semantic segmentation, deploying a pre-trained road scene segmenter to a city whose images are not presented in the training set would not achieve satisfactory performance due to dataset biases.
Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
In the strategically-timed attack, the adversary aims at minimizing the agent's reward by only attacking the agent at a small subset of time steps in an episode.
Learning to Compose with Professional Photographs on the Web
Photo composition is an important factor affecting the aesthetics in photography.
Leveraging Video Descriptions to Learn Video Question Answering
Then, a large number of candidate QA pairs are automatically generated from descriptions rather than manually annotated.
Title Generation for User Generated Videos
Finally, our sentence augmentation method also outperforms the baselines on the M-VAD dataset.
Recognition from Hand Cameras
We have collected a new synchronized HandCam and HeadCam dataset with 20 videos captured in three scenes for hand states recognition.
