Search Results for author:
Found 139 papers, 86 papers with code
CI-AVSR: A Cantonese Audio-Visual Speech Datasetfor In-car Command Recognition
With the rise of deep learning and intelligent vehicles, the smart assistant has become an essential in-car component to facilitate driving and provide extra functionalities.
Latent Guard: a Safety Framework for Text-to-image Generation
Hence, we propose Latent Guard, a framework designed to improve safety measures in text-to-image generation.
Automatic Controllable Colorization via Imagination
Unlike most previous end-to-end automatic colorization algorithms, our framework allows for iterative and localized modifications of the colorization results because we explicitly model the coloring samples.
Robust Depth Enhancement via Polarization Prompt Fusion Tuning
Existing depth sensors are imperfect and may provide inaccurate depth values in challenging scenarios, such as in the presence of transparent or reflective objects.
OV9D: Open-Vocabulary Category-Level 9D Object Pose and Size Estimation
Derived from OmniObject3D, OO3D-9D is the largest and most diverse dataset in the field of category-level object pose and size estimation.
Follow-Your-Click: Open-domain Regional Image Animation via Short Prompts
Despite recent advances in image-to-video generation, better controllability and local animation are less explored.
Cross-Cluster Shifting for Efficient and Effective 3D Object Detection in Autonomous Driving
We present a new 3D point-based detector model, named Shift-SSD, for precise 3D object detection in autonomous driving.
Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners
Thus, instead of training the giant models from scratch, we propose to bridge the existing strong models with a shared latent representation space.
Real-time 3D-aware Portrait Editing from a Single Image
This work presents 3DPE, a practical method that can efficiently edit a face image following given prompts, like reference images or text descriptions, in a 3D-aware manner.
Make a Cheap Scaling: A Self-Cascade Diffusion Model for Higher-Resolution Adaptation
Diffusion models have proven to be highly effective in image and video generation; however, they still face composition challenges when generating images of varying sizes due to single-scale training data.
Using Left and Right Brains Together: Towards Vision and Language Planning
Large Language Models (LLMs) and Large Multi-modality Models (LMMs) have demonstrated remarkable decision masking capabilities on a variety of tasks.
ENTED: Enhanced Neural Texture Extraction and Distribution for Reference-based Blind Face Restoration
The latent code extracted from the degraded input image often contains corrupted features, making it difficult to align the semantic information from the input with the high-quality textures from the reference.
DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation
We propose DiffSHEG, a Diffusion-based approach for Speech-driven Holistic 3D Expression and Gesture generation with arbitrary length.
MagicScroll: Nontypical Aspect-Ratio Image Generation for Visual Storytelling via Multi-Layered Semantic-Aware Denoising
Visual storytelling often uses nontypical aspect-ratio images like scroll paintings, comic strips, and panoramas to create an expressive and compelling narrative.
TIP: Text-Driven Image Processing with Semantic and Restoration Instructions
Text-driven diffusion models have become increasingly popular for various image editing tasks, including inpainting, stylization, and object replacement.
HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation
The noisy image, landmarks, and text condition are then fed into the frozen ControlNet twice for noise prediction.
Learning Naturally Aggregated Appearance for Efficient 3D Editing
Neural radiance fields, which represent a 3D scene as a color field and a density field, have demonstrated great progress in novel view synthesis yet are unfavorable for editing due to the implicitness.
MagicStick: Controllable Video Editing via Control Handle Transformations
Yet succinct, our method is the first method to show the ability of video property editing from the pre-trained text-to-image model.
LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models
Specifically, to tailor the pre-trained latent diffusion model to operate on the RAW domain, we train a set of lightweight taming modules to inject the RAW information into the diffusion denoising process via modulating the intermediate features of UNet.
Gaussian Shell Maps for Efficient 3D Human Generation
Instead of rasterizing the shells directly, we sample 3D Gaussians on the shells whose attributes are encoded in the texture features.
TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering
The diffusion model has been proven a powerful generative model in recent years, yet remains a challenge in generating visual text.
PPAD: Iterative Interactions of Prediction and Planning for End-to-end Autonomous Driving
Unlike existing end-to-end autonomous driving frameworks, PPAD models the interactions among ego, agents, and the dynamic environment in an autoregressive manner by interleaving the Prediction and Planning processes at every timestep, instead of a single sequential process of prediction followed by planning.
VideoCrafter1: Open Diffusion Models for High-Quality Video Generation
The I2V model is designed to produce videos that strictly adhere to the content of the provided reference image, preserving its content, structure, and style.
Ranked #3 on
Text-to-Video Generation
on EvalCrafter Text-to-Video (ECTV) Dataset
(using extra training data)
ControlLLM: Augment Language Models with Tools by Searching on Graphs
We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving complex real-world tasks.
ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models
Our work also suggests that a pre-trained diffusion model trained on low-resolution images can be directly used for high-resolution visual generation without further tuning, which may provide insights for future research on ultra-high-resolution image and video synthesis.
In-Domain GAN Inversion for Faithful Reconstruction and Editability
This work fills in this gap by proposing in-domain GAN inversion, which consists of a domain-guided encoder and a domain-regularized optimizer, to regularize the inverted code in the native latent space of the pre-trained GAN model.
AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections
For the new task, we base our method on the generative radiance manifold representation and equip it with learnable facial and head-shoulder deformations.
Online Overexposed Pixels Hallucination in Videos with Adaptive Reference Frame Selection
We present a learning-based system to reduce these artifacts without resorting to complex acquisition mechanisms like alternating exposures or costly processing that are typical of high dynamic range (HDR) imaging.
CoDeF: Content Deformation Fields for Temporally Consistent Video Processing
We present the content deformation field CoDeF as a new type of video representation, which consists of a canonical content field aggregating the static contents in the entire video and a temporal deformation field recording the transformations from the canonical image (i. e., rendered from the canonical content field) to each individual frame along the time axis. Given a target video, these two fields are jointly optimized to reconstruct it through a carefully tailored rendering pipeline. We advisedly introduce some regularizations into the optimization process, urging the canonical content field to inherit semantics (e. g., the object shape) from the video. With such a design, CoDeF naturally supports lifting image algorithms for video processing, in the sense that one can apply an image algorithm to the canonical image and effortlessly propagate the outcomes to the entire video with the aid of the temporal deformation field. We experimentally show that CoDeF is able to lift image-to-image translation to video-to-video translation and lift keypoint detection to keypoint tracking without any training. More importantly, thanks to our lifting strategy that deploys the algorithms on only one image, we achieve superior cross-frame consistency in processed videos compared to existing video-to-video translation approaches, and even manage to track non-rigid objects like water and smog. Project page can be found at https://qiuyu96. github. io/CoDeF/.
Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation
For the first module, we leverage an off-the-shelf video retrieval system and extract video depths as motion structure.
CMDFusion: Bidirectional Fusion Network with Cross-modality Knowledge Distillation for LIDAR Semantic Segmentation
In this way, RGB images are not required during inference anymore since the 2D knowledge branch provides 2D information according to the 3D LIDAR input.
SAD: Segment Any RGBD
The Segment Anything Model (SAM) has demonstrated its effectiveness in segmenting any part of 2D RGB images.
Open Vocabulary Semantic Segmentation
Panoptic Segmentation
+1
TextDiffuser: Diffusion Models as Text Painters
Diffusion models have gained increasing attention for their impressive generation abilities but currently struggle with rendering accurate and coherent text.
Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos
Generating text-editable and pose-controllable character videos have an imperious demand in creating various digital human.
Real-time 6K Image Rescaling with Rate-distortion Optimization
Then, an efficient frequency-aware decoder reconstructs a high-fidelity HR image from the LR one in real time.
Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition
In this paper, we begin with analyzing the feature representation behavior in the open-set action recognition (OSAR) problem based on the information bottleneck (IB) theory, and propose to enlarge the instance-specific (IS) and class-specific (CS) information contained in the feature for better performance.
Rotating without Seeing: Towards In-hand Dexterity through Touch
Relying on touch-only sensing, we can directly deploy the policy in a real robot hand and rotate novel objects that are not presented in training.
FateZero: Fusing Attentions for Zero-shot Text-based Video Editing
We also have a better zero-shot shape-aware editing ability based on the text-to-video model.
Blind Video Deflickering by Neural Filtering with a Flawed Atlas
Prior work usually requires specific guidance such as the flickering frequency, manual annotations, or extra consistent videos to remove the flicker.
Human MotionFormer: Transferring Human Motions with Vision Transformers
In this paper, we propose Human MotionFormer, a hierarchical ViT framework that leverages global and local perceptions to capture large and subtle motion matching, respectively.
Video Waterdrop Removal via Spatio-Temporal Fusion in Driving Scenes
The waterdrops on windshields during driving can cause severe visual obstructions, which may lead to car accidents.
The Devil is in the Wrongly-classified Samples: Towards Unified Open-set Recognition
Recently, Unified Open-set Recognition (UOSR) has been proposed to reject not only unknown samples but also known but wrongly classified samples, which tends to be more practical in real-world applications.
Learning 3D-aware Image Synthesis with Unknown Pose Distribution
Existing methods for 3D-aware image synthesis largely depend on the 3D pose distribution pre-estimated on the training set.
LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis
This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image.
Randomized Quantization: A Generic Augmentation for Data Agnostic Self-supervised Learning
Self-supervised representation learning follows a paradigm of withholding some part of the data and tasking the network to predict it from the remaining part.
MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation
In this work, we propose an ID-preserving talking head generation framework, which advances previous methods in two aspects.
Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusion
This paper presents a 3D generative model that uses diffusion models to automatically generate 3D digital avatars represented as neural radiance fields.
High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization
We present a high-fidelity 3D generative adversarial network (GAN) inversion framework that can synthesize photo-realistic novel views while preserving specific details of the input image.
Latent Video Diffusion Models for High-Fidelity Long Video Generation
Diffusion models have shown remarkable results recently but require significant computational resources.
Ranked #2 on
Video Generation
on Taichi
Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint
In this work, we propose to first obtain the precise latent code in foundation latent space $\mathcal{W}$.
DYNAFED: Tackling Client Data Heterogeneity with Global Dynamics
Specifically, we first reserve a short trajectory of global model snapshots on the server.
Robust Federated Learning against both Data Heterogeneity and Poisoning Attack via Aggregation Optimization
Since at each round, the number of tunable parameters optimized on the server side equals the number of participating clients (thus independent of the model size), we are able to train a global model with massive parameters using only a small amount of proxy data (e. g., around one hundred samples).
Robust Reflection Removal with Flash-only Cues in the Wild
We propose a simple yet effective reflection-free cue for robust reflection removal from a pair of flash and ambient (no-flash) images.
Planning for Sample Efficient Imitation Learning
Inspired by the recent success of EfficientZero in RL, we propose EfficientImitate (EI), a planning-based imitation learning method that can achieve high in-environment sample efficiency and performance simultaneously.
One Model to Edit Them All: Free-Form Text-Driven Image Manipulation with Semantic Modulations
Based on the visual latent space of StyleGAN[21] and text embedding space of CLIP[34], studies focus on how to map these two latent spaces for text-driven attribute manipulations.
AniFaceGAN: Animatable 3D-Aware Face Image Generation for Video Avatars
To achieve meaningful control over facial expressions via deformation, we propose a 3D-level imitative learning scheme between the generator and a parametric 3D face model during adversarial training of the 3D-aware GAN.
Federated Domain Generalization for Image Recognition via Cross-Client Style Transfer
Our style representation is exceptionally lightweight and can hardly be used for the reconstruction of the dataset.
Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator
We argue that, considering the two-player game in the formulation of GANs, only making the generator 3D-aware is not enough.
A Portable Multiscopic Camera for Novel View and Time Synthesis in Dynamic Scenes
We present a portable multiscopic camera system with a dedicated model for novel view and time synthesis in dynamic scenes.
Optimizing Image Compression via Joint Learning with Denoising
The key is to transform the original noisy images to noise-free bits by eliminating the undesired noise during compression, where the bits are later decompressed as clean images.
Real-time Streaming Video Denoising with Bidirectional Buffers
Recent multi-output inference works propagate the bidirectional temporal feature with a parallel or recurrent framework, which either suffers from performance drops on the temporal edges of clips or can not achieve online inference.
Ranked #1 on
Video Denoising
on CRVD
Optimizing Video Prediction via Video Frame Interpolation
Extensive experiments on the Cityscapes, KITTI, DAVIS, Middlebury, and Vimeo90K datasets show that our video prediction results are robust in general scenarios, and our approach outperforms other video prediction methods that require a large amount of training data or extra semantic information.
Pretraining is All You Need for Image-to-Image Translation
We propose to use pretraining to boost general image-to-image translation.
Ranked #1 on
Sketch-to-Image Translation
on COCO-Stuff
Point Cloud Compression with Sibling Context and Surface Priors
We present a novel octree-based multi-level framework for large-scale point cloud compression, which can organize sparse and unstructured point clouds in a memory-efficient way.
Real-Time Neural Character Rendering with Pose-Guided Multiplane Images
We propose pose-guided multiplane image (MPI) synthesis which can render an animatable character in real scenes with photorealistic quality.
Bootstrap Motion Forecasting With Self-Consistent Constraints
Also, to model the multi-modality in motion forecasting, we design a novel self-ensembling scheme to obtain accurate teacher targets to enforce the self-constraints with multi-modality supervision.
Ranked #9 on
Motion Forecasting
on Argoverse CVPR 2020
FS6D: Few-Shot 6D Pose Estimation of Novel Objects
6D object pose estimation networks are limited in their capability to scale to large numbers of object instances due to the close-set assumption and their reliance on high-fidelity object CAD models.
Interpreting Class Conditional GANs with Channel Awareness
To describe such a phenomenon, we propose channel awareness, which quantitatively characterizes how a single channel contributes to the final synthesis.
Towards Self-Supervised Category-Level Object Pose and Size Estimation
Instead, we propose a label-free method that learns to enforce the geometric consistency between category template mesh and observed object point cloud under a self-supervision manner.
Region-Based Semantic Factorization in GANs
Despite the rapid advancement of semantic discovery in the latent space of Generative Adversarial Networks (GANs), existing approaches either are limited to finding global attributes or rely on a number of segmentation masks to identify local attributes.
3D-Aware Indoor Scene Synthesis with Depth Priors
In this way, the discriminator can take the spatial arrangement into account and advise the generator to learn an appropriate depth condition.
Deep Video Prior for Video Consistency and Propagation
A progressive propagation strategy with pseudo labels is also proposed to enhance DVP's performance on video propagation.
CI-AVSR: A Cantonese Audio-Visual Speech Dataset for In-car Command Recognition
With the rise of deep learning and intelligent vehicle, the smart assistant has become an essential in-car component to facilitate driving and provide extra functionalities.
Automatic Speech Recognition Datasets in Cantonese: A Survey and New Dataset
We further conduct experiments with Fairseq S2T Transformer, a state-of-the-art ASR model, on the biggest existing dataset, Common Voice zh-HK, and our proposed MDCC, and the results show the effectiveness of our dataset.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
Shape from Polarization for Complex Scenes in the Wild
We present a new data-driven approach with physics-based priors to scene-level normal estimation from a single polarization image.
ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversation
ASCEND (A Spontaneous Chinese-English Dataset) is a high-quality Mandarin Chinese-English code-switching corpus built on spontaneous multi-turn conversational dialogue sources collected in Hong Kong.
DRINet++: Efficient Voxel-as-point Point Cloud Segmentation
The Sparse Feature Encoder extracts the local context information for each point, and the Sparse Geometry Feature Enhancement enhances the geometric properties of a sparse point cloud via multi-scale sparse projection and attentive multi-scale fusion.
Physics Assisted Deep Learning for Indoor Imaging using Phaseless Wi-Fi Measurements
Thus, our proposed method is the first inverse scattering-based deep learning framework which can image large scatterers with high permittivity and achieve accurate indoor RF imaging using phaseless Wi-Fi measurements.
High-Fidelity GAN Inversion for Image Attribute Editing
With a low bit-rate latent code, previous works have difficulties in preserving high-fidelity details in reconstructed and edited images.
IICNet: A Generic Framework for Reversible Image Conversion
Reversible image conversion (RIC) aims to build a reversible transformation between specific visual content (e. g., short videos) and an embedding image, where the original content can be restored from the embedding when necessary.
Safety-aware Motion Prediction with Unseen Vehicles for Autonomous Driving
The ability to predict unseen vehicles is critical for safety in autonomous driving.
Dual-Camera Super-Resolution with Aligned Attention Modules
We present a novel approach to reference-based super-resolution (RefSR) with the focus on dual-camera super-resolution (DCSR), which utilizes reference images for high-quality and high-fidelity results.
Embedding Novel Views in a Single JPEG Image
We propose a novel approach for embedding novel views in a single JPEG image while preserving the perceptual fidelity of the modified JPEG image and the restored novel views.
Towards Photorealistic Colorization by Imagination
We present a novel approach to automatic image colorization by imitating the imagination process of human experts.
DRINet: A Dual-Representation Iterative Learning Network for Point Cloud Segmentation
By utilizing these two modules iteratively, features can be propagated between two different representations.
Joint Depth and Normal Estimation from Real-world Time-of-flight Raw Data
We present a novel approach to joint depth and normal estimation for time-of-flight (ToF) sensors.
Enhanced Invertible Encoding for Learned Image Compression
Although deep learning based image compression methods have achieved promising progress these days, the performance of these methods still cannot match the latest compression standard Versatile Video Coding (VVC).
A Categorized Reflection Removal Dataset with Diverse Real-world Scenes
Due to the lack of a large-scale reflection removal dataset with diverse real-world scenes, many existing reflection removal methods are trained on synthetic data plus a small amount of real-world data, which makes it difficult to evaluate the strengths or weaknesses of different reflection removal methods thoroughly.
Stereo Waterdrop Removal with Row-wise Dilated Attention
Thus, we propose a learning-based model for waterdrop removal with stereo images.
Unsupervised Portrait Shadow Removal via Generative Priors
Qualitative and quantitative experiments on a real-world portrait shadow dataset demonstrate that our approach achieves comparable performance with supervised shadow removal methods.
MFuseNet: Robust Depth Estimation with Learned Multiscopic Fusion
We design a multiscopic vision system that utilizes a low-cost monocular RGB camera to acquire accurate depth estimation.
Video Super-Resolution with Long-Term Self-Exemplars
Existing video super-resolution methods often utilize a few neighboring frames to generate a higher-resolution image for each frame.
SinIR: Efficient General Image Manipulation with Single Image Reconstruction
We train our model on a single image with cascaded multi-scale learning, where each network at each scale is responsible for image reconstruction.
Low-Rank Subspaces in GANs
Concretely, given an arbitrary image and a region of interest (e. g., eyes of face images), we manage to relate the latent space to the image region with the Jacobian matrix and then use low-rank factorization to discover steerable latent subspaces.
Image Inpainting with External-internal Learning and Monochromic Bottleneck
Although recent inpainting approaches have demonstrated significant improvements with deep neural networks, they still suffer from artifacts such as blunt structures and abrupt colors when filling in the missing regions.
Neural Camera Simulators
To facilitate the learning of a simulator model, we collect a dataset of the 10, 000 raw images of 450 scenes with different exposure settings.
Stereo Matching by Self-supervision of Multiscopic Vision
Self-supervised learning for depth estimation possesses several advantages over supervised learning.
Invertible Image Signal Processing
Unprocessed RAW data is a highly valuable image format for image editing and computer vision.
Involution: Inverting the Inherence of Convolution for Visual Recognition
Convolution has been the core ingredient of modern neural networks, triggering the surge of deep learning in vision.
Ranked #703 on
Image Classification
on ImageNet
Robust Reflection Removal with Reflection-free Flash-only Cues
The flash-only image is equivalent to an image taken in a dark environment with only a flash on.
Learning to Predict Vehicle Trajectories with Model-based Planning
Predicting the future trajectories of on-road vehicles is critical for autonomous driving.
TPCN: Temporal Point Cloud Networks for Motion Forecasting
We propose the Temporal Point Cloud Networks (TPCN), a novel and flexible framework with joint spatial and temporal learning for trajectory prediction.
Ranked #52 on
Motion Forecasting
on Argoverse CVPR 2020
FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation
Moreover, at the output representation stage, we designed a simple but effective 3D keypoints selection algorithm considering the texture and geometry information of objects, which simplifies keypoint localization for precise pose estimation.
Ranked #1 on
6D Pose Estimation
on LineMOD
Robust Federated Learning with Attack-Adaptive Aggregation
To the best of our knowledge, our aggregation strategy is the first one that can be adapted to defend against various attacks in a data-driven fashion.
Normalized Human Pose Features for Human Action Video Alignment
We present a novel approach for extracting human pose features from human action videos.
Video Deblurring by Fitting to Test Data
Our key observation is that some frames in a video with motion blur are much sharper than others, and thus we can transfer the texture information in those sharp frames to blurry frames.
Evaluating adversarial robustness in simulated cerebellum
It is well known that artificial neural networks are vulnerable to adversarial examples, in which great efforts have been made to improve the robustness.
Blind Video Temporal Consistency via Deep Video Prior
Extensive quantitative and perceptual experiments show that our approach obtains superior performance than state-of-the-art methods on blind video temporal consistency.
Self-supervised Object Tracking with Cycle-consistent Siamese Networks
Self-supervised learning for visual object tracking possesses valuable advantages compared to supervised learning, such as the non-necessity of laborious human annotations and online training.
Fully Convolutional Networks for Continuous Sign Language Recognition
Continuous sign language recognition (SLR) is a challenging task that requires learning on both spatial and temporal dimensions of signing frame sequences.
PSConv: Squeezing Feature Pyramid into One Compact Poly-Scale Convolutional Layer
Despite their strong modeling capacities, Convolutional Neural Networks (CNNs) are often scale-sensitive.
Learning to Learn Parameterized Classification Networks for Scalable Input Images
To achieve efficient and flexible image classification at runtime, we employ meta learners to generate convolutional weights of main networks for various input scales and maintain privatized Batch Normalization layers per scale.
Deep Reinforced Attention Learning for Quality-Aware Visual Recognition
In this paper, we build upon the weakly-supervised generation mechanism of intermediate attention maps in any convolutional neural networks and disclose the effectiveness of attention modules more straightforwardly to fully exploit their potential.
Depth Sensing Beyond LiDAR Range
Depth sensing is a critical component of autonomous driving technologies, but today's LiDAR- or stereo camera-based solutions have limited range.
Future Video Synthesis with Object Motion Prediction
We present an approach to predict future video frames given a sequence of continuous video frames in the past.
Ranked #2 on
Video Prediction
on Cityscapes
(using extra training data)
Polarized Reflection Removal with Perfect Alignment in the Wild
We present a novel formulation to removing reflection from polarized images in the wild.
PiP: Planning-informed Trajectory Prediction for Autonomous Driving
Moreover, our approach enables a novel pipeline which couples the prediction and planning, by conditioning PiP on multiple candidate trajectories of the ego vehicle, which is highly beneficial for autonomous driving in interactive scenarios.
Dynamic Hierarchical Mimicking Towards Consistent Optimization Objectives
While the depth of modern Convolutional Neural Networks (CNNs) surpasses that of the pioneering networks with a significant margin, the traditional way of appending supervision only over the final classifier and progressively propagating gradient flow upstream remains the training mainstay.
Active Perception with A Monocular Camera for Multiscopic Vision
We design a multiscopic vision system that utilizes a low-cost monocular RGB camera to acquire accurate depth estimation for robotic applications.
Video Depth Estimation by Fusing Flow-to-Depth Proposals
Our flow-to-depth layer is differentiable, and thus we can refine camera poses by maximizing the aggregated confidence in the camera pose refinement module.
Attack-Resistant Federated Learning with Residual-based Reweighting
Federated learning has a variety of applications in multiple domains by utilizing private training data stored on different devices.
Music-oriented Dance Video Synthesis with Pose Perceptual Loss
We present a learning-based approach with pose perceptual loss for automatic music video generation.
Fully Automatic Video Colorization with Self-Regularization and Diversity
We present a fully automatic approach to video colorization with self-regularization and diversity.
Zoom To Learn, Learn To Zoom
We show how to obtain the ground-truth data with optically zoomed images and contribute a dataset, SR-RAW, for real-world computational zoom.
Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Search
We present a learning-based approach to computing solutions for certain NP-hard problems.
Speech Denoising with Deep Feature Losses
We present an end-to-end deep learning approach to denoising speech signals by processing the raw waveform directly.
Speech Denoising Convolutional Neural Network trained with Deep Feature Losses.
We present an end-to-end deep learning approach to denoising speech signals by processing the raw waveform directly.
Single Image Reflection Separation with Perceptual Losses
Our loss function includes two perceptual losses: a feature loss from a visual perception network, and an adversarial loss that encodes characteristics of images in the transmission layers.
Interactive Image Segmentation With Latent Diversity
The first is trained to synthesize a diverse set of plausible segmentations that conform to the user's input.
Ranked #10 on
Interactive Segmentation
on SBD
Learning to See in the Dark
Imaging in low light is challenging due to low photon count and low SNR.
Ranked #4 on
Image Denoising
on SID x300
Semi-parametric Image Synthesis
We present a semi-parametric approach to photographic image synthesis from semantic layouts.
Fast Image Processing with Fully-Convolutional Networks
Our approach uses a fully-convolutional network that is trained on input-output pairs that demonstrate the operator's action.
Photographic Image Synthesis with Cascaded Refinement Networks
We present an approach to synthesizing photographic images conditioned on semantic layouts.
Dense Monocular Depth Estimation in Complex Dynamic Scenes
We present an approach to dense depth estimation from a single monocular camera that is moving through a dynamic scene.
Full Flow: Optical Flow Estimation By Global Optimization over Regular Grids
The approach optimizes a classical optical flow objective over the full space of mappings between discrete grids.
Robust Nonrigid Registration by Convex Optimization
We present an approach to nonrigid registration of 3D surfaces.
1-HKUST: Object Detection in ILSVRC 2014
We participated in the object detection track of ILSVRC 2014 and received the fourth place among the 38 teams.
Fast MRF Optimization with Application to Depth Reconstruction
We describe a simple and fast algorithm for optimizing Markov random fields over images.
