Search Results for author:
Found 113 papers, 47 papers with code
Photon-Efficient 3D Imaging with A Non-Local Neural Network
Photon-efficient imaging has enabled a number of applications relying on single-photon sensors that can capture a 3D image with as few as one photon per pixel.
Learning Trailer Moments in Full-Length Movies with Co-Contrastive Attention
We introduce a novel ranking network that utilizes the Co-Attention between movies and trailers as guidance to generate the training pairs, where the moments highly corrected with trailers are expected to be scored higher than the uncorrelated moments.
HiLo: Detailed and Robust 3D Clothed Human Reconstruction with High-and Low-Frequency Information of Parametric Models
We empirically find that the high-frequency (HF) and low-frequency (LF) information from a parametric model has the potential to enhance geometry details and improve robustness to noise, respectively.
KunquDB: An Attempt for Speaker Verification in the Chinese Opera Scenario
This work aims to promote Chinese opera research in both musical and speech domains, with a primary focus on overcoming the data limitations.
Object Segmentation-Assisted Inter Prediction for Versatile Video Coding
Moreover, segmentation mask is considered in the joint rate-distortion optimization for motion estimation and partition estimation to derive the motion vector of different regions and partition more accurately.
MEDPNet: Achieving High-Precision Adaptive Registration for Complex Die Castings
Due to their complex spatial structure and diverse geometric features, achieving high-precision and robust point cloud registration for complex Die Castings has been a significant challenge in the die-casting industry.
Wavelet-Like Transform-Based Technology in Response to the Call for Proposals on Neural Network-Based Image Coding
Neural network-based image coding has been developing rapidly since its birth.
Spatial Decomposition and Temporal Fusion based Inter Prediction for Learned Video Compression
With the SDD-based motion model and long short-term temporal contexts fusion, our proposed learned video codec can obtain more accurate inter prediction.
Language-Conditioned Robotic Manipulation with Fast and Slow Thinking
The language-conditioned robotic manipulation aims to transfer natural language instructions into executable actions, from simple pick-and-place to tasks requiring intent recognition and visual reasoning.
Object-Centric Instruction Augmentation for Robotic Manipulation
Humans interpret scenes by recognizing both the identities and positions of objects in their observations.
Enhancing CT Image synthesis from multi-modal MRI data based on a multi-task neural network framework
Image segmentation, real-value prediction, and cross-modal translation are critical challenges in medical imaging.
Towards Open-World Co-Salient Object Detection with Generative Uncertainty-aware Group Selective Exchange-Masking
To simultaneously consider the uncertainty introduced by irrelevant images and the consensus features of the remaining relevant images in the group, we designed a latent variable generator branch and CoSOD transformer branch.
On Uniform Scalar Quantization for Learned Image Compression
In principle, we find two factors crucial: one is the discrepancy between the surrogate and rounding, leading to train-test mismatch; the other is gradient estimation risk due to the surrogate, which consists of bias and variance of the gradient estimation.
Learning Fine-Grained Features for Pixel-wise Video Correspondences
We address the problem of learning features for establishing pixel-wise correspondences.
DTF-Net: Category-Level Pose Estimation and Shape Reconstruction via Deformable Template Field
We introduce a pose regression module that shares the deformation features and template codes from the fields to estimate the accurate 6D pose of each object in the scene.
On the Effectiveness of Spectral Discriminators for Perceptual Quality Improvement
We tackle this issue by examining the spectral discriminators in the context of perceptual image super-resolution (i. e., GAN-based SR), as SR image quality is susceptible to spectral changes.
Image Super-Resolution
No-Reference Image Quality Assessment
Offline and Online Optical Flow Enhancement for Deep Video Compression
In the offline stage, we fine-tune a trained optical flow estimation network with the motion information provided by a traditional (non-deep) video compression scheme, e. g. H. 266/VVC, as we believe the motion information of H. 266/VVC achieves a better rate-distortion trade-off.
VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision
Such compact representations need to be decoded back to pixels before being displayed to humans and - as usual - before being enhanced/analyzed by machine vision algorithms.
A Dataset for Deep Learning-based Bone Structure Analyses in Total Hip Arthroplasty
For THA, it is of clinical significance to analyze the bone structure from the CT images, especially to observe the structure of the acetabulum and femoral head, before the surgical procedure.
Towards Interactive Image Inpainting via Sketch Refinement
To address this problem, we propose a two-stage image inpainting method termed SketchRefiner.
Imbalance-Agnostic Source-Free Domain Adaptation via Avatar Prototype Alignment
To address this issue, we study a more practical SF-UDA task, termed imbalance-agnostic SF-UDA, where the class distributions of both the unseen source domain and unlabeled target domain are unknown and could be arbitrarily skewed.
Late-Constraint Diffusion Guidance for Controllable Image Synthesis
Specifically, we train a lightweight condition adapter to establish the correlation between external conditions and internal representations of diffusion models.
Ranked #1 on
Conditional Text-to-Image Synthesis
on COCO 2017 val
Conditional Image Generation
Conditional Text-to-Image Synthesis
Customized Segment Anything Model for Medical Image Segmentation
Different from the previous methods, SAMed is built upon the large-scale image segmentation model, Segment Anything Model (SAM), to explore the new research paradigm of customizing large-scale models for medical image segmentation.
Mask-Based Modeling for Neural Radiance Fields
Most Neural Radiance Fields (NeRFs) exhibit limited generalization capabilities, which restrict their applicability in representing multiple scenes using a single model.
PyramidFlow: High-Resolution Defect Contrastive Localization using Pyramid Normalizing Flow
During industrial processing, unforeseen defects may arise in products due to uncontrollable factors.
Ranked #5 on
Anomaly Detection
on BTAD
(using extra training data)
Exploiting Optical Flow Guidance for Transformer-Based Video Inpainting
Transformers have been widely used for video processing owing to the multi-head self attention (MHSA) mechanism.
Ranked #1 on
Video Inpainting
on DAVIS
(SSIM (square) metric)
Co-Salient Object Detection With Uncertainty-Aware Group Exchange-Masking
To address this issue, this paper presents a group exchange-masking (GEM) strategy for robust CoSOD model learning.
Unsupervised Video Object Segmentation with Online Adversarial Self-Tuning
We integrate our offline training and online fine-tuning in a unified framework for unsupervised video object segmentation and dub our method Online Adversarial Self-Tuning (OAST).
Spatial-then-Temporal Self-Supervised Learning for Video Correspondence
Specifically, we firstly extract spatial features from unlabeled images via contrastive learning, and secondly enhance the features by exploiting the temporal cues in unlabeled videos via reconstructive learning.
Flow-Guided Transformer for Video Inpainting
Especially in spatial transformer, we design a dual perspective spatial MHSA, which integrates the global tokens to the window-based attention.
Towards Hybrid-Optimization Video Coding
Our idea is to provide multiple discrete starting points in the global space and optimize the local optimum around each point by numerical algorithm efficiently.
Recurrent Dynamic Embedding for Video Object Segmentation
In this paper, we propose a Recurrent Dynamic Embedding (RDE) to build a memory bank of constant size.
Ranked #16 on
Semi-Supervised Video Object Segmentation
on MOSE
Neural Compression-Based Feature Learning for Video Restoration
The temporal features usually contain various noisy and uncorrelated information, and they may interfere with the restoration of the current frame.
aiWave: Volumetric Image Compression with 3-D Trained Affine Wavelet-like Transform
Volumetric image compression has become an urgent task to effectively transmit and store images produced in biological research and clinical practice.
Retinal Vessel Segmentation with Pixel-wise Adaptive Filters
Accurate retinal vessel segmentation is challenging because of the complex texture of retinal vessels and low imaging contrast.
Motion-Focused Contrastive Learning of Video Representations
To this end, we compose a duet of exploiting the motion for data augmentation and feature learning in the regime of contrastive learning.
Inertia-Guided Flow Completion and Style Fusion for Video Inpainting
We propose a flow completion network to align and aggregate flow features from the consecutive flow sequences based on the inertia prior.
Attribute Artifacts Removal for Geometry-based Point Cloud Compression
In this paper, we propose a Multi-Scale Graph Attention Network (MS-GAT) to remove the artifacts of point cloud attributes compressed by G-PCC.
Temporal Context Mining for Learned Video Compression
From the stored propagated features, we propose to learn multi-scale temporal contexts, and re-fill the learned temporal contexts into the modules of our compression scheme, including the contextual encoder-decoder, the frame generator, and the temporal context encoder.
Deep Reinforcement Learning Aided Packet-Routing For Aeronautical Ad-Hoc Networks Formed by Passenger Planes
Data packet routing in aeronautical ad-hoc networks (AANETs) is challenging due to their high-dynamic topology.
Deep Learning Aided Routing for Space-Air-Ground Integrated Networks Relying on Real Satellite, Flight, and Shipping Data
Current maritime communications mainly rely on satellites having meager transmission resources, hence suffering from poorer performance than modern terrestrial wireless networks.
Deep Learning Aided Packet Routing in Aeronautical Ad-Hoc Networks Relying on Real Flight Data: From Single-Objective to Near-Pareto Multi-Objective Optimization
Furthermore, we extend the DL-aided routing algorithm to a multi-objective scenario, where we aim for simultaneously minimizing the delay, maximizing the path capacity, and maximizing the path lifetime.
End-to-End Image Compression with Probabilistic Decoding
However, this nature was seldom considered in previous studies on image compression, which usually chose one possible image as reconstruction, e. g. the one with the maximal a posteriori probability.
Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining
Conventional methods often require (iterative) pruning followed by re-training, which not only incurs large overhead beyond the original DNN training but also can be sensitive to retraining hyperparameters.
iWave3D: End-to-end Brain Image Compression with Trainable 3-D Wavelet Transform
With the rapid development of whole brain imaging technology, a large number of brain images have been produced, which puts forward a great demand for efficient brain image compression methods.
Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images
Following the top-down paradigm, we decompose the task into two stages, i. e. person localization and pose estimation.
Ranked #2 on
3D Multi-Person Pose Estimation
on Panoptic
(using extra training data)
CERL: A Unified Optimization Framework for Light Enhancement with Realistic Noise
We present \underline{C}oordinated \underline{E}nhancement for \underline{R}eal-world \underline{L}ow-light Noisy Images (CERL), that seamlessly integrates light enhancement and noise suppression parts into a unified and physics-grounded optimization framework.
Learned Image Compression with Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modules
However, due to the vast diversity of images, it is not optimal to use one model for all images, even different regions within one image.
THE DCASE 2021 CHALLENGE TASK 6 SYSTEM: AUTOMATED AUDIO CAPTIONING WITH WEAKLY SUPERVISED PRE-TRAING AND WORD SELECTION METHODS
Our solution focuses on solving two problems in automated audio captioning: data insufficiency and word selection indeterminacy.
Ranked #1 on
Audio captioning
on Clotho
(using extra training data)
Normalizing Flow based Hidden Markov Models for Classification of Speech Phones with Explainability
For a standard speech phone classification setup involving 39 phones (classes) and the TIMIT dataset, we show that the use of standard features called mel-frequency-cepstral-coeffcients (MFCCs), the proposed generative models, and the decision fusion together can achieve $86. 6\%$ accuracy by generative training only.
PSD: Principled Synthetic-to-Real Dehazing Guided by Physical Priors
Deep learning-based methods have achieved remarkable performance for image dehazing.
Structured Multi-Level Interaction Network for Video Moment Localization via Language Query
In particular, for cross-modal interaction, we interact the sentence-level query with the whole moment while interact the word-level query with content and boundary, as in a coarse-to-fine manner.
Light Field Super-Resolution With Zero-Shot Learning
To fill this gap, we propose a zero-shot learning framework for light field SR, which learns a mapping to super-resolve the reference view with examples extracted solely from the input low-resolution light field itself.
Adaptive Domain-Specific Normalization for Generalizable Person Re-Identification
It describes unseen target domain as a combination of the known source ones, and explicitly learns domain-specific representation with target distribution to improve the model's generalization by a meta-learning pipeline.
Simultaneous Navigation and Construction Benchmarking Environments
We need intelligent robots for mobile construction, the process of navigating in an environment and modifying its structure according to a geometric design.
Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAE
We propose a two-stage model for diverse inpainting, where the first stage generates multiple coarse results each of which has a different structure, and the second stage refines each coarse result separately by augmenting texture.
Synergy Between Semantic Segmentation and Image Denoising via Alternate Boosting
We observe that not only denoising helps combat the drop of segmentation accuracy due to noise, but also pixel-wise semantic information boosts the capability of denoising.
Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows
We test the robustness of a maximum-likelihood (ML) based classifier where sequential data as observation is corrupted by noise.
Deep Transport Network for Unsupervised Video Object Segmentation
The popular unsupervised video object segmentation methods fuse the RGB frame and optical flow via a two-stream network.
Ranked #4 on
Unsupervised Video Object Segmentation
on FBMS test
An efficient Quasi-Newton method for nonlinear inverse problems via learned singular values
Here we present a highly efficient data-driven Quasi-Newton method applicable to nonlinear inverse problems.
Learning Trailer Moments in Full-Length Movies
A movie's key moments stand out of the screenplay to grab an audience's attention and make movie browsing efficient.
Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation
By introduces a improved transformer, elements in speech sequences can interact directly, which enables DPTNet can model for the speech sequences with direct context-awareness.
Ranked #15 on
Speech Separation
on WSJ0-2mix
Speech Separation
Audio and Speech Processing
Sound
Graph Neural Networks for Massive MIMO Detection
In this paper, we innovately use graph neural networks (GNNs) to learn a message-passing solution for the inference task of massive multiple multiple-input multiple-output (MIMO) detection in wireless communication.
Bottom-Up Human Pose Estimation by Ranking Heatmap-Guided Adaptive Keypoint Estimates
The typical bottom-up human pose estimation framework includes two stages, keypoint detection and grouping.
$α$ Belief Propagation for Approximate Inference
To gain a better understanding of BP in general graphs, we derive an interpretable belief propagation algorithm that is motivated by minimization of a localized $\alpha$-divergence.
Efficient Integer-Arithmetic-Only Convolutional Neural Networks
Considering the tradeoff between activation quantization error and network learning ability, we set an empirical rule to tune the bound of each Bounded ReLU.
Region-based Energy Neural Network for Approximate Inference
We term our model Region-based Energy Neural Network (RENN).
Foreground-Background Imbalance Problem in Deep Object Detectors: A Review
Recent years have witnessed the remarkable developments made by deep learning techniques for object detection, a fundamentally challenging problem of computer vision.
Transferring and Regularizing Prediction for Semantic Segmentation
In the view of extremely expensive expert labeling, recent research has shown that the models trained on photo-realistic synthetic data (e. g., computer games) with computer-generated annotations can be adapted to real images.
Ranked #17 on
Domain Adaptation
on SYNTHIA-to-Cityscapes
Accelerating Deep Reinforcement Learning With the Aid of Partial Model: Energy-Efficient Predictive Video Streaming
Predictive power allocation is conceived for energy-efficient video streaming over mobile networks using deep reinforcement learning.
Dual Temporal Memory Network for Efficient Video Object Segmentation
We present an end-to-end network which stores short- and long-term video sequence information preceding the current frame as the temporal memories to address the temporal modeling in VOS.
Is There Tradeoff between Spatial and Temporal in Video Super-Resolution?
Recent advances of deep learning lead to great success of image and video super-resolution (SR) methods that are based on convolutional neural networks (CNN).
On Dominant Interference in Random Networks and Communication Reliability
In this paper, we study the characteristics of dominant interference power with directional reception in a random network modelled by a Poisson Point Process.
Information Theory Signal Processing Information Theory
Optimizing Wireless Systems Using Unsupervised and Reinforced-Unsupervised Deep Learning
If the objective and constraint functions are unavailable, reinforcement learning can be applied to find the solution of a functional optimization problem, which is however not tailored to optimization problems in wireless networks.
Hidden Markov Models for sepsis detection in preterm infants
We explore the use of traditional and contemporary hidden Markov models (HMMs) for sequential physiological data analysis and sepsis prediction in preterm infants.
Optimizing electrode positions in 2D Electrical Impedance Tomography using deep learning
Further, it is found that the use of optimized electrode positions computed using the approach derived herein can reduce errors in EIT reconstructions as well as improve the distinguishability of EIT measurements.
Powering Hidden Markov Model by Neural Network based Generative Models
In the proposed GenHMM, each HMM hidden state is associated with a neural network based generative model that has tractability of exact likelihood and provides efficient likelihood computation.
Is Heuristic Sampling Necessary in Training Deep Object Detectors?
In this paper, we challenge the necessity of such hard/soft sampling methods for training accurate deep object detectors.
A Comprehensive Benchmark for Single Image Compression Artifacts Reduction
We present a comprehensive study and evaluation of existing single image compression artifacts removal algorithms, using a new 4K resolution benchmark including diversified foreground objects and background scenes with rich structures, called Large-scale Ideal Ultra high definition 4K (LIU4K) benchmark.
Customizable Architecture Search for Semantic Segmentation
As a result, our CAS is able to search an optimized architecture with customized constraints.
Residual Objectness for Imbalance Reduction
For a long time, object detectors have suffered from extreme imbalance between foregrounds and backgrounds.
$α$ Belief Propagation as Fully Factorized Approximation
Belief propagation (BP) can do exact inference in loop-free graphs, but its performance could be poor in graphs with loops, and the understanding of its solution is limited.
Deep High-Resolution Representation Learning for Visual Recognition
High-resolution representations are essential for position-sensitive vision problems, such as human pose estimation, semantic segmentation, and object detection.
Ranked #1 on
Object Detection
on COCO test-dev
(Hardware Burden metric)
Neural Network based Explicit Mixture Models and Expectation-maximization based Learning
A single latent variable is used as the common input to all the neural networks.
Model-Free Unsupervised Learning for Optimization Problems with Constraints
In many optimization problems in wireless communications, the expressions of objective function or constraints are hard or even impossible to derive, which makes the solutions difficult to find.
Composition-Aware Image Aesthetics Assessment
In this work, we propose to model the image composition information as the mutual dependency of its local regions, and design a novel architecture to leverage such information to boost the performance of aesthetics assessment.
SSFN -- Self Size-estimating Feed-forward Network with Low Complexity, Limited Need for Human Intervention, and Consistent Behaviour across Trials
We design a self size-estimating feed-forward network (SSFN) using a joint optimization approach for estimation of number of layers, number of nodes and learning of weight matrices.
Deep Learning-Based Video Coding: A Review and A Case Study
For deep schemes, pixel probability modeling and auto-encoder are the two approaches, that can be viewed as predictive coding scheme and transform coding scheme, respectively.
Multimedia Image and Video Processing
On The Classification-Distortion-Perception Tradeoff
In this paper, we extend the previous perception-distortion tradeoff to the case of classification-distortion-perception (CDP) tradeoff, where we introduced the classification error rate of the restored signal in addition to distortion and perceptual difference.
High-Resolution Representations for Labeling Pixels and Regions
The proposed approach achieves superior results to existing single-model networks on COCO object detection.
Ranked #7 on
Semantic Segmentation
on LIP val
Two-Stream Action Recognition-Oriented Video Super-Resolution
Tailored for two-stream action recognition networks, we propose two video SR methods for the spatial and temporal streams respectively.
Deep High-Resolution Representation Learning for Human Pose Estimation
We start from a high-resolution subnetwork as the first stage, gradually add high-to-low resolution subnetworks one by one to form more stages, and connect the mutli-resolution subnetworks in parallel.
Ranked #1 on
Pose Estimation
on BRACE
Entropy-regularized Optimal Transport Generative Models
We investigate the use of entropy-regularized optimal transport (EOT) cost in developing generative models to learn implicit distributions.
Endowing Robots with Longer-term Autonomy by Recovering from External Disturbances in Manipulation through Grounded Anomaly Classification and Recovery Policies
Robot manipulation is increasingly poised to interact with humans in co-shared workspaces.
DADA: Deep Adversarial Data Augmentation for Extremely Low Data Regime Classification
Given insufficient data, while many techniques have been developed to help combat overfitting, the challenge remains if one tries to train deep networks, especially in the ill-posed extremely low data regimes: only a small set of labeled data are available, and nothing -- including unlabeled data -- else.
IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks
In this paper, we are interested in building lightweight and efficient convolutional neural networks.
Fully Convolutional Adaptation Networks for Semantic Segmentation
The recent advances in deep neural networks have convincingly demonstrated high capability in learning vision models on large datasets.
Frank-Wolfe Network: An Interpretable Deep Structure for Non-Sparse Coding
We propose an interpretable deep structure namely Frank-Wolfe Network (F-W Net), whose architecture is inspired by unrolling and truncating the Frank-Wolfe algorithm for solving an $L_p$-norm constrained problem with $p\geq 1$.
A Learning-based Approach to Joint Content Caching and Recommendation at Base Stations
We then formulate a joint caching and recommendation problem maximizing the successful offloading probability, which is a mixed integer programming problem.
Human Pose Estimation using Global and Local Normalization
We present a two-stage normalization scheme, human body normalization and limb normalization, to make the distribution of the relative joint locations compact, resulting in easier learning of convolutional spatial models and more accurate pose estimation.
Neural network-based arithmetic coding of intra prediction modes in HEVC
In this paper, we propose an arithmetic coding strategy by training neural networks, and make preliminary studies on coding of the intra prediction modes in HEVC.
Multimedia
Snapshot Hyperspectral Light Field Imaging
This paper presents the first snapshot hyperspectral light field imager in practice.
A Convolutional Neural Network Approach for Half-Pel Interpolation in Video Coding
To further improve the coding efficiency, sub-pel motion compensation has been utilized, which requires interpolation of fractional samples.
Multimedia
Convolutional Neural Network-Based Block Up-sampling for Intra Frame Coding
A block can be down-sampled before being compressed by normal intra coding, and then up-sampled to its original resolution.
Multimedia
A Convolutional Neural Network Approach for Post-Processing in HEVC Intra Coding
Lossy image and video compression algorithms yield visually annoying artifacts including blocking, blurring, and ringing, especially at low bit-rates.
Multimedia
Comparative Deep Learning of Hybrid Representations for Image Recommendations
In many image-related tasks, learning expressive and discriminative representations of images is essential, and deep learning has been studied for automating the learning of such representations.
EventNet: A Large Scale Structured Concept Library for Complex Event Detection in Video
Extensive experiments over the zero-shot event retrieval task when no training samples are available show that the EventNet concept library consistently and significantly outperforms the state-of-the-art (such as the 20K ImageNet concepts trained with CNN) by a large margin up to 207%.
Building A Large Concept Bank for Representing Events in Video
In this paper, we propose to build Concept Bank, the largest concept library consisting of 4, 876 concepts specifically designed to cover 631 real-world events.
$\propto$SVM for learning with label proportions
We study the problem of learning with label proportions in which the training data is provided in groups and only the proportion of each class in each group is known.
Sample-Specific Late Fusion for Visual Category Recognition
However, the existing methods generally use a fixed fusion weight for all the scores of a classifier, and thus fail to optimally determine the fusion weight for the individual samples.
Robust Object Co-detection
Object co-detection aims at simultaneous detection of objects of the same category from a pool of related images by exploiting consistent visual patterns present in candidate objects in the images.
A Bayesian Approach to Multimodal Visual Dictionary Learning
nary learning methods rely on image descriptors alone or together with class labels.
