Search Results for author:
Found 101 papers, 35 papers with code
Unsupervised Multi-scale Expressive Speaking Style Modeling with Hierarchical Context Information for Audiobook Speech Synthesis
On top of these, a bi-reference attention mechanism is used to align both local-scale reference style embedding sequence and local-scale context style embedding sequence with corresponding phoneme embedding sequence.
RaFE: Generative Radiance Fields Restoration
Instead of reconstructing a blurred NeRF by averaging inconsistencies, we introduce a novel approach using Generative Adversarial Networks (GANs) for NeRF generation to better accommodate the geometric and appearance inconsistencies present in the multi-view images.
Bioinformatics and Biomedical Informatics with ChatGPT: Year One Review
The year 2023 marked a significant surge in the exploration of applying large language model (LLM) chatbots, notably ChatGPT, across various disciplines.
AI-Generated Content Enhanced Computer-Aided Diagnosis Model for Thyroid Nodules: A ChatGPT-Style Assistant
After training, ThyGPT could automatically evaluate thyroid nodule and engage in effective communication with physicians through human-computer interaction.
Data-Free Generalized Zero-Shot Learning
Firstly, to recover the virtual features of the base data, we model the CLIP features of base class images as samples from a von Mises-Fisher (vMF) distribution based on the pre-trained classifier.
Harnessing Neuron Stability to Improve DNN Verification
We evaluate the effectiveness of VeriStable across a range of challenging benchmarks including fully-connected feedforward networks (FNNs), convolutional neural networks (CNNs) and residual networks (ResNets) applied to the standard MNIST and CIFAR datasets.
SVGDreamer: Text Guided SVG Generation with Diffusion Model
However, existing text-to-SVG generation methods lack editability and struggle with visual quality and result diversity.
Multi-modality Affinity Inference for Weakly Supervised 3D Semantic Segmentation
The point affinity proposed in this paper is characterized by features from multiple modalities (e. g., point cloud and RGB), and is further refined by normalizing the classifier weights to alleviate the detrimental effects of long-tailed distribution without the need of the prior of category distribution.
A Video is Worth 256 Bases: Spatial-Temporal Expectation-Maximization Inversion for Zero-Shot Video Editing
This paper presents a video inversion approach for zero-shot video editing, which aims to model the input video with low-rank representation during the inversion process.
Progressive Target-Styled Feature Augmentation for Unsupervised Domain Adaptation on Point Clouds
Unlike previous works that focus on feature extractor adaptation, our PTSFA approach focuses on classifier adaptation.
UFDA: Universal Federated Domain Adaptation with Practical Assumptions
Conventional Federated Domain Adaptation (FDA) approaches usually demand an abundance of assumptions, which makes them significantly less feasible for real-world situations and introduces security hazards.
Adapting Segment Anything Model (SAM) through Prompt-based Learning for Enhanced Protein Identification in Cryo-EM Micrographs
Cryo-electron microscopy (cryo-EM) remains pivotal in structural biology, yet the task of protein particle picking, integral for 3D protein structure construction, is laden with manual inefficiencies.
E4S: Fine-grained Face Swapping via Editing With Regional GAN Inversion
Based on this disentanglement, face swapping can be simplified as style and mask swapping.
Diffusion Model is Secretly a Training-free Open Vocabulary Semantic Segmenter
The pre-trained text-image discriminative models, such as CLIP, has been explored for open-vocabulary semantic segmentation with unsatisfactory results due to the loss of crucial localization information and awareness of object shapes.
Inversion-by-Inversion: Exemplar-based Sketch-to-Photo Synthesis via Stochastic Differential Equations without Training
In the full-control inversion process, we propose an appearance-energy function to control the color and texture of the final generated photo. Importantly, our Inversion-by-Inversion pipeline is training-free and can accept different types of exemplars for color and texture control.
Reinforcement Learning-based Non-Autoregressive Solver for Traveling Salesman Problems
The Traveling Salesman Problem (TSP) is a well-known combinatorial optimization problem with broad real-world applications.
VideoControlNet: A Motion-Guided Video-to-Video Translation Framework by Using Diffusion Model with ControlNet
In this work, by using the diffusion model with ControlNet, we proposed a new motion-guided video-to-video translation framework called VideoControlNet to generate various videos based on the given prompts and the condition from the input video.
DiffSketcher: Text Guided Vector Sketch Synthesis through Latent Diffusion Models
Even though trained mainly on images, we discover that pretrained diffusion models show impressive power in guiding sketch synthesis.
Bid Optimization for Offsite Display Ad Campaigns on eCommerce
Online retailers often use third-party demand-side-platforms (DSPs) to conduct offsite advertising and reach shoppers across the Internet on behalf of their advertisers.
Association of stroke lesion distributions with atrial fibrillation detected after stroke
The knowledge of the unique characteristics of the population with atrial fibrillation detected after stroke (AFDAS) enables more ischemic stroke patients to benefit from more aggressive anticoagulation therapy and AF management.
Q2ATransformer: Improving Medical VQA via an Answer Querying Decoder
To bridge this gap, in this paper, we propose a new Transformer based framework for medical VQA (named as Q2ATransformer), which integrates the advantages of both the classification and the generation approaches and provides a unified treatment for the close-end and open-end questions.
VL-SAT: Visual-Linguistic Semantics Assisted Training for 3D Semantic Scene Graph Prediction in Point Cloud
Since 2D images provide rich semantics and scene graphs are in nature coped with languages, in this study, we propose Visual-Linguistic Semantics Assisted Training (VL-SAT) scheme that can significantly empower 3DSSG prediction models with discrimination about long-tailed and ambiguous semantic relations.
Ranked #1 on
3d scene graph generation
on 3DSSG
(using extra training data)
Large AI Models in Health Informatics: Applications, Challenges, and the Future
Large AI models, or foundation models, are models recently emerging with massive scales both parameter-wise and data-wise, the magnitudes of which can reach beyond billions.
Conflict-Based Cross-View Consistency for Semi-Supervised Semantic Segmentation
In this work, we propose a new conflict-based cross-view consistency (CCVC) method based on a two-branch co-training framework which aims at enforcing the two sub-nets to learn informative features from irrelevant views.
Bridging Synthetic and Real Images: a Transferable and Multiple Consistency aided Fundus Image Enhancement Framework
Moreover, we also propose a novel multi-stage multi-attention guided enhancement network (MAGE-Net) as the backbones of our teacher and student network.
Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Action
This review aims to provide a rather thorough overview of the applications of diffusion models in bioinformatics to aid their further development in bioinformatics and computational biology.
Complexity-Guided Slimmable Decoder for Efficient Deep Video Compression
In this work, we propose the complexity-guided slimmable decoder (cgSlimDecoder) in combination with skip-adaptive entropy coding (SaEC) for efficient deep video compression.
Content Adaptive Latents and Decoder for Neural Image Compression
Although several content adaptive methods have been proposed by updating the encoder-side components, the adaptability of both latents and the decoder is not well exploited.
Slow Motion Matters: A Slow Motion Enhanced Network for Weakly Supervised Temporal Action Localization
Weakly supervised temporal action localization (WTAL) aims to localize actions in untrimmed videos with only weak supervision information (e. g. video-level labels).
Weakly-supervised Temporal Action Localization
Weakly Supervised Temporal Action Localization
Towards Explainable 3D Grounded Visual Question Answering: A New Benchmark and Strong Baseline
We also propose a new 3D VQA framework to effectively predict the completely visually grounded and explainable answer.
Improving RGB-D Point Cloud Registration by Learning Multi-scale Local Linear Transformation
In addition to previous methods that seek correspondences by hand-crafted or learnt geometric features, recent point cloud registration methods have tried to apply RGB-D data to achieve more accurate correspondence.
SketchSampler: Sketch-based 3D Reconstruction via View-dependent Depth Sampling
Reconstructing a 3D shape based on a single sketch image is challenging due to the large domain gap between a sparse, irregular sketch and a regular, dense 3D shape.
Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction
The previous deep video compression approaches only use the single scale motion compensation strategy and rarely adopt the mode prediction technique from the traditional standards like H. 264/H. 265 for both motion and residual compression.
Revisiting Deep Semi-supervised Learning: An Empirical Distribution Alignment Framework and Its Generalization Bound
Benefited from this new perspective, we first propose a new deep semi-supervised learning framework called Semi-supervised Learning by Empirical Distribution Alignment (SLEDA), in which existing technologies from the domain adaptation community can be readily used to address the semi-supervised learning problem through reducing the empirical distribution distance between labeled and unlabeled data.
LSVC: A Learning-Based Stereo Video Compression Framework
In this work, we propose the first end-to-end optimized framework for compressing automotive stereo videos (i. e., stereo videos from autonomous driving applications) from both left and right views.
3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds
Observing that the 3D captioning task and the 3D grounding task contain both shared and complementary information in nature, in this work, we propose a unified framework to jointly solve these two distinct but closely related tasks in a synergistic fashion, which consists of both shared task-agnostic modules and lightweight task-specific modules.
Learning Based Multi-Modality Image and Video Compression
Specifically, given the image in the reference modality (e. g., the infrared image), we use the channel-wise alignment module to produce the aligned features based on the affine transform.
EndHiC: assemble large contigs into chromosomal-level scaffolds using the Hi-C links from contig ends
As the Hi-C links of two adjacent contigs concentrate only at the neighbor ends of the contigs, larger contig size will reduce the power to differentiate adjacent (signal) and non-adjacent (noise) contig linkages, leading to a higher rate of mis-assembly.
SRDAN: Scale-Aware and Range-Aware Domain Adaptation Network for Cross-Dataset 3D Object Detection
In this work, we propose a new cross-dataset 3D object detection method named Scale-aware and Range-aware Domain Adaptation Network (SRDAN).
NG+ : A Multi-Step Matrix-Product Natural Gradient Method for Deep Learning
In this paper, a novel second-order method called NG+ is proposed.
CBANet: Towards Complexity and Bitrate Adaptive Deep Image Compression using a Single Network
Furthermore, to achieve variable bitrate decoding with one single decoder, we propose a bitrate adaptive module to project the representation from a base bitrate to the expected representation at a target bitrate for transmission.
FVC: A New Framework towards Deep Video Compression in Feature Space
In this work, we propose a feature-space video coding network (FVC) by performing all major operations (i. e., motion estimation, motion compression, motion compensation and residual compression) in the feature space.
VoxelContext-Net: An Octree based Framework for Point Cloud Compression
In this paper, we propose a two-stage deep learning framework called VoxelContext-Net for both static and dynamic point cloud compression.
Back-tracing Representative Points for Voting-based 3D Object Detection in Point Clouds
Inspired by the back-tracing strategy in the conventional Hough voting methods, in this work, we introduce a new 3D object detection method, named as Back-tracing Representative Points Network (BRNet), which generatively back-traces the representative points from the vote centers and also revisits complementary seed points around these generated points, so as to better capture the fine local structural features surrounding the potential objects from the raw point clouds.
Ranked #17 on
3D Object Detection
on ScanNetV2
VDM-DA: Virtual Domain Modeling for Source Data-free Domain Adaptation
On the other hand, we also design an effective distribution alignment method to reduce the distribution divergence between the virtual domain and the target domain by gradually improving the compactness of the target domain distribution through model learning.
Salient Object Detection via Integrity Learning
We define the concept of integrity at both a micro and macro level.
StyleFormer: Real-Time Arbitrary Style Transfer via Parametric Style Composition
In this work, we propose a new feed-forward arbitrary style transfer method, referred to as StyleFormer, which can simultaneously fulfill fine-grained style diversity and semantic content coherency.
Formal Language Constrained Markov Decision Processes
In order to satisfy safety conditions, an agent may be constrained from acting freely.
3DVG-Transformer: Relation Modeling for Visual Grounding on Point Clouds
Visual grounding on 3D point clouds is an emerging vision and language task that benefits various applications in understanding the 3D visual world.
Inception Convolution with Efficient Dilation Search
To develop a practical method for learning complex inception convolution based on the data, a simple but effective search algorithm, referred to as efficient dilation optimization (EDO), is developed.
Human-centric Spatio-Temporal Video Grounding With Visual Transformers
HC-STVG is a video grounding task that requires both spatial (where) and temporal (when) localization.
Deep Learning Analysis and Age Prediction from Shoeprints
We collected 100, 000 shoeprints of subjects ranging from 7 to 80 years old and used the data to develop a deep learning end-to-end model ShoeNet to analyze age-related patterns and predict age.
A Method of Generating Measurable Panoramic Image for Indoor Mobile Measurement System
This paper designs a technique route to generate high-quality panoramic image with depth information, which involves two critical research hotspots: fusion of LiDAR and image data and image stitching.
A Simple and Efficient Registration of 3D Point Cloud and Image Data for Indoor Mobile Mapping System
Registration of 3D LiDAR point clouds with optical images is critical in the combination of multi-source data.
Improving Deep Video Compression by Resolution-adaptive Flow Coding
In this work, we propose a new framework called Resolution-adaptive Flow Coding (RaFC) to effectively compress the flow maps globally and locally, in which we use multi-resolution representations instead of single-resolution representations for both the input flow maps and the output motion features of the MV encoder.
Systematic Generation of Diverse Benchmarks for DNN Verification
The field of verification has advanced due to the interplay of theoretical development and empirical evaluation.
Simulating multi-exit evacuation using deep reinforcement learning
We propose a multi-exit evacuation simulation based on Deep Reinforcement Learning (DRL), referred to as the MultiExit-DRL, which involves in a Deep Neural Network (DNN) framework to facilitate state-to-action mapping.
Enhance Curvature Information by Structured Stochastic Quasi-Newton Methods
In this paper, we consider stochastic second-order methods for minimizing a finite summation of nonconvex functions.
Sketchy Empirical Natural Gradient Methods for Deep Learning
Experiments on the distributed large-batch training show that the scaling efficiency is quite reasonable.
Content Adaptive and Error Propagation Aware Deep Video Compression
Therefore, the encoder is adaptive to different video contents and achieves better compression performance by reducing the domain gap between the training and testing datasets.
Channel Pruning Guided by Classification Loss and Feature Importance
To this end, we propose a new strategy to suppress the influence of unimportant features (i. e., the features will be removed at the next pruning stage).
A Unified End-to-End Framework for Efficient Deep Image Compression
Our EDIC method can also be readily incorporated with the Deep Video Compression (DVC) framework to further improve the video compression performance.
Translating multispectral imagery to nighttime imagery via conditional generative adversarial networks
The results of this study prove the possibility of multispectral-to-nighttime translation and further indicate that, with the additional social media data, the generated nighttime imagery can be very similar to the ground-truth imagery.
Formal Language Constraints for Markov Decision Processes
In order to satisfy safety conditions, an agent may be constrained from acting freely.
IntersectGAN: Learning Domain Intersection for Generating Images with Multiple Attributes
For example, given two image domains $X_1$ and $X_2$ with certain attributes, the intersection $X_1 \cap X_2$ denotes a new domain where images possess the attributes from both $X_1$ and $X_2$ domains.
Refactoring Neural Networks for Verification
Deep neural networks (DNN) are growing in capability and applicability.
Deep Learning Detection of Inaccurate Smart Electricity Meters: A Case Study
Detecting inaccurate smart meters and targeting them for replacement can save significant resources.
Improving Action Localization by Progressive Cross-stream Cooperation
Specifically, we first generate a larger set of region proposals by combining the latest region proposals from both streams, from which we can readily obtain a larger set of labelled training samples to help learn better action detection models.
Hashing based Answer Selection
HAS adopts a hashing strategy to learn a binary matrix representation for each answer, which can dramatically reduce the memory cost for storing the matrix representations of answers.
Gated Group Self-Attention for Answer Selection
Nevertheless, it is difficult for RNN based models to capture the information about long-range dependency among words in the sentences of questions and answers.
DVC: An End-to-end Deep Video Compression Framework
Conventional video compression approaches use the predictive coding architecture and encode the corresponding motion information and residual information.
Constraining Action Sequences with Formal Languages for Deep Reinforcement Learning
We study the problem of deep reinforcement learning where the agent's action sequences are constrained, e. g., prohibition of dithering or overactuating action sequences that might damage a robot, drone, or other physical device.
Deep Kalman Filtering Network for Video Compression Artifact Reduction
In this paper, we model the video artifact reduction task as a Kalman filtering procedure and restore decoded frames through a deep Kalman filtering network.
Dividing and Aggregating Network for Multi-view Action Recognition
We then train view-specific action classifiers based on the view-specific representation for each view and a view classifier based on the shared representation at lower layers.
Collaborative and Adversarial Network for Unsupervised Domain Adaptation
In this paper, we propose a new unsupervised domain adaptation approach called Collaborative and Adversarial Network (CAN) through domain-collaborative and domain-adversarial training of neural networks.
Complex Event Detection by Identifying Reliable Shots From Untrimmed Videos
relevant) to the given event class, we formulate this task as a multi-instance learning (MIL) problem by taking each video as a bag and the video shots in each video as instances.
MUFold-SS: Protein Secondary Structure Prediction Using Deep Inception-Inside-Inception Networks
Results: Here, a very deep neural network, the deep inception-inside-inception networks (Deep3I), is proposed for protein secondary structure prediction and a software tool was implemented using this network.
Image Projective Invariants
In this paper, we propose relative projective differential invariants (RPDIs) which are invariant to general projective transformations.
SPFTN: A Self-Paced Fine-Tuning Network for Segmenting Objects in Weakly Labelled Videos
Object segmentation in weakly labelled videos is an interesting yet challenging task, which aims at learning to perform category-specific video object segmentation by only using video-level tags.
Skeleton-Based Action Recognition Using Spatio-Temporal LSTM Network with Trust Gates
Skeleton-based human action recognition has attracted a lot of research attention during the past few years.
Ranked #6 on
One-Shot 3D Action Recognition
on NTU RGB+D 120
Recent Advances in Transfer Learning for Cross-Dataset Visual Recognition: A Problem-Oriented Perspective
This paper takes a problem-oriented perspective and presents a comprehensive review of transfer learning methods, both shallow and deep, for cross-dataset visual recognition.
Shape DNA: Basic Generating Functions for Geometric Moment Invariants
Two fundamental building blocks or generating functions (GFs) for invariants are discovered, which are dot product and vector product of point vectors in Euclidean space.
Learning Multi-level Deep Representations for Image Emotion Classification
The proposed MldrNet combines deep representations of different levels, i. e. image semantics, image aesthetics, and low-level visual features to effectively classify the emotion types of different kinds of images, such as abstract paintings and web images.
A Siamese Long Short-Term Memory Architecture for Human Re-Identification
Matching pedestrians across multiple camera views known as human re-identification (re-identification) is a challenging problem in visual surveillance.
Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
To handle the noise and occlusion in 3D skeleton data, we introduce new gating mechanism within LSTM to learn the reliability of the sequential input data and accordingly adjust its effect on updating the long-term context information stored in the memory cell.
Ranked #8 on
Skeleton Based Action Recognition
on SBU
Full-Time Supervision based Bidirectional RNN for Factoid Question Answering
Hence, these existing models don't put supervision (loss or similarity calculation) at every time step, which will lose some useful information.
Proximal Riemannian Pursuit for Large-Scale Trace-Norm Minimization
Trace-norm regularization plays an important role in many areas such as machine learning and computer vision.
Fast Algorithms for Linear and Kernel SVM+
The SVM+ approach has shown excellent performance in visual recognition tasks for exploiting privileged information in the training data.
Dimensionality-Dependent Generalization Bounds for $k$-Dimensional Coding Schemes
Can we obtain dimensionality-dependent generalization bounds for $k$-dimensional coding schemes that are tighter than dimensionality-independent bounds when data is in a finite-dimensional feature space?
Multi-View Domain Generalization for Visual Recognition
Considering the recent works show the domain generalization capability can be enhanced by fusing multiple SVM classifiers, we build upon exemplar SVMs to learn a set of SVM classifiers by using one positive sample and all negative samples in the source domain each time.
Visual Recognition by Learning From Web Data: A Weakly Supervised Domain Generalization Approach
In this work, we formulate a new weakly supervised domain generalization problem for the visual recognition task by using loosely labeled web images/videos as training data.
FaLRR: A Fast Low Rank Representation Solver
In this paper, we develop a fast LRR solver called FaLRR, by reformulating LRR as a new optimization problem with regard to factorized data (which is obtained by skinny SVD of the original data matrix).
Object-Based RGBD Image Co-Segmentation With Mutex Constraint
We present an object-based co-segmentation method that takes advantage of depth data and is able to correctly handle noisy images in which the common foreground object is missing.
Scalable Nuclear-norm Minimization by Subspace Pursuit Proximal Riemannian Gradient
Nuclear-norm regularization plays a vital role in many learning tasks, such as low-rank matrix recovery (MR), and low-rank representation (LRR).
Recognizing RGB Images by Learning from RGB-D Data
In this work, we propose a new framework for recognizing RGB images captured by the conventional cameras by leveraging a set of labeled RGB-D data, in which the depth features can be additionally extracted from the depth images.
Object-based Multiple Foreground Video Co-segmentation
We present a video co-segmentation method that uses category-independent object proposals as its basic element and can extract multiple foreground objects in a video set.
Fusing Robust Face Region Descriptors via Multiple Metric Learning for Face Recognition in the Wild
Spatial-Temporal Face Region Descriptor, STFRD) for images (resp.
Learning by Associating Ambiguously Labeled Images
Our framework is motivated by the observation that samples from the same class repetitively appear in the collection of ambiguously labeled training images, while they are just ambiguously labeled in each image.
Event Recognition in Videos by Learning from Heterogeneous Web Sources
In this work, we propose to leverage a large number of loosely labeled web videos (e. g., from YouTube) and web images (e. g., from Google/Bing image search) for visual event recognition in consumer videos without requiring any labeled consumer videos.
