Search Results for author:
Found 83 papers, 48 papers with code
Towards Automatic Curation of Antibiotic Resistance Genes via Statement Extraction from Scientific Papers: A Benchmark Dataset and Models
Antibiotic resistance has become a growing worldwide concern as new resistance mechanisms are emerging and spreading globally, and thus detecting and collecting the cause – Antibiotic Resistance Genes (ARGs), have been more critical than ever.
DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning
The recent progress in text-to-image models pretrained on large-scale datasets has enabled us to generate various images as long as we provide a text prompt describing what we want.
Align and Aggregate: Compositional Reasoning with Video Alignment and Answer Aggregation for Video Question-Answering
The video aligner hierarchically selects the relevant video clips based on the question, while the answer aggregator deduces the answer to the question based on its sub-questions, with compositional consistency ensured by the information flow along question decomposition graph and the contrastive learning strategy.
PathoLM: Identifying pathogenicity from the DNA sequence through the Genome Foundation Model
Pathogen identification is pivotal in diagnosing, treating, and preventing diseases, crucial for controlling infections and safeguarding public health.
Self-Supervised Vision Transformer for Enhanced Virtual Clothes Try-On
Virtual clothes try-on has emerged as a vital feature in online shopping, offering consumers a critical tool to visualize how clothing fits.
User-Friendly Customized Generation with Multi-Modal Prompts
Text-to-image generation models have seen considerable advancement, catering to the increasing interest in personalized image creation.
Assessing Image Inpainting via Re-Inpainting Self-Consistency Evaluation
Image inpainting, the task of reconstructing missing segments in corrupted images using available data, faces challenges in ensuring consistency and fidelity, especially under information-scarce conditions.
Rethinking Classifier Re-Training in Long-Tailed Recognition: A Simple Logits Retargeting Approach
In the long-tailed recognition field, the Decoupled Training paradigm has demonstrated remarkable capabilities among various methods.
Painterly Image Harmonization by Learning from Painterly Objects
In particular, we learn a mapping from background style and object information to object style based on painterly objects in artistic paintings.
Progressive Painterly Image Harmonization from Low-level Styles to High-level Styles
Painterly image harmonization aims to harmonize a photographic foreground object on the painterly background.
Dataset Distillation in Latent Space
Dataset distillation (DD) is a newly emerging research area aiming at alleviating the heavy computational load in training models on large datasets.
Painterly Image Harmonization via Adversarial Residual Learning
In this work, we employ adversarial learning to bridge the domain gap between foreground feature map and background feature map.
Virtual Accessory Try-On via Keypoint Hallucination
Specifically, our approach learns the human body priors and hallucinates the target locations of specified foreground keypoints in the background.
Taming the Power of Diffusion Models for High-Quality Virtual Try-On with Appearance Flow
Our approach, namely Diffusion-based Conditional Inpainting for Virtual Try-ON (DCI-VTON), effectively utilizes the power of the diffusion model, and the incorporation of the warping module helps to produce high-quality and realistic virtual try-on results.
Deep Image Harmonization in Dual Color Spaces
The network comprises a $RGB$ harmonization backbone, an $Lab$ encoding module, and an $Lab$ control module.
Ranked #1 on
Image Harmonization
on HAdobe5k(1024$\times$1024)
Painterly Image Harmonization using Diffusion Model
Painterly image harmonization aims to insert photographic objects into paintings and obtain artistically coherent composite images.
Deep Image Harmonization with Learnable Augmentation
In particular, our designed SYthetic COmposite Network (SycoNet) takes in a real image with foreground mask and a random vector to learn suitable color transformation, which is applied to the foreground of this real image to produce a synthetic composite image.
Deep Image Harmonization with Globally Guided Feature Transformation and Relation Distillation
Given a composite image, image harmonization aims to adjust the foreground illumination to be consistent with background.
Mastering Long-Tail Complexity on Graphs: Characterization, Learning, and Generalization
To bridge this gap, we propose a generalization bound for long-tail classification on graphs by formulating the problem in the fashion of multi-task learning, i. e., each task corresponds to the prediction of one particular class.
WeditGAN: Few-Shot Image Generation via Latent Space Relocation
In this work, we introduce WeditGAN, which realizes model transfer by editing the intermediate latent codes $w$ in StyleGANs with learned constant offsets ($\Delta w$), discovering and constructing target latent spaces via simply relocating the distribution of source latent spaces.
Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation
First, we train a data-efficient StyleGAN2 on defect-free images as the backbone.
Fine-grained Visible Watermark Removal
Visible watermark removal aims to erase the watermark from watermarked image and recover the background image, which is a challenging task due to the diverse watermarks.
Knowledge Proxy Intervention for Deconfounded Video Question Answering
To tackle the challenge that the confounder in VideoQA is unobserved and non-enumerable in general, we propose a model-agnostic framework called Knowledge Proxy Intervention (KPI), which introduces an extra knowledge proxy variable in the causal graph to cut the backdoor path and remove the confounder.
Image Cropping With Spatial-Aware Feature and Rank Consistency
To address the first issue, we propose spatial-aware feature to encode the spatial relationship between candidate crops and aesthetic elements, by feeding the concatenation of crop mask and selectively aggregated feature maps to a light-weighted encoder.
Inharmonious Region Localization via Recurrent Self-Reasoning
Synthetic images created by image editing operations are prevalent, but the color or illumination inconsistency between the manipulated region and background may make it unrealistic.
Inharmonious Region Localization with Auxiliary Style Feature
Based on the extracted style features, we also propose a novel style voting module to guide the localization of inharmonious region.
Weak-shot Semantic Segmentation via Dual Similarity Transfer
Proposal segmentation allows proposal-pixel similarity transfer from base classes to novel classes, which enables the mask learning of novel classes.
Learning Object Placement via Dual-path Graph Completion
Object placement aims to place a foreground object over a background image with a suitable location and size.
Few-shot Image Generation Using Discrete Content Representation
Few-shot image translation disentangles an image into style vector and content map.
DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta
In this work, we propose a novel Delta Generative Adversarial Network (DeltaGAN), which consists of a reconstruction subnetwork and a generation subnetwork.
Human-centric Image Cropping with Partition-aware and Content-preserving Features
Image cropping aims to find visually appealing crops in an image, which is an important yet challenging task.
Spatial Transformation for Image Composition via Correspondence Learning
Instead, we propose a novel correspondence learning network (CorrelNet) to model the correspondence between foreground and background using cross-attention maps, based on which we can predict the target coordinate that each source coordinate of foreground should be mapped to on the background.
From Representation to Reasoning: Towards both Evidence and Commonsense Reasoning for Video Question-Answering
We hope that Causal-VidQA can guide the research of video understanding from representation learning to deeper reasoning.
Deep Video Harmonization with Color Mapping Consistency
Video harmonization aims to adjust the foreground of a composite video to make it compatible with the background.
XYLayoutLM: Towards Layout-Aware Multimodal Networks For Visually-Rich Document Understanding
Recently, various multimodal networks for Visually-Rich Document Understanding(VRDU) have been proposed, showing the promotion of transformers by integrating visual and layout information with the text embeddings.
document understanding
Optical Character Recognition (OCR)
+1
STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation
In contrast, we present a simple and efficient single-stage VIS framework based on the instance segmentation method CondInst by adding an extra tracking head.
Ranked #38 on
Video Instance Segmentation
on YouTube-VIS validation
Dual-Path Image Inpainting With Auxiliary GAN Inversion
Different from feed-forward methods, they seek for a closest latent code to the corrupted image and feed it to a pretrained generator.
Mixed Supervised Object Detection by Transferring Mask Prior and Semantic Similarity
Specifically, the ability of using mask prior to help detect objects is learned from base categories and transferred to novel categories.
Weak-shot Semantic Segmentation by Transferring Semantic Affinity and Boundary
As a result, we find that pixel-level annotation of base categories can facilitate affinity learning and propagation, leading to higher-quality CAMs of novel categories.
Large-Scale Adversarial Attacks on Graph Neural Networks via Graph Coarsening
Graph Neural Networks (GNNs) are fragile to adversarial attacks.
Defending Graph Neural Networks via Tensor-Based Robust Graph Aggregation
In this paper, we propose a tensor-based framework for GNNs to learn robust graphs from adversarial graphs by aggregating predefined robust graphs to enhance the robustness of GNNs via tensor approximation.
HYouTube: Video Harmonization Dataset
In this work, we construct a new video harmonization dataset HYouTube by adjusting the foreground of real videos to create synthetic composite videos.
High-Resolution Image Harmonization via Collaborative Dual Transformations
Conventional image harmonization methods learn global RGB-to-RGB transformation which could effortlessly scale to high resolution, but ignore diverse local context.
Ranked #3 on
Image Harmonization
on HAdobe5k(1024$\times$1024)
Visible Watermark Removal via Self-calibrated Localization and Background Refinement
In the refinement stage, we integrate multi-level features to improve the texture quality of watermarked area.
Evo-ViT: Slow-Fast Token Evolution for Dynamic Vision Transformer
Vision transformers (ViTs) have recently received explosive popularity, but the huge computational cost is still a severe issue.
Ranked #11 on
Efficient ViTs
on ImageNet-1K (with DeiT-T)
OPA: Object Placement Assessment Dataset
Image composition aims to generate realistic composite image by inserting an object from one image into another background image, where the placement (e. g., location, size, occlusion) of inserted object may be unreasonable, which would significantly degrade the quality of the composite image.
Making Images Real Again: A Comprehensive Survey on Deep Image Composition
We have also contributed the first image composition toolbox: libcom https://github. com/bcmi/libcom, which assembles 10+ image composition related functions (e. g., image blending, image harmonization, object placement, shadow generation, generative composition).
End-to-End Video Object Detection with Spatial-Temporal Transformers
Recently, DETR and Deformable DETR have been proposed to eliminate the need for many hand-designed components in object detection while demonstrating good performance as previous complex hand-crafted detectors.
Shadow Generation for Composite Image in Real-world Scenes
In this work, we focus on generating plausible shadow for the foreground object in the composite image.
Inharmonious Region Localization
The advance of image editing techniques allows users to create artistic works, but the manipulated regions may be incompatible with the background.
Image Composition Assessment with Saliency-augmented Multi-pattern Pooling
Image composition assessment is crucial in aesthetic assessment, which aims to assess the overall composition quality of a given image.
Ranked #1 on
Aesthetics Quality Assessment
on CADB
Deep Image Harmonization by Bridging the Reality Gap
Image harmonization has been significantly advanced with large-scale harmonization dataset.
Parallel Multi-Resolution Fusion Network for Image Inpainting
Conventional deep image inpainting methods are based on auto-encoder architecture, in which the spatial details of images will be lost in the down-sampling process, leading to the degradation of generated results.
Disentangled Information Bottleneck
The information bottleneck (IB) method is a technique for extracting information that is relevant for predicting the target random variable from the source random variable, which is typically implemented by optimizing the IB Lagrangian that balances the compression and prediction terms.
From Pixel to Patch: Synthesize Context-aware Features for Zero-shot Semantic Segmentation
Thus, we focus on zero-shot semantic segmentation, which aims to segment unseen objects with only category-level semantic representations provided for unseen categories.
Graph Neural Network for Fraud Detection via Spatial-Temporal Attention
But manually generating features needs domain knowledge and may lay behind the modus operandi of fraud, which means we need to automatically focus on the most relevant fraudulent behavior patterns in the online detection system.
BargainNet: Background-Guided Domain Translation for Image Harmonization
Therefore, we propose an image harmonization network with a novel domain code extractor and well-tailored triplet losses, which could capture the background domain information to guide the foreground harmonization.
Ranked #13 on
Image Harmonization
on iHarmony4
Weak-shot Fine-grained Classification via Similarity Transfer
In this setting, we propose a method called SimTrans to transfer pairwise semantic similarity from base categories to novel categories.
DeltaGAN: Towards Diverse Few-shot Image Generation with Sample-Specific Delta
In this work, we propose a novel Delta Generative Adversarial Network (DeltaGAN), which consists of a reconstruction subnetwork and a generation subnetwork.
Context-aware Feature Generation for Zero-shot Semantic Segmentation
In this paper, we propose a novel context-aware feature generation method for zero-shot segmentation named CaGNet.
Ranked #9 on
Zero-Shot Semantic Segmentation
on PASCAL VOC
F2GAN: Fusing-and-Filling GAN for Few-shot Image Generation
In this paper, we propose a Fusing-and-Filling Generative Adversarial Network (F2GAN) to generate realistic and diverse images for a new category with only a few images.
Spatio-Temporal Attention-Based Neural Network for Credit Card Fraud Detection
But manually generating features needs domain knowledge and may lay behind the modus operandi of fraud, which means we need to automatically focus on the most relevant patterns in fraudulent behavior.
Beyond without Forgetting: Multi-Task Learning for Classification with Disjoint Datasets
To address these issues, we propose our MTL with Selective Augmentation (MTL-SA) method to select the training samples in unlabeled datasets with confident pseudo labels and close data distribution to the labeled dataset.
MatchingGAN: Matching-based Few-shot Image Generation
Matching generator can match random vectors with a few conditional images from the same category and generate new images for this category based on the fused features.
Exploiting Motion Information from Unlabeled Videos for Static Image Action Recognition
Specifically, the VRE module includes a proxy task which imposes pseudo motion label constraint and temporal coherence constraint on unlabeled videos, while the MRA module could predict the motion information of a static action image by exploiting unlabeled videos.
Zero-Shot Sketch-Based Image Retrieval with Structure-aware Asymmetric Disentanglement
The goal of Sketch-Based Image Retrieval (SBIR) is using free-hand sketches to retrieve images of the same category from a natural image gallery.
DoveNet: Deep Image Harmonization via Domain Verification
Image composition is an important operation in image processing, but the inconsistency between foreground and background significantly degrades the quality of composite image.
Ranked #6 on
Image Harmonization
on HAdobe5k(1024$\times$1024)
Image Cropping with Composition and Saliency Aware Aesthetic Score Map
Aesthetic image cropping is a practical but challenging task which aims at finding the best crops with the highest aesthetic quality in an image.
A Proposal-based Approach for Activity Image-to-Video Retrieval
Activity image-to-video retrieval task aims to retrieve videos containing the similar activity as the query image, which is a challenging task because videos generally have many background segments irrelevant to the activity.
Image Harmonization Dataset iHarmony4: HCOCO, HAdobe5k, HFlickr, and Hday2night
Image composition is an important operation in image processing, but the inconsistency between foreground and background significantly degrades the quality of composite image.
Learning from Web Data with Self-Organizing Memory Module
However, crawled web images usually have two types of noises, label noise and background noise, which induce extra difficulties in utilizing them effectively.
Ranked #14 on
Image Classification
on WebVision-1000
Hard Pixel Mining for Depth Privileged Semantic Segmentation
Specifically, we propose a novel Loss Weight Module, which outputs a loss weight map by employing two depth-related measurements of hard pixels: Depth Prediction Error and Depthaware Segmentation Error.
Multi-shot Pedestrian Re-identification via Sequential Decision Making
In contrary to existing works that aggregate single frames features by time series model such as recurrent neural network, in this paper, we propose an interpretable reinforcement learning based approach to this problem.
Rectifier Neural Network with a Dual-Pathway Architecture for Image Denoising
Recently deep neural networks based on tanh activation function have shown their impressive power in image denoising.
Tensor Ring Decomposition
In this paper, we introduce a fundamental tensor decomposition model to represent a large dimensional tensor by a circular multilinear products over a sequence of low dimensional cores, which can be graphically interpreted as a cyclic interconnection of 3rd-order tensors, and thus termed as tensor ring (TR) decomposition.
Object Proposal by Multi-Branch Hierarchical Segmentation
Hierarchical segmentation based object proposal methods have become an important step in modern object detection paradigm.
Bayesian Sparse Tucker Models for Dimension Reduction and Tensor Completion
Tucker decomposition is the cornerstone of modern machine learning on tensorial data analysis, which have attracted considerable attention for multiway feature extraction, compressive sensing, and tensor completion.
Bayesian Robust Tensor Factorization for Incomplete Multiway Data
We propose a generative model for robust tensor factorization in the presence of both missing data and outliers.
Bayesian CP Factorization of Incomplete Tensors with Automatic Rank Determination
CANDECOMP/PARAFAC (CP) tensor factorization of incomplete data is a powerful technique for tensor completion through explicitly capturing the multilinear latent factors.
Spatial-Spectral Boosting Analysis for Stroke Patients' Motor Imagery EEG in Rehabilitation Training
Current studies about motor imagery based rehabilitation training systems for stroke subjects lack an appropriate analytic method, which can achieve a considerable classification accuracy, at the same time detects gradual changes of imagery patterns during rehabilitation process and disinters potential mechanisms about motor function recovery.
Higher-Order Partial Least Squares (HOPLS): A Generalized Multi-Linear Regression Method
A new generalized multilinear regression model, termed the Higher-Order Partial Least Squares (HOPLS), is introduced with the aim to predict a tensor (multiway array) $\tensor{Y}$ from a tensor $\tensor{X}$ through projecting the data onto the latent space and performing regression on the corresponding latent variables.
Dynamic visual attention: searching for coding length increments
A visual attention system should respond placidly when common stimuli are presented, while at the same time keep alert to anomalous visual inputs.
