Search Results for author:
Found 101 papers, 37 papers with code
Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding
Existing methods for multi-modal manipulation detection and grounding primarily focus on fusing vision-language features to make predictions, while overlooking the importance of modality-specific features, leading to sub-optimal results.
Improving Adversarial Robustness of Masked Autoencoders via Test-time Frequency-domain Prompting
Based on our analysis, we provide a simple yet effective way to boost the adversarial robustness of MAE.
MotionGPT: Finetuned LLMs are General-Purpose Motion Generators
Generating realistic human motion from given action descriptions has experienced significant advancements because of the emerging requirement of digital humans.
EVOPOSE: A Recursive Transformer For 3D Human Pose Estimation With Kinematic Structure Priors
Transformer is popular in recent 3D human pose estimation, which utilizes long-term modeling to lift 2D keypoints into the 3D space.
Ranked #83 on
3D Human Pose Estimation
on Human3.6M
Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal
Inspired by the various successful applications of large-scale pre-trained models (e. g, CLIP), in this paper, we explore the potential benefits of them for this task through both spatial feature representation learning and semantic information embedding aspects: 1) for spatial feature representation learning, we design a Spatially-Adaptive Residual (\textbf{SAR}) Encoder to extract degraded areas adaptively.
HQ-50K: A Large-scale, High-quality Dataset for Image Restoration
This paper introduces a new large-scale image restoration dataset, called HQ-50K, which contains 50, 000 high-quality images with rich texture details and semantic diversity.
GPT Paternity Test: GPT Generated Text Detection with GPT Genetic Inheritance
By comparing the similarity between the original text and the generated re-answered text, it can be determined whether the text is machine-generated.
Multi-spectral Class Center Network for Face Manipulation Detection and Localization
Next, a novel Multi-Spectral Class Center Network (MSCCNet) is proposed for face manipulation detection and localization.
Watermarking Text Generated by Black-Box Language Models
To allow third-parties to autonomously inject watermarks into generated text, we develop a watermarking framework for black-box language model usage scenarios.
Clothes-Invariant Feature Learning by Causal Intervention for Clothes-Changing Person Re-identification
Clothes-invariant feature extraction is critical to the clothes-changing person re-identification (CC-ReID).
Clothes Changing Person Re-Identification
Person Re-Identification
Deep Intellectual Property Protection: A Survey
Deep Neural Networks (DNNs), from AlexNet to ResNet to ChatGPT, have made revolutionary progress in recent years, and are widely used in various fields.
Diversity-Aware Meta Visual Prompting
We present Diversity-Aware Meta Visual Prompting~(DAM-VP), an efficient and effective prompting method for transferring pre-trained models to downstream tasks with frozen backbone.
Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network
At first, a top-n selection strategy is proposed to provide pseudo-labels for public data, and use pseudo-labels to guide the training of the cGAN.
VQNet 2.0: A New Generation Machine Learning Framework that Unifies Classical and Quantum
Therefore, based on VQNet 1. 0, we further propose VQNet 2. 0, a new generation of unified classical and quantum machine learning framework that supports hybrid optimization.
CLIP Itself is a Strong Fine-tuner: Achieving 85.7% and 88.0% Top-1 Accuracy with ViT-B and ViT-L on ImageNet
Recent studies have shown that CLIP has achieved remarkable success in performing zero-shot inference while its fine-tuning performance is not satisfactory.
X-Paste: Revisiting Scalable Copy-Paste for Instance Segmentation using CLIP and StableDiffusion
We demonstrate for the first time that using a text2image model to generate images or zero-shot recognition model to filter noisily crawled images for different object categories is a feasible way to make Copy-Paste truly scalable.
Ranked #5 on
Instance Segmentation
on LVIS v1.0 val
Exploring the Limits of Differentially Private Deep Learning with Group-wise Clipping
To reduce the compute time overhead of private learning, we show that \emph{per-layer clipping}, where the gradient of each neural network layer is clipped separately, allows clipping to be performed in conjunction with backpropagation in differentially private optimization.
Ada3Diff: Defending against 3D Adversarial Point Clouds via Adaptive Diffusion
Deep 3D point cloud models are sensitive to adversarial attacks, which poses threats to safety-critical applications such as autonomous driving.
UIA-ViT: Unsupervised Inconsistency-Aware Method based on Vision Transformer for Face Forgery Detection
UPCL is designed for learning the consistency-related representation with progressive optimized pseudo annotations.
PointCAT: Contrastive Adversarial Training for Robust Point Cloud Recognition
Notwithstanding the prominent performance achieved in various applications, point cloud recognition models have often suffered from natural corruptions and adversarial perturbations.
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining
Second, masked self-distillation is also consistent with vision-language contrastive from the perspective of training objective as both utilize the visual encoder for feature aligning, and thus is able to learn local semantics getting indirect supervision from the language.
Counterfactual Intervention Feature Transfer for Visible-Infrared Person Re-identification
But we find existing graph-based methods in the visible-infrared person re-identification task (VI-ReID) suffer from bad generalization because of two issues: 1) train-test modality balance gap, which is a property of VI-ReID task.
Bootstrapped Masked Autoencoders for Vision BERT Pretraining
The first design is motivated by the observation that using a pretrained MAE to extract the features as the BERT prediction target for masked tokens can achieve better pretraining performance.
Towards Intrinsic Common Discriminative Features Learning for Face Forgery Detection using Adversarial Learning
Existing face forgery detection methods usually treat face forgery detection as a binary classification problem and adopt deep convolution neural networks to learn discriminative features.
Reduce Information Loss in Transformers for Pluralistic Image Inpainting
The indices of quantized pixels are used as tokens for the inputs and prediction targets of transformer.
Ranked #6 on
Seeing Beyond the Visible
on KITTI360-EX
Invertible Mask Network for Face Privacy-Preserving
Then, put the "Mask" face onto the protected face and generate the masked face, in which the masked face is indistinguishable from "Mask" face.
Real-time Online Multi-Object Tracking in Compressed Domain
Recent online Multi-Object Tracking (MOT) methods have achieved desirable tracking performance.
Shape-invariant 3D Adversarial Point Clouds
In this paper, we propose a novel Point-Cloud Sensitivity Map to boost both the efficiency and imperceptibility of point perturbations.
Protecting Celebrities from DeepFake with Identity Consistency Transformer
In this work we propose Identity Consistency Transformer, a novel face forgery detection method that focuses on high-level semantics, specifically identity information, and detecting a suspect face by finding identity inconsistency in inner and outer face regions.
Self-supervised Transformer for Deepfake Detection
After pre-training with our method, the model will then be partially fine-tuned for deepfake detection task.
Online Multi-Object Tracking with Unsupervised Re-Identification Learning and Occlusion Estimation
In addition, such practice of re-identification still can not track those highly occluded objects when they are missed by the detector.
Ranked #7 on
Multi-Object Tracking
on MOT16
(using extra training data)
Initiative Defense against Facial Manipulation
To this end, we first imitate the target manipulation model with a surrogate model, and then devise a poison perturbation generator to obtain the desired venom.
Tracing Text Provenance via Context-Aware Lexical Substitution
Tracing text provenance can help claim the ownership of text content or identify the malicious users who distribute misleading content like machine-generated fake news.
HairCLIP: Design Your Hair by Text and Reference Image
Hair editing is an interesting and challenging problem in computer vision and graphics.
PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers
This paper explores a better prediction target for BERT pre-training of vision transformers.
Speech Pattern based Black-box Model Watermarking for Automatic Speech Recognition
As an effective method for intellectual property (IP) protection, model watermarking technology has been applied on a wide variety of deep neural networks (DNN), including speech classification models.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Unsupervised Finetuning
This problem is more challenging than the supervised counterpart, as the low data density in the small-scale target data is not friendly for unsupervised learning, leading to the damage of the pretrained representation and poor representation in the target domain.
Temporal RoI Align for Video Object Recognition
In this work, considering the features of the same object instance are highly similar among frames in a video, a novel Temporal RoI Align operator is proposed to extract features from other frames feature maps for current frame proposals by utilizing feature similarity.
Ranked #1 on
Video Instance Segmentation
on YouTube-VIS
ISNet: Integrate Image-Level and Semantic-Level Context for Semantic Segmentation
Third, we compute the similarities between each pixel representation and the image-level contextual information, the semantic-level contextual information, respectively.
Exploring Structure Consistency for Deep Model Watermarking
However, little attention has been devoted to the protection of DNNs in image processing tasks.
Poison Ink: Robust and Invisible Backdoor Attack
As the image structure can keep its semantic meaning during the data transformation, such trigger pattern is inherently robust to data transformations.
Cascaded Residual Density Network for Crowd Counting
Crowd counting is a challenging task due to the issues such as scale variation and perspective variation in real crowd scenes.
Abnormal Behavior Detection Based on Target Analysis
To this end, we propose a multivariate fusion method that analyzes each target through three branches: object, action and motion.
Improve Unsupervised Pretraining for Few-label Transfer
Unsupervised pretraining has achieved great success and many recent works have shown unsupervised pretraining can achieve comparable or even slightly better transfer performance than supervised pretraining on downstream target datasets.
CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows
By further pretraining on the larger dataset ImageNet-21K, we achieve 87. 5% Top-1 accuracy on ImageNet-1K and high segmentation performance on ADE20K with 55. 7 mIoU.
Ranked #25 on
Semantic Segmentation
on ADE20K val
E2Style: Improve the Efficiency and Effectiveness of StyleGAN Inversion
This paper studies the problem of StyleGAN inversion, which plays an essential role in enabling the pretrained StyleGAN to be used for real image editing tasks.
Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication
Specifically, we present the reversible watermarking problem of deep convolutional neural networks and utilize the pruning theory of model compression technology to construct a host sequence used for embedding watermarking information by histogram shift.
Towards Generalizable and Robust Face Manipulation Detection via Bag-of-local-feature
Over the past several years, in order to solve the problem of malicious abuse of facial manipulation technology, face manipulation detection technology has obtained considerable attention and achieved remarkable progress.
Diverse Semantic Image Synthesis via Probability Distribution Modeling
In this paper, we propose a novel diverse semantic image synthesis framework from the perspective of semantic class distributions, which naturally supports diverse generation at semantic or even instance level.
Ranked #1 on
Image-to-Image Translation
on ADE20K Labels-to-Photos
(LPIPS metric)
Deep Model Intellectual Property Protection via Deep Watermarking
By jointly training the target model and watermark embedding, the extra barrier can even be absorbed into the target model.
Multi-attentional Deepfake Detection
Most of them model deepfake detection as a vanilla binary classification problem, i. e, first use a backbone network to extract a global feature and then feed it into a binary classifier (real/fake).
Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain
The remarkable success in face forgery techniques has received considerable attention in computer vision due to security concerns.
Adversarial Examples Detection beyond Image Space
To detect both few-perturbation attacks and large-perturbation attacks, we propose a method beyond image space by a two-stream architecture, in which the image stream focuses on the pixel artifacts and the gradient stream copes with the confidence artifacts.
Return-Based Contrastive Representation Learning for Reinforcement Learning
Recently, various auxiliary tasks have been proposed to accelerate representation learning and improve sample efficiency in deep reinforcement learning (RL).
Improved Image Matting via Real-time User Clicks and Uncertainty Estimation
Image matting is a fundamental and challenging problem in computer vision and graphics.
Are Fewer Labels Possible for Few-shot Learning?
We conduct experiments on 10 different few-shot target datasets, and our average few-shot performance outperforms both vanilla inductive unsupervised transfer and supervised transfer by a large margin.
Efficient Semantic Image Synthesis via Class-Adaptive Normalization
Spatially-adaptive normalization (SPADE) is remarkably successful recently in conditional semantic image synthesis \cite{park2019semantic}, which modulates the normalized activation with spatially-varying transformations learned from semantic layouts, to prevent the semantic information from being washed away.
Identity-Driven DeepFake Detection
Our approach takes as input the suspect image/video as well as the target identity information (a reference image or video).
LG-GAN: Label Guided Adversarial Network for Flexible Targeted Attack of Point Cloud-based Deep Networks
To overcome these shortcomings, this paper proposes a novel label guided adversarial network (LG-GAN) for real-time flexible targeted point cloud attack.
MichiGAN: Multi-Input-Conditioned Hair Image Generation for Portrait Editing
In this paper, we present MichiGAN (Multi-Input-Conditioned Hair Image GAN), a novel conditional image generation method for interactive portrait hair manipulation.
Passport-aware Normalization for Deep Model Protection
Only when the model IP is suspected to be stolen by someone, the private passport-aware branch is added back for ownership verification.
GreedyFool: Distortion-Aware Sparse Adversarial Attack
Sparse adversarial samples are a special branch of adversarial samples that can fool the target model by only perturbing a few pixels.
Rethinking Spatially-Adaptive Normalization
Despite its impressive performance, a more thorough understanding of the true advantages inside the box is still highly demanded, to help reduce the significant computation and parameter overheads introduced by these new structures.
Density-Aware Graph for Deep Semi-Supervised Visual Recognition
Motivated by these limitations, this paper proposes to solve the SSL problem by building a novel density-aware graph, based on which the neighborhood information can be easily leveraged and the feature learning and label propagation can also be trained in an end-to-end way.
Cross-modality Person re-identification with Shared-Specific Feature Transfer
In this paper, we tackle the above limitation by proposing a novel cross-modality shared-specific feature transfer algorithm (termed cm-SSFT) to explore the potential of both the modality-shared information and the modality-specific characteristics to boost the re-identification performance.
Cross-Modality Person Re-identification
Person Re-Identification
Model Watermarking for Image Processing Networks
In this way, when the attacker trains one surrogate model by using the input-output pairs of the target model, the hidden watermark will be learned and extracted afterward.
Self-supervised Adversarial Training
Recent work has demonstrated that neural networks are vulnerable to adversarial examples.
Demonstration Actor Critic
One approach leverages demonstration data in a supervised manner, which is simple and direct, but can only provide supervision signal over those states seen in the demonstrations.
Once a MAN: Towards Multi-Target Attack via Learning Multi-Target Adversarial Network Once
Recently, generation-based methods have received much attention since they directly use feed-forward networks to generate the adversarial samples, which avoid the time-consuming iterative attacking procedure in optimization-based and gradient-based methods.
Memory-Based Neighbourhood Embedding for Visual Recognition
Learning discriminative image feature embeddings is of great importance to visual recognition.
A General Decoupled Learning Framework for Parameterized Image Operators
To overcome this limitation, we propose a new decoupled learning algorithm to learn from the operator parameters to dynamically adjust the weights of a deep network for image operators, denoted as the base network.
A Large Scale Urban Surveillance Video Dataset for Multiple-Object Tracking and Behavior Analysis
Multiple-object tracking and behavior analysis have been the essential parts of surveillance video analysis for public security and urban management.
Context and Attribute Grounded Dense Captioning
Dense captioning aims at simultaneously localizing semantic regions and describing these regions-of-interest (ROIs) with short phrases or sentences in natural language.
Ranked #2 on
Dense Captioning
on Visual Genome
Semantics Disentangling for Text-to-Image Generation
Synthesizing photo-realistic images from text descriptions is a challenging problem.
DUP-Net: Denoiser and Upsampler Network for 3D Adversarial Point Clouds Defense
We propose a Denoiser and UPsampler Network (DUP-Net) structure as defenses for 3D adversarial point cloud classification, where the two modules reconstruct surface smoothness by dropping or adding points.
Real-Time Anomaly Detection With HMOF Feature
Most existing methods are computation consuming, which cannot satisfy the real-time requirement.
Emerging Applications of Reversible Data Hiding
Reversible data hiding (RDH) is one special type of information hiding, by which the host sequence as well as the embedded data can be both restored from the marked sequence without loss.
CAAD 2018: Iterative Ensemble Adversarial Attack
Deep Neural Networks (DNNs) have recently led to significant improvements in many fields.
CAAD 2018: Powerful None-Access Black-Box Attack Based on Adversarial Transformation Network
In this paper, we propose an improvement of Adversarial Transformation Networks(ATN) to generate adversarial examples, which can fool white-box models and black-box models with a state of the art performance and won the 2rd place in the non-target task in CAAD 2018.
Capacity Control of ReLU Neural Networks by Basis-path Norm
Motivated by this, we propose a new norm \emph{Basis-path Norm} based on a group of linearly independent paths to measure the capacity of neural networks more accurately.
Decouple Learning for Parameterized Image Operators
Many different deep networks have been used to approximate, accelerate or improve traditional image operators, such as image smoothing, super-resolution and denoising.
Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition
We show that by encouraging deep message propagation and interactions between local object features and global predicate features, one can achieve compelling performance in recognizing complex relationships without using any linguistic priors.
Model-Level Dual Learning
Many artificial intelligence tasks appear in dual forms like English$\leftrightarrow$French translation and speech$\leftrightarrow$text transformation.
Detection based Defense against Adversarial Examples from the Steganalysis Point of View
Moreover, secondary adversarial attacks cannot be directly performed to our method because our method is not based on a neural network but based on high-dimensional artificial features and FLD (Fisher Linear Discriminant) ensemble.
Stereoscopic Neural Style Transfer
This paper presents the first attempt at stereoscopic neural style transfer, which responds to the emerging demand for 3D movies or AR/VR.
Deliberation Networks: Sequence Generation Beyond One-Pass Decoding
In this work, we introduce the deliberation process into the encoder-decoder framework and propose deliberation networks for sequence generation.
Enhanced Attacks on Defensively Distilled Deep Neural Networks
In this paper, we first propose the epsilon-neighborhood attack, which can fool the defensively distilled networks with 100% success rate in the white-box setting, and it is fast to generate adversarial examples with good visual quality.
Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism
The visibility map of the target is learned and used for inferring the spatial attention map.
Dual Supervised Learning
Many supervised learning tasks are emerged in dual forms, e. g., English-to-French translation vs. French-to-English translation, speech recognition vs. text to speech, and image classification vs. image generation.
Coherent Online Video Style Transfer
Training a feed-forward network for fast neural style transfer of images is proven to be successful.
StyleBank: An Explicit Representation for Neural Image Style Transfer
It also enables us to conduct incremental learning to add a new image style by learning a new filter bank while holding the auto-encoder fixed.
Learning Spatial Regularization with Image-level Supervisions for Multi-label Image Classification
Analysis of the learned SRN model demonstrates that it can effectively capture both semantic and spatial relations of labels for improving classification performance.
Ranked #6 on
Multi-Label Classification
on NUS-WIDE
Dual Learning for Machine Translation
Based on the feedback signals generated during this process (e. g., the language-model likelihood of the output of a model, and the reconstruction error of the original sentence after the primal and dual translations), we can iteratively update the two models until convergence (e. g., using the policy gradient methods).
SOL: A Library for Scalable Online Learning Algorithms
SOL is an open-source library for scalable online learning algorithms, and is particularly suitable for learning with high-dimensional data.
Asynchronous Stochastic Gradient Descent with Delay Compensation
We propose a novel technology to compensate this delay, so as to make the optimization behavior of ASGD closer to that of sequential SGD.
Graph Construction with Label Information for Semi-Supervised Learning
In the literature, most existing graph-based semi-supervised learning (SSL) methods only use the label information of observed samples in the label propagation stage, while ignoring such valuable information when learning the graph.
Thompson Sampling for Budgeted Multi-armed Bandits
In this paper, we extend the Thompson sampling to Budgeted MAB, where there is random cost for pulling an arm and the total cost is constrained by a budget.
Large-scale Online Feature Selection for Ultra-high Dimensional Sparse Data
However, unlike many second-order learning methods that often suffer from extra high computational cost, we devise a novel smart algorithm for second-order online feature selection using a MaxHeap-based approach, which is not only more effective than the existing first-order approaches, but also significantly more efficient and scalable for large-scale feature selection with ultra-high dimensional sparse data, as validated from our extensive experiments.
Learning Bregman Distance Functions and Its Application for Semi-Supervised Clustering
Learning distance functions with side information plays a key role in many machine learning and data mining applications.
