Search Results for author: Jie Zhang

Found 275 papers, 86 papers with code

DiPrompT: Disentangled Prompt Tuning for Multiple Latent Domain Generalization in Federated Learning

no code implementations11 Mar 2024 Sikai Bai, Jie Zhang, Shuaicheng Li, Song Guo, Jingcai Guo, Jun Hou, Tao Han, Xiaocheng Lu

Federated learning (FL) has emerged as a powerful paradigm for learning from decentralized data, and federated domain generalization further considers the test dataset (target domain) is absent from the decentralized training data (source domains).

Domain Generalization Federated Learning +1

DiffClass: Diffusion-Based Class Incremental Learning

no code implementations8 Mar 2024 Zichong Meng, Jie Zhang, Changdi Yang, Zheng Zhan, Pu Zhao, Yanzhi Wang

On top of that, Exemplar-free Class Incremental Learning is even more challenging due to forbidden access to previous task data.

Class Incremental Learning Domain Adaptation +2

RSAM-Seg: A SAM-based Approach with Prior Knowledge Integration for Remote Sensing Image Semantic Segmentation

no code implementations29 Feb 2024 Jie Zhang, Xubing Yang, Rui Jiang, Wei Shao, Li Zhang

While the direct application of SAM to remote sensing image segmentation tasks does not yield satisfactory results, we propose RSAM-Seg, which stands for Remote Sensing SAM with Semantic Segmentation, as a tailored modification of SAM for the remote sensing field and eliminates the need for manual intervention to provide prompts.

Cloud Detection Image Segmentation +2

Towards Tracing Trustworthiness Dynamics: Revisiting Pre-training Period of Large Language Models

1 code implementation29 Feb 2024 Chen Qian, Jie Zhang, Wei Yao, Dongrui Liu, Zhenfei Yin, Yu Qiao, Yong liu, Jing Shao

This research provides an initial exploration of trustworthiness modeling during LLM pre-training, seeking to unveil new insights and spur further developments in the field.

Fairness Mutual Information Estimation

Model X-ray:Detect Backdoored Models via Decision Boundary

no code implementations27 Feb 2024 Yanghao Su, Jie Zhang, Ting Xu, Tianwei Zhang, Weiming Zhang, Nenghai Yu

To address it, in this paper, we begin by presenting an intriguing observation: the decision boundary of the backdoored model exhibits a greater degree of closeness than that of the clean model.

Pan-Mamba: Effective pan-sharpening with State Space Model

1 code implementation19 Feb 2024 Xuanhua He, Ke Cao, Keyu Yan, Rui Li, Chengjun Xie, Jie Zhang, Man Zhou

To the best of our knowledge, this work is the first attempt in exploring the potential of the Mamba model and establishes a new frontier in the pan-sharpening techniques.

Pansharpening

Towards Cross-Domain Continual Learning

1 code implementation19 Feb 2024 Marcus de Carvalho, Mahardhika Pratama, Jie Zhang, Chua Haoyan, Edward Yapp

In this work, we introduce a novel approach called Cross-Domain Continual Learning (CDCL) that addresses the limitations of being limited to single supervised domains.

Continual Learning

SIBO: A Simple Booster for Parameter-Efficient Fine-Tuning

no code implementations19 Feb 2024 Zhihao Wen, Jie Zhang, Yuan Fang

Fine-tuning all parameters of large language models (LLMs) necessitates substantial computational power and extended time.

Large Language Model with Graph Convolution for Recommendation

no code implementations14 Feb 2024 Yingpeng Du, Ziyan Wang, Zhu Sun, Haoyan Chua, Hongzhi Liu, Zhonghai Wu, Yining Ma, Jie Zhang, Youchen Sun

To adapt text-based LLMs with structured graphs, We use the LLM as an aggregator in graph processing, allowing it to understand graph-based information step by step.

Hallucination Language Modelling +1

Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning

1 code implementation4 Feb 2024 Jiacheng Chen, Zeyuan Ma, Hongshu Guo, Yining Ma, Jie Zhang, Yue-Jiao Gong

Recent Meta-learning for Black-Box Optimization (MetaBBO) methods harness neural networks to meta-learn configurations of traditional black-box optimizers.

Meta-Learning Zero-shot Generalization

PRIME: Protect Your Videos From Malicious Editing

no code implementations2 Feb 2024 Guanlin Li, Shuai Yang, Jie Zhang, Tianwei Zhang

With the development of generative models, the quality of generated content keeps increasing.

Multi-granularity Correspondence Learning from Long-term Noisy Videos

1 code implementation30 Jan 2024 Yijie Lin, Jie Zhang, Zhenyu Huang, Jia Liu, Zujie Wen, Xi Peng

Existing video-language studies mainly focus on learning short video clips, leaving long-term temporal dependencies rarely explored due to over-high computational cost of modeling long videos.

Action Segmentation Long Video Retrieval (Background Removed) +2

Adversarial speech for voice privacy protection from Personalized Speech generation

no code implementations22 Jan 2024 Shihao Chen, Liping Chen, Jie Zhang, KongAik Lee, ZhenHua Ling, LiRong Dai

For validation, we employ the open-source pre-trained YourTTS model for speech generation and protect the target speaker's speech in the white-box scenario.

Speaker Verification Voice Conversion

CBVS: A Large-Scale Chinese Image-Text Benchmark for Real-World Short Video Search Scenarios

1 code implementation19 Jan 2024 Xiangshuo Qiao, Xianxin Li, Xiaozhe Qu, Jie Zhang, Yang Liu, Yu Luo, Cihang Jin, Jin Ma

Differently, video covers in short video search scenarios are presented as user-originated contents that provide important visual summaries of videos.

Common Sense Reasoning Image Retrieval

Generalized Face Liveness Detection via De-spoofing Face Generator

no code implementations17 Jan 2024 Xingming Long, Shiguang Shan, Jie Zhang

In this paper, we conduct an Anomalous cue Guided FAS (AG-FAS) method, which leverages real faces for improving model generalization via a De-spoofing Face Generator (DFG).

Face Anti-Spoofing

Collaboratively Self-supervised Video Representation Learning for Action Recognition

no code implementations15 Jan 2024 Jie Zhang, Zhifan Wan, Lanqing Hu, Stephen Lin, Shuzhe Wu, Shiguang Shan

Considering the close connection between action recognition and human pose estimation, we design a Collaboratively Self-supervised Video Representation (CSVR) learning framework specific to action recognition by jointly considering generative pose prediction and discriminative context matching as pretext tasks.

Action Recognition Pose Estimation +2

Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness

no code implementations9 Jan 2024 Sibo Wang, Jie Zhang, Zheng Yuan, Shiguang Shan

Large-scale pre-trained vision-language models like CLIP have demonstrated impressive performance across various tasks, and exhibit remarkable zero-shot generalization capability, while they are also vulnerable to imperceptible adversarial examples.

Adversarial Robustness Zero-shot Generalization

Multichannel AV-wav2vec2: A Framework for Learning Multichannel Multi-Modal Speech Representation

1 code implementation7 Jan 2024 Qiushi Zhu, Jie Zhang, Yu Gu, Yuchen Hu, LiRong Dai

Considering that visual information helps to improve speech recognition performance in noisy scenes, in this work we propose a multichannel multi-modal speech self-supervised learning framework AV-wav2vec2, which utilizes video and multichannel audio data as inputs.

Audio-Visual Speech Recognition Automatic Speech Recognition +7

Frequency-Adaptive Pan-Sharpening with Mixture of Experts

no code implementations4 Jan 2024 Xuanhua He, Keyu Yan, Rui Li, Chengjun Xie, Jie Zhang, Man Zhou

Pan-sharpening involves reconstructing missing high-frequency information in multi-spectral images with low spatial resolution, using a higher-resolution panchromatic image as guidance.

Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain

no code implementations4 Jan 2024 Xuanhua He, Tao Hu, Guoli Wang, Zejin Wang, Run Wang, Qian Zhang, Keyu Yan, Ziyi Chen, Rui Li, Chenjun Xie, Jie Zhang, Man Zhou

However, current methods often ignore the difference between cell phone RAW images and DSLR camera RGB images, a difference that goes beyond the color matrix and extends to spatial structure due to resolution variations.

Image Restoration

FullLoRA-AT: Efficiently Boosting the Robustness of Pretrained Vision Transformers

no code implementations3 Jan 2024 Zheng Yuan, Jie Zhang, Shiguang Shan

In recent years, the Vision Transformer (ViT) model has gradually become mainstream in various computer vision tasks, and the robustness of the model has received increasing attention.

Adversarial Robustness

SAME: Sample Reconstruction against Model Extraction Attacks

1 code implementation17 Dec 2023 Yi Xie, Jie Zhang, Shiqian Zhao, Tianwei Zhang, Xiaofeng Chen

While deep learning models have shown significant performance across various domains, their deployment needs extensive resources and advanced computing infrastructure.

Model extraction

ParsNets: A Parsimonious Orthogonal and Low-Rank Linear Networks for Zero-Shot Learning

no code implementations15 Dec 2023 Jingcai Guo, Qihua Zhou, Ruibing Li, Xiaocheng Lu, Ziming Liu, Junyang Chen, Xin Xie, Jie Zhang

Then, to facilitate the generalization of local linearities, we construct a maximal margin geometry on the learned features by enforcing low-rank constraints on intra-class samples and high-rank constraints on inter-class samples, resulting in orthogonal subspaces for different classes and each subspace lies on a compact manifold.

Zero-Shot Learning

iComMa: Inverting 3D Gaussians Splatting for Camera Pose Estimation via Comparing and Matching

no code implementations14 Dec 2023 Yuan Sun, Xuan Wang, Yunfan Zhang, Jie Zhang, Caigui Jiang, Yu Guo, Fei Wang

By contrast, we model the pose estimation as the problem of inverting the 3D Gaussian Splatting (3DGS) with both the comparing and matching loss.

6D Pose Estimation

MaTe3D: Mask-guided Text-based 3D-aware Portrait Editing

no code implementations12 Dec 2023 Kangneng Zhou, Daiheng Gao, Xuan Wang, Jie Zhang, Peng Zhang, Xusen Sun, Longhao Zhang, Shiqi Yang, Bang Zhang, Liefeng Bo, Yaxing Wang

To address this limitation, we propose \textbf{MaTe3D}: mask-guided text-based 3D-aware portrait editing.

Calibration-free quantitative phase imaging in multi-core fiber endoscopes using end-to-end deep learning

no code implementations12 Dec 2023 Jiawei Sun, Bin Zhao, Dong Wang, Zhigang Wang, Jie Zhang, Nektarios Koukourakis, Juergen W. Czarske, Xuelong Li

Quantitative phase imaging (QPI) through multi-core fibers (MCFs) has been an emerging in vivo label-free endoscopic imaging modality with minimal invasiveness.

Retrieval

Control Risk for Potential Misuse of Artificial Intelligence in Science

1 code implementation11 Dec 2023 Jiyan He, Weitao Feng, Yaosen Min, Jingwei Yi, Kunsheng Tang, Shuai Li, Jie Zhang, Kejiang Chen, Wenbo Zhou, Xing Xie, Weiming Zhang, Nenghai Yu, Shuxin Zheng

In this study, we aim to raise awareness of the dangers of AI misuse in science, and call for responsible AI development and use in this domain.

Data-Free Hard-Label Robustness Stealing Attack

1 code implementation10 Dec 2023 Xiaojian Yuan, Kejiang Chen, Wen Huang, Jie Zhang, Weiming Zhang, Nenghai Yu

In response to these identified gaps, we introduce a novel Data-Free Hard-Label Robustness Stealing (DFHL-RS) attack in this paper, which enables the stealing of both model accuracy and robustness by simply querying hard labels of the target model without the help of any natural data.

Singular Regularization with Information Bottleneck Improves Model's Adversarial Robustness

no code implementations4 Dec 2023 Guanlin Li, Naishan Zheng, Man Zhou, Jie Zhang, Tianwei Zhang

However, these works lack analysis of adversarial information or perturbation, which cannot reveal the mystery of adversarial examples and lose proper interpretation.

Adversarial Robustness

FreePIH: Training-Free Painterly Image Harmonization with Diffusion Model

no code implementations25 Nov 2023 Ruibin Li, Jingcai Guo, Song Guo, Qihua Zhou, Jie Zhang

Specifically, we find that the very last few steps of the denoising (i. e., generation) process strongly correspond to the stylistic information of images, and based on this, we propose to augment the latent features of both the foreground and background images with Gaussians for a direct denoising-based harmonization.

Denoising Image Harmonization +1

Attribute-Aware Representation Rectification for Generalized Zero-Shot Learning

no code implementations23 Nov 2023 Zhijie Rao, Jingcai Guo, Xiaocheng Lu, Qihua Zhou, Jie Zhang, Kang Wei, Chenxin Li, Song Guo

In this paper, we propose a simple yet effective Attribute-Aware Representation Rectification framework for GZSL, dubbed $\mathbf{(AR)^{2}}$, to adaptively rectify the feature extractor to learn novel features while keeping original valuable features.

Attribute Generalized Zero-Shot Learning +1

Sparsity-Driven EEG Channel Selection for Brain-Assisted Speech Enhancement

no code implementations22 Nov 2023 Jie Zhang, Qing-Tian Xu, Zhen-Hua Ling

In this work, we therefore propose a novel end-to-end brain-assisted speech enhancement network (BASEN), which incorporates the listeners' EEG signals and adopts a temporal convolutional network together with a convolutional multi-layer cross attention module to fuse EEG-audio features.

EEG Electroencephalogram (EEG) +1

Improving Adversarial Transferability by Stable Diffusion

no code implementations18 Nov 2023 Jiayang Liu, Siyu Zhu, Siyuan Liang, Jie Zhang, Han Fang, Weiming Zhang, Ee-Chien Chang

Various techniques have emerged to enhance the transferability of adversarial attacks for the black-box scenario.

Segue: Side-information Guided Generative Unlearnable Examples for Facial Privacy Protection in Real World

no code implementations24 Oct 2023 Zhiling Zhang, Jie Zhang, Kui Zhang, Wenbo Zhou, Weiming Zhang, Nenghai Yu

To address these concerns, researchers are actively exploring the concept of ``unlearnable examples", by adding imperceptible perturbation to data in the model training stage, which aims to prevent the model from learning discriminate features of the target face.

Face Recognition

Pix2HDR -- A pixel-wise acquisition and deep learning-based synthesis approach for high-speed HDR videos

no code implementations24 Oct 2023 Caixin Wang, Jie Zhang, Matthew A. Wilson, Ralph Etienne-Cummings

By combining the versatility of pixel-wise sampling patterns with the strength of deep neural networks at decoding complex scenes, our method greatly enhances the vision system's adaptability and performance in dynamic conditions.

IntentDial: An Intent Graph based Multi-Turn Dialogue System with Reasoning Path Visualization

no code implementations18 Oct 2023 Zengguang Hao, Jie Zhang, Binxia Xu, Yafang Wang, Gerard de Melo, Xiaolong Li

Intent detection and identification from multi-turn dialogue has become a widely explored technique in conversational agents, for example, voice assistants and intelligent customer services.

Intent Detection

Real-Fake: Effective Training Data Synthesis Through Distribution Matching

1 code implementation16 Oct 2023 Jianhao Yuan, Jie Zhang, Shuyang Sun, Philip Torr, Bo Zhao

Synthetic training data has gained prominence in numerous learning tasks and scenarios, offering advantages such as dataset augmentation, generalization evaluation, and privacy preservation.

Image Classification Out-of-Distribution Generalization

Multiview Transformer: Rethinking Spatial Information in Hyperspectral Image Classification

no code implementations11 Oct 2023 Jie Zhang, Yongshan Zhang, Yicong Zhou

To aggregate the multiview information, a fully-convolutional SED with a U-shape in spectral dimension is introduced to extract a multiview feature map.

Classification Dimensionality Reduction +1

Latent Diffusion Model for Medical Image Standardization and Enhancement

no code implementations8 Oct 2023 Md Selim, Jie Zhang, Faraneh Fathi, Michael A. Brooks, Ge Wang, Guoqiang Yu, Jin Chen

Finally, the decoder uses the transformed latent representation to generate a standardized CT image, providing a more consistent basis for downstream analysis.

Computed Tomography (CT)

Cluster-based Method for Eavesdropping Identification and Localization in Optical Links

no code implementations25 Sep 2023 Haokun Song, Rui Lin, Andrea Sgambelluri, Filippo Cugini, Yajie Li, Jie Zhang, Paolo Monti

We propose a cluster-based method to detect and locate eavesdropping events in optical line systems characterized by small power losses.

Understanding Data Augmentation from a Robustness Perspective

no code implementations7 Sep 2023 Zhendong Liu, Jie Zhang, Qiangqiang He, Chongjun Wang

In the realm of visual recognition, data augmentation stands out as a pivotal technique to amplify model robustness.

Data Augmentation

Bias Testing and Mitigation in LLM-based Code Generation

no code implementations3 Sep 2023 Dong Huang, Qingwen Bu, Jie Zhang, Xiaofei Xie, Junjie Chen, Heming Cui

To mitigate bias for code generation models, we evaluate five bias mitigation prompt strategies, i. e., utilizing bias testing results to refine the code (zero-shot), one-, few-shot, and two Chain-of-Thought (CoT) prompts.

Code Generation Fairness +1

Rep2wav: Noise Robust text-to-speech Using self-supervised representations

no code implementations28 Aug 2023 Qiushi Zhu, Yu Gu, Rilin Chen, Chao Weng, Yuchen Hu, LiRong Dai, Jie Zhang

Noise-robust TTS models are often trained using the enhanced speech, which thus suffer from speech distortion and background noise that affect the quality of the synthesized speech.

Speech Enhancement

Backdooring Textual Inversion for Concept Censorship

no code implementations21 Aug 2023 Yutong Wu, Jie Zhang, Florian Kerschbaum, Tianwei Zhang

Users can easily download the word embedding from public websites like Civitai and add it to their own stable diffusion model without fine-tuning for personalization.

Patch Is Not All You Need

no code implementations21 Aug 2023 Changzhen Li, Jie Zhang, Yang Wei, Zhilong Ji, Jinfeng Bai, Shiguang Shan

Vision Transformers have achieved great success in computer visions, delivering exceptional performance across various tasks.

Semantic-Human: Neural Rendering of Humans from Monocular Video with Human Parsing

no code implementations19 Aug 2023 Jie Zhang, Pengcheng Shi, Zaiwang Gu, Yiyang Zhou, Zhi Wang

In this paper, we present Semantic-Human, a novel method that achieves both photorealistic details and viewpoint-consistent human parsing for the neural rendering of humans.

Denoising Human Parsing +2

Overlap Bias Matching is Necessary for Point Cloud Registration

no code implementations18 Aug 2023 Pengcheng Shi, Jie Zhang, Haozhe Cheng, Junyang Wang, Yiyang Zhou, Chenlin Zhao, Jihua Zhu

Specifically, we propose a plug-and-play Overlap Bias Matching Module (OBMM) comprising two integral components, overlap sampling module and bias prediction module.

Point Cloud Registration

Mobile Supply: The Last Piece of Jigsaw of Recommender System

no code implementations7 Aug 2023 Zhenhao Jiang, Biao Zeng, Hao Feng, Jin Liu, Jie Zhang, Jia Jia, Ning Hu

In order to address the problem of pagination trigger mechanism, we propose a completely new module in the pipeline of recommender system named Mobile Supply.

Recommendation Systems Re-Ranking

Sampling to Distill: Knowledge Transfer from Open-World Data

no code implementations31 Jul 2023 Yuzheng Wang, Zhaoyu Chen, Jie Zhang, Dingkang Yang, Zuhao Ge, Yang Liu, Siao Liu, Yunquan Sun, Wenqiang Zhang, Lizhe Qi

Then, we introduce a low-noise representation to alleviate the domain shifts and build a structured relationship of multiple data examples to exploit data knowledge.

Data-free Knowledge Distillation Transfer Learning

Rethinking Data Distillation: Do Not Overlook Calibration

1 code implementation ICCV 2023 Dongyao Zhu, Bowen Lei, Jie Zhang, Yanbo Fang, Ruqi Zhang, Yiqun Xie, Dongkuan Xu

Neural networks trained on distilled data often produce over-confident output and require correction by calibration methods.

Enhancing Job Recommendation through LLM-based Generative Adversarial Networks

no code implementations20 Jul 2023 Yingpeng Du, Di Luo, Rui Yan, Hongzhi Liu, Yang song, HengShu Zhu, Jie Zhang

However, directly leveraging LLMs to enhance recommendation results is not a one-size-fits-all solution, as LLMs may suffer from fabricated generation and few-shot problems, which degrade the quality of resume completion.

Our Model Achieves Excellent Performance on MovieLens: What Does it Mean?

1 code implementation19 Jul 2023 Yu-chen Fan, Yitong Ji, Jie Zhang, Aixin Sun

In this study, we conduct a meticulous analysis on the MovieLens dataset and explain the potential impact on using the dataset for evaluating recommendation algorithms.

Recommendation Systems

ESMC: Entire Space Multi-Task Model for Post-Click Conversion Rate via Parameter Constraint

no code implementations18 Jul 2023 Zhenhao Jiang, Biao Zeng, Hao Feng, Jin Liu, Jicong Fan, Jie Zhang, Jia Jia, Ning Hu, Xingyu Chen, Xuguang Lan

We propose a novel Entire Space Multi-Task Model for Post-Click Conversion Rate via Parameter Constraint (ESMC) and two alternatives: Entire Space Multi-Task Model with Siamese Network (ESMS) and Entire Space Multi-Task Model in Global Domain (ESMG) to address the PSC issue.

Decision Making Recommendation Systems +1

Learning Subjective Time-Series Data via Utopia Label Distribution Approximation

no code implementations15 Jul 2023 Wenxin Xu, Hexin Jiang, Xuefeng Liang, Ying Zhou, Yin Zhao, Jie Zhang

In this work, we propose Utopia Label Distribution Approximation (ULDA) for time-series data, which makes the training label distribution closer to real-world but unknown (utopia) label distribution.

Age Estimation Depth Estimation +4

Using electrical impedance spectroscopy to identify equivalent circuit models of lubricated contacts with complex geometry: in-situ application to mini traction machine

no code implementations7 Jul 2023 Min Yu, Jie Zhang, Arndt Joedicke, Tom Reddyhoff

Overall, the proposed method is applicable to general lubricated interfaces for the identification of equivalent circuit models, which in turn facilitates in-situ tribo-contacts with electric impedance measurement of oil film thickness.

That's BAD: Blind Anomaly Detection by Implicit Local Feature Clustering

no code implementations6 Jul 2023 Jie Zhang, Masanori Suganuma, Takayuki Okatani

They consider an unsupervised setting, specifically the one-class setting, in which we assume the availability of a set of normal (\textit{i. e.}, anomaly-free) images for training.

Anomaly Detection Clustering +1

Contextual Affinity Distillation for Image Anomaly Detection

no code implementations6 Jul 2023 Jie Zhang, Masanori Suganuma, Takayuki Okatani

The local student, which is used in previous studies mainly focuses on structural anomaly detection while the global student pays attention to logical anomalies.

Anomaly Detection Knowledge Distillation

Federated Generative Learning with Foundation Models

1 code implementation28 Jun 2023 Jie Zhang, Xiaohua Qi, Bo Zhao

Existing federated learning solutions focus on transmitting features, parameters or gadients between clients and server, which suffer from serious low-efficiency and privacy-leakage problems.

Federated Learning

Phonon dynamic behaviors induced by amorphous interlayer at heterointerfaces

no code implementations8 Jun 2023 Quanjie Wang, Jie Zhang, Vladimir Chernysh, Xiangjun Liu

However, due to mode conversion and inelastic scattering, we found a portion of high-frequency TA phonons, which are higher than the cut-off frequency and cannot transmit across the ideal sharp interface, can partially transmit across the amorphous interlayer, which introduces additional thermal transport channels through the interface and has positive effect on interfacial thermal conductance.

On Knowledge Editing in Federated Learning: Perspectives, Challenges, and Future Directions

no code implementations2 Jun 2023 Leijie Wu, Song Guo, Junxiao Wang, Zicong Hong, Jie Zhang, Jingren Zhou

As Federated Learning (FL) has gained increasing attention, it has become widely acknowledged that straightforwardly applying stochastic gradient descent (SGD) on the overall framework when learning over a sequence of tasks results in the phenomenon known as ``catastrophic forgetting''.

Federated Learning knowledge editing

Dissecting Arbitrary-scale Super-resolution Capability from Pre-trained Diffusion Generative Models

no code implementations1 Jun 2023 Ruibin Li, Qihua Zhou, Song Guo, Jie Zhang, Jingcai Guo, Xinyang Jiang, Yifei Shen, Zhenhua Han

Diffusion-based Generative Models (DGMs) have achieved unparalleled performance in synthesizing high-quality visual content, opening up the opportunity to improve image super-resolution (SR) tasks.

Image Super-Resolution

Towards Omni-generalizable Neural Methods for Vehicle Routing Problems

1 code implementation31 May 2023 Jianan Zhou, Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang

Learning heuristics for vehicle routing problems (VRPs) has gained much attention due to the less reliance on hand-crafted rules.

Combinatorial Optimization Meta-Learning +1

Towards Efficient Deep Hashing Retrieval: Condensing Your Data via Feature-Embedding Matching

1 code implementation29 May 2023 Tao Feng, Jie Zhang, Peizheng Wang, Zhijie Wang

The expenses involved in training state-of-the-art deep hashing retrieval models have witnessed an increase due to the adoption of more sophisticated models and large-scale datasets.

Dataset Condensation Deep Hashing

Eeg2vec: Self-Supervised Electroencephalographic Representation Learning

no code implementations23 May 2023 Qiushi Zhu, Xiaoying Zhao, Jie Zhang, Yu Gu, Chao Weng, Yuchen Hu

Recently, many efforts have been made to explore how the brain processes speech using electroencephalographic (EEG) signals, where deep learning-based approaches were shown to be applicable in this field.

EEG Representation Learning

Layer-adaptive Structured Pruning Guided by Latency

no code implementations23 May 2023 Siyuan Pan, Linna Zhang, Jie Zhang, Xiaoshuang Li, Liang Hou, Xiaobing Tu

Structured pruning can simplify network architecture and improve inference speed.

Network Pruning

CASA-ASR: Context-Aware Speaker-Attributed ASR

no code implementations21 May 2023 Mohan Shi, Zhihao Du, Qian Chen, Fan Yu, Yangze Li, Shiliang Zhang, Jie Zhang, Li-Rong Dai

In addition, a two-pass decoding strategy is further proposed to fully leverage the contextual modeling ability resulting in a better recognition performance.

Automatic Speech Recognition speech-recognition +1

Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models

no code implementations18 May 2023 Yihao Huang, Felix Juefei-Xu, Qing Guo, Jie Zhang, Yutong Wu, Ming Hu, Tianlin Li, Geguang Pu, Yang Liu

Although recent personalization methods have democratized high-resolution image synthesis by enabling swift concept acquisition with minimal examples and lightweight computation, they also present an exploitable avenue for high accessible backdoor attacks.

Backdoor Attack Image Generation

BASEN: Time-Domain Brain-Assisted Speech Enhancement Network with Convolutional Cross Attention in Multi-talker Conditions

1 code implementation17 May 2023 Jie Zhang, Qing-Tian Xu, Qiu-Shi Zhu, Zhen-Hua Ling

In this paper, we thus propose a novel time-domain brain-assisted SE network (BASEN) incorporating electroencephalography (EEG) signals recorded from the listener for extracting the target speaker from monaural speech mixtures.

EEG Speech Enhancement

Watermarking Text Generated by Black-Box Language Models

1 code implementation14 May 2023 Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, Nenghai Yu

To allow third-parties to autonomously inject watermarks into generated text, we develop a watermarking framework for black-box language model usage scenarios.

Adversarial Robustness Language Modelling +2

A Black-Box Attack on Code Models via Representation Nearest Neighbor Search

no code implementations10 May 2023 Jie Zhang, Wei Ma, Qiang Hu, Shangqing Liu, Xiaofei Xie, Yves Le Traon, Yang Liu

Furthermore, the perturbation of adversarial examples introduced by RNNS is smaller compared to the baselines in terms of the number of replaced variables and the change in variable length.

Adversarial Attack Clone Detection

Retraining A Graph-based Recommender with Interests Disentanglement

no code implementations5 May 2023 Yitong Ji, Aixin Sun, Jie Zhang

Then we blend the historical and new preferences in the form of node embeddings in the new graph, through a Disentanglement Module.

Disentanglement Incremental Learning +2

Towards Unbiased Training in Federated Open-world Semi-supervised Learning

no code implementations1 May 2023 Jie Zhang, Xiaosong Ma, Song Guo, Wenchao Xu

Federated Semi-supervised Learning (FedSSL) has emerged as a new paradigm for allowing distributed clients to collaboratively train a machine learning model over scarce labeled data and abundant unlabeled data.

Open-World Semi-Supervised Learning Transfer Learning

Multi-Sample Consensus Driven Unsupervised Normal Estimation for 3D Point Clouds

no code implementations10 Apr 2023 Jie Zhang, Minghui Nie, Junjie Cao, Jian Liu, Ligang Liu

Comprehensive experiments demonstrate that the two proposed unsupervised methods are noticeably superior to some supervised deep normal estimators on the most common synthetic dataset.

Real Face Foundation Representation Learning for Generalized Deepfake Detection

no code implementations15 Mar 2023 Liang Shi, Jie Zhang, Shiguang Shan

In this study, we propose Real Face Foundation Representation Learning (RFFR), which aims to learn a general representation from large-scale real face datasets and detect potential artifacts outside the distribution of RFFR.

DeepFake Detection Face Swapping +1

TARGET: Federated Class-Continual Learning via Exemplar-Free Distillation

1 code implementation ICCV 2023 Jie Zhang, Chen Chen, Weiming Zhuang, LingJuan Lv

This paper focuses on an under-explored yet important problem: Federated Class-Continual Learning (FCCL), where new classes are dynamically added in federated learning.

Continual Learning Federated Learning

Neural Airport Ground Handling

1 code implementation4 Mar 2023 Yaoxin Wu, Jianan Zhou, Yunwen Xia, Xianli Zhang, Zhiguang Cao, Jie Zhang

Airport ground handling (AGH) offers necessary operations to flights during their turnarounds and is of great importance to the efficiency of airport management and the economics of aviation.

Combinatorial Optimization Reinforcement Learning (RL) +1

Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network

1 code implementation20 Feb 2023 Xiaojian Yuan, Kejiang Chen, Jie Zhang, Weiming Zhang, Nenghai Yu, Yang Zhang

At first, a top-n selection strategy is proposed to provide pseudo-labels for public data, and use pseudo-labels to guide the training of the cGAN.

Generative Adversarial Network Pseudo Label

Delving into the Adversarial Robustness of Federated Learning

no code implementations19 Feb 2023 Jie Zhang, Bo Li, Chen Chen, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chao Wu

In this work, we propose a novel algorithm called Decision Boundary based Federated Adversarial Training (DBFAT), which consists of two components (local re-weighting and global regularization) to improve both accuracy and robustness of FL systems.

Adversarial Robustness Federated Learning

Speech Enhancement with Multi-granularity Vector Quantization

no code implementations16 Feb 2023 Xiao-Ying Zhao, Qiu-Shi Zhu, Jie Zhang

With advances in deep learning, neural network based speech enhancement (SE) has developed rapidly in the last decade.

Denoising Quantization +2

Towards Fairer and More Efficient Federated Learning via Multidimensional Personalized Edge Models

no code implementations9 Feb 2023 Yingchun Wang, Jingcai Guo, Jie Zhang, Song Guo, Weizhan Zhang, Qinghua Zheng

Federated learning (FL) is an emerging technique that trains massive and geographically distributed edge data while maintaining privacy.

Computational Efficiency Fairness +1

TAP: Accelerating Large-Scale DNN Training Through Tensor Automatic Parallelisation

no code implementations1 Feb 2023 Ziji Shi, Le Jiang, Ang Wang, Jie Zhang, Xianyan Jia, Yong Li, Chencan Wu, Jialin Li, Wei Lin

However, finding a suitable model parallel schedule for an arbitrary neural network is a non-trivial task due to the exploding search space.

DiffusionCT: Latent Diffusion Model for CT Image Standardization

no code implementations20 Jan 2023 Md Selim, Jie Zhang, Michael A. Brooks, Ge Wang, Jin Chen

This work addresses the issue of CT image harmonization using a new diffusion-based model, named DiffusionCT, to standardize CT images acquired from different vendors and protocols.

Computed Tomography (CT) Image Harmonization +1

Quantum-Inspired Spectral-Spatial Pyramid Network for Hyperspectral Image Classification

no code implementations CVPR 2023 Jie Zhang, Yongshan Zhang, Yicong Zhou

Using QSSN as the building block, we propose an end-to-end quantum-inspired spectral-spatial pyramid network (QSSPN) for HSI feature extraction and classification.

Hyperspectral Image Classification

Pyramid Dual Domain Injection Network for Pan-sharpening

no code implementations ICCV 2023 Xuanhua He, Keyu Yan, Rui Li, Chengjun Xie, Jie Zhang, Man Zhou

To this end, we first revisit the degradation process of pan-sharpening in Fourier space, and then devise a Pyramid Dual Domain Injection Pan-sharpening Network upon the above observation by fully exploring and exploiting the distinguished information in both the spatial and frequency domains.

Spectral Super-Resolution Super-Resolution

Are Code Pre-trained Models Powerful to Learn Code Syntax and Semantics?

no code implementations20 Dec 2022 Wei Ma, Mengjie Zhao, Xiaofei Xie, Qiang Hu, Shangqing Liu, Jie Zhang, Wenhan Wang, Yang Liu

To further understand the code features learnt by these models, in this paper, we target two well-known representative code pre-trained models (i. e., CodeBERT and GraphCodeBERT) and devise a set of probing tasks for the syntax and semantics analysis.

Code Completion Code Search +2

Exploring Optimal Substructure for Out-of-distribution Generalization via Feature-targeted Model Pruning

no code implementations19 Dec 2022 Yingchun Wang, Jingcai Guo, Song Guo, Weizhan Zhang, Jie Zhang

Recent studies show that even highly biased dense networks contain an unbiased substructure that can achieve better out-of-distribution (OOD) generalization than the original model.

Out-of-Distribution Generalization

Accelerating Dataset Distillation via Model Augmentation

2 code implementations CVPR 2023 Lei Zhang, Jie Zhang, Bowen Lei, Subhabrata Mukherjee, Xiang Pan, Bo Zhao, Caiwen Ding, Yao Li, Dongkuan Xu

Dataset Distillation (DD), a newly emerging field, aims at generating much smaller but efficient synthetic training datasets from large ones.

Efficient Stein Variational Inference for Reliable Distribution-lossless Network Pruning

no code implementations7 Dec 2022 Yingchun Wang, Song Guo, Jingcai Guo, Weizhan Zhang, Yida Xu, Jie Zhang, Yi Liu

Extensive experiments based on small Cifar-10 and large-scaled ImageNet demonstrate that our method can obtain sparser networks with great generalization performance while providing quantified reliability for the pruned model.

Network Pruning Variational Inference

Ada3Diff: Defending against 3D Adversarial Point Clouds via Adaptive Diffusion

no code implementations29 Nov 2022 Kui Zhang, Hang Zhou, Jie Zhang, Qidong Huang, Weiming Zhang, Nenghai Yu

Deep 3D point cloud models are sensitive to adversarial attacks, which poses threats to safety-critical applications such as autonomous driving.

Autonomous Driving Denoising

Decision-making with Speculative Opponent Models

no code implementations22 Nov 2022 Jing Sun, Shuo Chen, Cong Zhang, Jie Zhang

Specifically, the actor maintains a speculated belief of the opponents, which we call the speculative opponent models, to predict opponent actions using local observations and makes decisions accordingly.

Decision Making

Boosting Personalised Musculoskeletal Modelling with Physics-informed Knowledge Transfer

no code implementations22 Nov 2022 Jie Zhang, Yihui Zhao, Tianzhe Bao, Zhenhong Li, Kun Qian, Alejandro F. Frangi, Sheng Quan Xie, Zhi-Qiang Zhang

The salient advantages of the proposed framework are twofold: 1) For the generic model, physics-based domain knowledge is embedded into the loss function of the data-driven model as soft constraints to penalise/regularise the data-driven model.

Transfer Learning

Exploiting Personalized Invariance for Better Out-of-distribution Generalization in Federated Learning

no code implementations21 Nov 2022 Xueyang Tang, Song Guo, Jie Zhang

Recently, data heterogeneity among the training datasets on the local clients (a. k. a., Non-IID data) has attracted intense interest in Federated Learning (FL), and many personalized federated learning methods have been proposed to handle it.

Out-of-Distribution Generalization Personalized Federated Learning

VATLM: Visual-Audio-Text Pre-Training with Unified Masked Prediction for Speech Representation Learning

no code implementations21 Nov 2022 Qiushi Zhu, Long Zhou, Ziqiang Zhang, Shujie Liu, Binxing Jiao, Jie Zhang, LiRong Dai, Daxin Jiang, Jinyu Li, Furu Wei

Although speech is a simple and effective way for humans to communicate with the outside world, a more realistic speech interaction contains multimodal information, e. g., vision, text.

Audio-Visual Speech Recognition Language Modelling +3

Deep Reinforcement Learning Guided Improvement Heuristic for Job Shop Scheduling

1 code implementation20 Nov 2022 Cong Zhang, Zhiguang Cao, Wen Song, Yaoxin Wu, Jie Zhang

Recent studies in using deep reinforcement learning (DRL) to solve Job-shop scheduling problems (JSSP) focus on construction heuristics.

Job Shop Scheduling reinforcement-learning +2

FedTune: A Deep Dive into Efficient Federated Fine-Tuning with Pre-trained Transformers

no code implementations15 Nov 2022 Jinyu Chen, Wenchao Xu, Song Guo, Junxiao Wang, Jie Zhang, Haozhao Wang

Federated Learning (FL) is an emerging paradigm that enables distributed users to collaboratively and iteratively train machine learning models without sharing their private data.

Federated Learning Language Modelling +1

Feature Correlation-guided Knowledge Transfer for Federated Self-supervised Learning

no code implementations14 Nov 2022 Yi Liu, Song Guo, Jie Zhang, Qihua Zhou, Yingchun Wang, Xiaohan Zhao

We prove that FedFoA is a model-agnostic training framework and can be easily compatible with state-of-the-art unsupervised FL methods.

Feature Correlation Federated Learning +4

Reinforcement Learning Enhanced Weighted Sampling for Accurate Subgraph Counting on Fully Dynamic Graph Streams

1 code implementation13 Nov 2022 Kaixin Wang, Cheng Long, Da Yan, Jie Zhang, H. V. Jagadish

Specifically, we propose a weighted sampling algorithm called WSD for estimating the subgraph count in a fully dynamic graph stream, which samples the edges based on their weights that indicate their importance and reflect their properties.

Subgraph Counting

Demystify Self-Attention in Vision Transformers from a Semantic Perspective: Analysis and Application

no code implementations13 Nov 2022 Leijie Wu, Song Guo, Yaohong Ding, Junxiao Wang, Wenchao Xu, Richard Yida Xu, Jie Zhang

In contrast, visual data exhibits a fundamentally different structure: Its basic unit (pixel) is a natural low-level representation with significant redundancies in the neighbourhood, which poses obvious challenges to the interpretability of MSA mechanism in ViT.

Robust Data2vec: Noise-robust Speech Representation Learning for ASR by Combining Regression and Improved Contrastive Learning

1 code implementation27 Oct 2022 Qiu-Shi Zhu, Long Zhou, Jie Zhang, Shu-Jie Liu, Yu-Chen Hu, Li-Rong Dai

Self-supervised pre-training methods based on contrastive learning or regression tasks can utilize more unlabeled data to improve the performance of automatic speech recognition (ASR).

Automatic Speech Recognition Automatic Speech Recognition (ASR) +4

Speech Enhancement Using Self-Supervised Pre-Trained Model and Vector Quantization

no code implementations28 Sep 2022 Xiao-Ying Zhao, Qiu-Shi Zhu, Jie Zhang

Specifically, the encoder and bottleneck layer of the DEMUCS model are initialized using the self-supervised pretrained WavLM model, the convolution in the encoder is replaced by causal convolution, and the transformer encoder in the bottleneck layer is based on causal attention mask.

Denoising Quantization +1

Learning to Solve Multiple-TSP with Time Window and Rejections via Deep Reinforcement Learning

1 code implementation13 Sep 2022 Rongkai Zhang, Cong Zhang, Zhiguang Cao, Wen Song, Puay Siew Tan, Jie Zhang, Bihan Wen, Justin Dauwels

We propose a manager-worker framework based on deep reinforcement learning to tackle a hard yet nontrivial variant of Travelling Salesman Problem (TSP), \ie~multiple-vehicle TSP with time window and rejections (mTSPTWR), where customers who cannot be served before the deadline are subject to rejections.

Class-Incremental Learning via Knowledge Amalgamation

1 code implementation5 Sep 2022 Marcus de Carvalho, Mahardhika Pratama, Jie Zhang, Yajuan San

Catastrophic forgetting has been a significant problem hindering the deployment of deep learning algorithms in the continual learning setting.

Class Incremental Learning Incremental Learning

A Multi-Channel Next POI Recommendation Framework with Multi-Granularity Check-in Signals

1 code implementation1 Sep 2022 Zhu Sun, Yu Lei, Lu Zhang, Chen Li, Yew-Soon Ong, Jie Zhang

Being equipped with three modules (i. e., global user behavior encoder, local multi-channel encoder, and region-aware weighting strategy), MCMG is capable of capturing both fine- and coarse-grained sequential regularities as well as exploring the dynamic impact of multi-channel by differentiating the region check-in patterns.

Federated Learning with Label Distribution Skew via Logits Calibration

2 code implementations1 Sep 2022 Jie Zhang, Zhiqi Li, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Chao Wu

Extensive experiments on federated datasets and real-world datasets demonstrate that FedLC leads to a more accurate global model and much improved performance.

Federated Learning

Fed-FSNet: Mitigating Non-I.I.D. Federated Learning via Fuzzy Synthesizing Network

no code implementations21 Aug 2022 Jingcai Guo, Song Guo, Jie Zhang, Ziming Liu

Concretely, we maintain an edge-agnostic hidden model in the cloud server to estimate a less-accurate while direction-aware inversion of the global model.

Federated Learning Privacy Preserving

Hierarchical Compositional Representations for Few-shot Action Recognition

no code implementations19 Aug 2022 Changzhen Li, Jie Zhang, Shuzhe Wu, Xin Jin, Shiguang Shan

Recently action recognition has received more and more attention for its comprehensive and practical applications in intelligent surveillance and human-computer interaction.

Few-Shot action recognition Few Shot Action Recognition

Multi-modal Transformer Path Prediction for Autonomous Vehicle

no code implementations15 Aug 2022 Chia Hong Tseng, Jie Zhang, Min-Te Sun, Kazuya Sakai, Wei-Shinn Ku

To better utilize the lane information, the lanes which are in opposite direction to target agent are not likely to be taken by the target agent and are consequently filtered out.

Autonomous Driving Trajectory Forecasting

Microwave QR Code: An IRS-Based Solution

no code implementations5 Aug 2022 Sai Xu, Yanan Du, Jiliang Zhang, Jie Zhang

This letter proposes to employ intelligent reflecting surface (IRS) as an information media to display a microwave quick response (QR) code for Internet-of-Things applications.

An IRS Backscatter Enabled Integrated Sensing, Communication and Computation System

no code implementations20 Jul 2022 Sai Xu, Yanan Du, Jiliang Zhang, Jiangzhou Wang, Jie Zhang

This paper proposes to leverage intelligent reflecting surface (IRS) backscatter to realize radio-frequency-chain-free uplink-transmissions (RFCF-UT).

On the Performance of Data Compression in Clustered Fog Radio Access Networks

no code implementations1 Jul 2022 Haonan Hu, Yan Jiang, Jiliang Zhang, Yanan Zheng, Qianbin Chen, Jie Zhang

The fog-radio-access-network (F-RAN) has been proposed to address the strict latency requirements, which offloads computation tasks generated in user equipments (UEs) to the edge to reduce the processing latency.

Data Compression

DaisyRec 2.0: Benchmarking Recommendation for Rigorous Evaluation

2 code implementations22 Jun 2022 Zhu Sun, Hui Fang, Jie Yang, Xinghua Qu, Hongyang Liu, Di Yu, Yew-Soon Ong, Jie Zhang

Recently, one critical issue looms large in the field of recommender systems -- there are no effective benchmarks for rigorous evaluation -- which consequently leads to unreproducible evaluation and unfair comparison.

Benchmarking Recommendation Systems

Sampling Efficient Deep Reinforcement Learning through Preference-Guided Stochastic Exploration

1 code implementation20 Jun 2022 Wenhui Huang, Cong Zhang, Jingda Wu, Xiangkun He, Jie Zhang, Chen Lv

We theoretically prove that the policy improvement theorem holds for the preference-guided $\epsilon$-greedy policy and experimentally show that the inferred action preference distribution aligns with the landscape of corresponding Q-values.

Atari Games Q-Learning +2

Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASR

no code implementations26 May 2022 Qiu-Shi Zhu, Jie Zhang, Zi-Qiang Zhang, Li-Rong Dai

Speech enhancement (SE) is usually required as a front end to improve the speech quality in noisy environments, while the enhanced speech might not be optimal for automatic speech recognition (ASR) systems due to speech distortion.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

IDEAL: Query-Efficient Data-Free Learning from Black-box Models

1 code implementation23 May 2022 Jie Zhang, Chen Chen, Lingjuan Lyu

Knowledge Distillation (KD) is a typical method for training a lightweight student model with the help of a well-trained teacher model.

Knowledge Distillation

Snake net and balloon force with a neural network for detecting multiple phases

no code implementations19 May 2022 Xiaodong Sun, Huijiong Yang, Nan Wu, T. C. Scott, Jie Zhang, Wanzhou Zhang

In order to obtain a physical phase diagram, the snake model with an artificial neural network is applied in an unsupervised learning way by the authors of [Phys. Rev. Lett.

Boosting Pruned Networks with Linear Over-parameterization

no code implementations25 Apr 2022 Yu Qian, Jian Cao, Xiaoshuang Li, Jie Zhang, Hufei Li, Jue Chen

To address this challenge, we propose a novel method that first linearly over-parameterizes the compact layers in pruned networks to enlarge the number of fine-tuning parameters and then re-parameterizes them to the original layers after fine-tuning.

Knowledge Distillation

Sign Bit is Enough: A Learning Synchronization Framework for Multi-hop All-reduce with Ultimate Compression

no code implementations14 Apr 2022 Feijie Wu, Shiqi He, Song Guo, Zhihao Qu, Haozhao Wang, Weihua Zhuang, Jie Zhang

Traditional one-bit compressed stochastic gradient descent can not be directly employed in multi-hop all-reduce, a widely adopted distributed training paradigm in network-intensive high-performance computing systems such as public clouds.

On Scheduling Mechanisms Beyond the Worst Case

no code implementations14 Apr 2022 Yansong Gao, Jie Zhang

That is, mechanism K is pointwise better than mechanism P. Next, for each task $j$, when machines' execution costs $t_i^j$ are independent and identically drawn from a task-specific distribution $F^j(t)$, we show that the average-case approximation ratio of mechanism K converges to a constant.

Scheduling

Adaptive Modulation for Wobbling UAV Air-to-Ground Links in Millimeter-wave Bands

no code implementations13 Apr 2022 Songjiang Yang, Zitian Zhang, Jiliang Zhang, Xiaoli Chu, Jie Zhang

Based on the designed detectors, we propose an adaptive modulation scheme to maximize the average transmission rate under imperfect CSI by optimizing the data transmission time subject to the maximum tolerable BEP.

Do Loyal Users Enjoy Better Recommendations? Understanding Recommender Accuracy from a Time Perspective

1 code implementation12 Apr 2022 Yitong Ji, Aixin Sun, Jie Zhang, Chenliang Li

Our study offers a different perspective to understand recommender accuracy, and our findings could trigger a revisit of recommender model design.

Recommendation Systems

PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systems

1 code implementation11 Apr 2022 Yuanxing Zhang, Langshi Chen, Siran Yang, Man Yuan, Huimin Yi, Jie Zhang, Jiamang Wang, Jianbo Dong, Yunlong Xu, Yue Song, Yong Li, Di Zhang, Wei Lin, Lin Qu, Bo Zheng

However, we observe that GPU devices in training recommender systems are underutilized, and they cannot attain an expected throughput improvement as what it has achieved in CV and NLP areas.

Marketing Recommendation Systems

A Complementary Joint Training Approach Using Unpaired Speech and Text for Low-Resource Automatic Speech Recognition

no code implementations5 Apr 2022 Ye-Qian Du, Jie Zhang, Qiu-Shi Zhu, Li-Rong Dai, Ming-Hui Wu, Xin Fang, Zhou-Wang Yang

Unpaired data has shown to be beneficial for low-resource automatic speech recognition~(ASR), which can be involved in the design of hybrid models with multi-task training or language model dependent pre-training.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

Spectral Graph Clustering for Intentional Islanding Operations in Resilient Hybrid Energy Systems

no code implementations13 Mar 2022 Jiaxin Wu, Xin Chen, Sobhan Badakhshan, Jie Zhang, Pingfeng Wang

Establishing cleaner energy generation therefore improving the sustainability of the power system is a crucial task in this century, and one of the key strategies being pursued is to shift the dependence on fossil fuel to renewable technologies such as wind, solar, and nuclear.

Clustering Graph Clustering +1

Learning Contextually Fused Audio-visual Representations for Audio-visual Speech Recognition

no code implementations15 Feb 2022 Zi-Qiang Zhang, Jie Zhang, Jian-Shu Zhang, Ming-Hui Wu, Xin Fang, Li-Rong Dai

The proposed approach explores both the complementarity of audio-visual modalities and long-term context dependency using a transformer-based fusion module and a flexible masking strategy.

Audio-Visual Speech Recognition Lipreading +4

Learning to Solve Routing Problems via Distributionally Robust Optimization

1 code implementation15 Feb 2022 Yuan Jiang, Yaoxin Wu, Zhiguang Cao, Jie Zhang

Recent deep models for solving routing problems always assume a single distribution of nodes for training, which severely impairs their cross-distribution generalization ability.

A Coalition Formation Game Approach for Personalized Federated Learning

no code implementations5 Feb 2022 Leijie Wu, Song Guo, Yaohong Ding, Yufeng Zhan, Jie Zhang

Facing the challenge of statistical diversity in client local data distribution, personalized federated learning (PFL) has become a growing research hotspot.

Personalized Federated Learning

Adversarial Examples for Good: Adversarial Examples Guided Imbalanced Learning

1 code implementation28 Jan 2022 Jie Zhang, Lei Zhang, Gang Li, Chao Wu

Adversarial examples are inputs for machine learning models that have been designed by attackers to cause the model to make mistakes.

Supervised and Self-supervised Pretraining Based COVID-19 Detection Using Acoustic Breathing/Cough/Speech Signals

no code implementations22 Jan 2022 Xing-Yu Chen, Qiu-Shi Zhu, Jie Zhang, Li-Rong Dai

By using the acoustic signals to train the network, respectively, we can build individual models for three tasks, whose parameters are averaged to obtain an average model, which is then used as the initialization for the BiLSTM model training of each task.

Towards Efficient Data Free Black-Box Adversarial Attack

1 code implementation CVPR 2022 Jie Zhang, Bo Li, Jianghe Xu, Shuang Wu, Shouhong Ding, Lei Zhang, Chao Wu

The proposed method can efficiently imitate the target model through a small number of queries and achieve high attack success rate.

Adversarial Attack

Enhancing Face Recognition With Self-Supervised 3D Reconstruction

no code implementations CVPR 2022 Mingjie He, Jie Zhang, Shiguang Shan, Xilin Chen

In this paper, we propose to enhance face recognition with a bypass of self-supervised 3D reconstruction, which enforces the neural backbone to focus on the identity-related depth and albedo information while neglects the identity-irrelevant pose and illumination information.

3D Face Reconstruction 3D Reconstruction +3

DENSE: Data-Free One-Shot Federated Learning

1 code implementation23 Dec 2021 Jie Zhang, Chen Chen, Bo Li, Lingjuan Lyu, Shuang Wu, Shouhong Ding, Chunhua Shen, Chao Wu

One-shot Federated Learning (FL) has recently emerged as a promising approach, which allows the central server to learn a model in a single communication round.

Federated Learning

Initiative Defense against Facial Manipulation

1 code implementation19 Dec 2021 Qidong Huang, Jie Zhang, Wenbo Zhou, WeimingZhang, Nenghai Yu

To this end, we first imitate the target manipulation model with a surrogate model, and then devise a poison perturbation generator to obtain the desired venom.

Attribute Face Reenactment

From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimization

1 code implementation17 Dec 2021 Feijie Wu, Song Guo, Haozhao Wang, Zhihao Qu, Haobo Zhang, Jie Zhang, Ziming Liu

In the setting of federated optimization, where a global model is aggregated periodically, step asynchronism occurs when participants conduct model training by efficiently utilizing their computational resources.

GIMIRec: Global Interaction Information Aware Multi-Interest Framework for Sequential Recommendation

no code implementations16 Dec 2021 Jie Zhang, Ke-Jia Chen, Jingqiang Chen

Sequential recommendation based on multi-interest framework models the user's recent interaction sequence into multiple different interest vectors, since a single low-dimensional vector cannot fully represent the diversity of user interests.

Sequential Recommendation

Tracing Text Provenance via Context-Aware Lexical Substitution

no code implementations15 Dec 2021 Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, Nenghai Yu

Tracing text provenance can help claim the ownership of text content or identify the malicious users who distribute misleading content like machine-generated fake news.

Optical Character Recognition (OCR) Sentence

Deep Auto-encoder with Neural Response

no code implementations30 Nov 2021 Xuming Ran, Jie Zhang, Ziyuan Ye, Haiyan Wu, Qi Xu, Huihui Zhou, Quanying Liu

In this study, we propose an integrated framework called Deep Autoencoder with Neural Response (DAE-NR), which incorporates information from ANN and the visual cortex to achieve better image reconstruction performance and higher neural representation similarity between biological and artificial neurons.

Image Reconstruction

Adaptive Perturbation for Adversarial Attack

no code implementations27 Nov 2021 Zheng Yuan, Jie Zhang, Zhaoyan Jiang, Liangliang Li, Shiguang Shan

Instead of using the sign function, we propose to directly utilize the exact gradient direction with a scaling factor for generating adversarial perturbations, which improves the attack success rates of adversarial examples even with fewer perturbations.

Adversarial Attack

Adaptive Image Transformations for Transfer-based Adversarial Attack

2 code implementations27 Nov 2021 Zheng Yuan, Jie Zhang, Shiguang Shan

Adversarial attacks provide a good way to study the robustness of deep learning models.

Adversarial Attack

3D Lip Event Detection via Interframe Motion Divergence at Multiple Temporal Resolutions

no code implementations18 Nov 2021 Jie Zhang, Robert B. Fisher

We define a motion divergence measure using 3D lip landmarks to quantify the interframe dynamics of a 3D speaking lip.

Event Detection Motion Detection

Parameterized Knowledge Transfer for Personalized Federated Learning

1 code implementation NeurIPS 2021 Jie Zhang, Song Guo, Xiaosong Ma, Haozhao Wang, Wencao Xu, Feijie Wu

To deal with such model constraints, we exploit the potentials of heterogeneous model settings and propose a novel training framework to employ personalized models for different clients.

Personalized Federated Learning Transfer Learning

Learning Large Neighborhood Search Policy for Integer Programming

1 code implementation NeurIPS 2021 Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang

We then design a neural network to learn policies for each variable in parallel, trained by a customized actor-critic algorithm.

Reinforcement Learning (RL)

A Novel Sequence Tagging Framework for Consumer Event-Cause Extraction

no code implementations28 Oct 2021 Congqing He, Jie Zhang, Xiangyu Zhu, Huan Liu, Yukun Huang

To this end, we introduce a fresh perspective to revisit the relational event-cause extraction task and propose a novel sequence tagging framework, instead of extracting event types and events-causes separately.

SenseMag: Enabling Low-Cost Traffic Monitoring using Non-invasive Magnetic Sensing

no code implementations24 Oct 2021 Kafeng Wang, Haoyi Xiong, Jie Zhang, Hongyang Chen, Dejing Dou, Cheng-Zhong Xu

Extensive experiment based on real-word field deployment (on the highways in Shenzhen, China) shows that SenseMag significantly outperforms the existing methods in both classification accuracy and the granularity of vehicle types (i. e., 7 types by SenseMag versus 4 types by the existing work in comparisons).

Management

Cross-Vendor CT Image Data Harmonization Using CVH-CT

no code implementations19 Oct 2021 Md Selim, Jie Zhang, Baowei Fei, Guo-Qiang Zhang, Gary Yeeming Ge, Jin Chen

We propose a novel deep learning approach called CVH-CT for harmonizing CT images captured using scanners from different vendors.

Computed Tomography (CT)

NeuroLKH: Combining Deep Learning Model with Lin-Kernighan-Helsgaun Heuristic for Solving the Traveling Salesman Problem

1 code implementation NeurIPS 2021 Liang Xin, Wen Song, Zhiguang Cao, Jie Zhang

We present NeuroLKH, a novel algorithm that combines deep learning with the strong traditional heuristic Lin-Kernighan-Helsgaun (LKH) for solving Traveling Salesman Problem.

Traveling Salesman Problem

M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining

no code implementations8 Oct 2021 Junyang Lin, An Yang, Jinze Bai, Chang Zhou, Le Jiang, Xianyan Jia, Ang Wang, Jie Zhang, Yong Li, Wei Lin, Jingren Zhou, Hongxia Yang

Recent expeditious developments in deep learning algorithms, distributed training, and even hardware design for large models have enabled training extreme-scale models, say GPT-3 and Switch Transformer possessing hundreds of billions or even trillions of parameters.

Heterogeneous Attentions for Solving Pickup and Delivery Problem via Deep Reinforcement Learning

no code implementations6 Oct 2021 Jingwen Li, Liang Xin, Zhiguang Cao, Andrew Lim, Wen Song, Jie Zhang

In particular, the heterogeneous attention mechanism specifically prescribes attentions for each role of the nodes while taking into account the precedence constraint, i. e., the pickup node must precede the pairing delivery node.

reinforcement-learning Reinforcement Learning (RL)

Deep Reinforcement Learning for Solving the Heterogeneous Capacitated Vehicle Routing Problem

1 code implementation6 Oct 2021 Jingwen Li, Yining Ma, Ruize Gao, Zhiguang Cao, Andrew Lim, Wen Song, Jie Zhang

To solve those problems, we propose a DRL method based on the attention mechanism with a vehicle selection decoder accounting for the heterogeneous fleet constraint and a node selection decoder accounting for the route construction, which learns to construct a solution by automatically selecting both a vehicle and a node for this vehicle at each step.

reinforcement-learning Reinforcement Learning (RL)

ACDC: Online Unsupervised Cross-Domain Adaptation

1 code implementation4 Oct 2021 Marcus de Carvalho, Mahardhika Pratama, Jie Zhang, Edward Yapp

We consider the problem of online unsupervised cross-domain adaptation, where two independent but related data streams with different feature spaces -- a fully labeled source stream and an unlabeled target stream -- are learned together.

Online unsupervised domain adaptation

Learning Scenario Representation for Solving Two-stage Stochastic Integer Programs

no code implementations ICLR 2022 Yaoxin Wu, Wen Song, Zhiguang Cao, Jie Zhang

Many practical combinatorial optimization problems under uncertainty can be modeled as stochastic integer programs (SIPs), which are extremely challenging to solve due to the high complexity.

Combinatorial Optimization Vocal Bursts Valence Prediction

Generative Adversarial Training for Neural Combinatorial Optimization Models

no code implementations29 Sep 2021 Liang Xin, Wen Song, Zhiguang Cao, Jie Zhang

Recent studies show that deep neural networks can be trained to learn good heuristics for various Combinatorial Optimization Problems (COPs).

Combinatorial Optimization Traveling Salesman Problem

Pairwise Emotional Relationship Recognition in Drama Videos: Dataset and Benchmark

1 code implementation23 Sep 2021 Xun Gao, Yin Zhao, Jie Zhang, Longjun Cai

We expect the ERATO as well as our proposed SMTA to open up a new way for PERR task in video understanding and further improve the research of multi-modal fusion methodology.

Video Understanding

Optimisation of two-stage biomass gasification for hydrogen production via artificial neural network

no code implementations Applied Energy 2021 Hannah O. Kargbo, Jie Zhang, Anh N. Phan

The developed neural network model was then applied for optimising operating conditions of the two-stage gasification for high carbon conversion, high hydrogen yield and low carbon dioxide in nitrogen and carbon dioxide environments.

Poison Ink: Robust and Invisible Backdoor Attack

1 code implementation5 Aug 2021 Jie Zhang, Dongdong Chen, Qidong Huang, Jing Liao, Weiming Zhang, Huamin Feng, Gang Hua, Nenghai Yu

As the image structure can keep its semantic meaning during the data transformation, such trigger pattern is inherently robust to data transformations.

Backdoor Attack Data Poisoning

Exploring Structure Consistency for Deep Model Watermarking

no code implementations5 Aug 2021 Jie Zhang, Dongdong Chen, Jing Liao, Han Fang, Zehua Ma, Weiming Zhang, Gang Hua, Nenghai Yu

However, little attention has been devoted to the protection of DNNs in image processing tasks.

Data Augmentation

Locality-aware Channel-wise Dropout for Occluded Face Recognition

no code implementations20 Jul 2021 Mingjie He, Jie Zhang, Shiguang Shan, Xiao Liu, Zhongqin Wu, Xilin Chen

Furthermore, by randomly dropping out several feature channels, our method can well simulate the occlusion of larger area.

Face Recognition

Supply Chain Digital Twin Framework Design: An Approach of Supply Chain Operations Reference Model and System of Systems

no code implementations19 Jul 2021 Jie Zhang, Alexandra Brintrup, Anisoara Calinescu, Edward Kosasih, Angira Sharma

This paper explains what is 'twined' in supply chain digital twin and how to 'twin' them to handle the spatio-temporal dynamic issue.

Impact of Rotary-Wing UAV Wobbling on Millimeter-wave Air-to-Ground Wireless Channel

no code implementations14 Jul 2021 Songjiang Yang, Zitian Zhang, Jiliang Zhang, Jie Zhang

Our contributions of this paper lie in: i) modeling the wobbling process of a hovering RW UAV; ii) developing an analytical model to derive the channel temporal autocorrelation function (ACF) for the millimeter-wave RW UAV A2G link in a closed-form expression; and iii) investigating how RW UAV wobbling impacts the Doppler effect on the millimeter-wave RW UAV A2G link.

SelfCF: A Simple Framework for Self-supervised Collaborative Filtering

2 code implementations7 Jul 2021 Xin Zhou, Aixin Sun, Yong liu, Jie Zhang, Chunyan Miao

Collaborative filtering (CF) is widely used to learn informative latent representations of users and items from observed interactions.

Collaborative Filtering Self-Supervised Learning

CT Image Harmonization for Enhancing Radiomics Studies

no code implementations3 Jul 2021 Md Selim, Jie Zhang, Baowei Fei, Guo-Qiang Zhang, Jin Chen

While remarkable advances have been made in Computed Tomography (CT), capturing CT images with non-standardized protocols causes low reproducibility regarding radiomic features, forming a barrier on CT image analysis in a large scale.

Computed Tomography (CT) Image Harmonization

From Personalized Medicine to Population Health: A Survey of mHealth Sensing Techniques

no code implementations2 Jul 2021 Zhiyuan Wang, Haoyi Xiong, Jie Zhang, Sijia Yang, Mehdi Boukhechba, Laura E. Barnes, Daqing Zhang, Dejing Dou

Mobile Sensing Apps have been widely used as a practical approach to collect behavioral and health-related information from individuals and provide timely intervention to promote health and well-beings, such as mental health and chronic cares.

M6-T: Exploring Sparse Expert Models and Beyond

no code implementations31 May 2021 An Yang, Junyang Lin, Rui Men, Chang Zhou, Le Jiang, Xianyan Jia, Ang Wang, Jie Zhang, Jiamang Wang, Yong Li, Di Zhang, Wei Lin, Lin Qu, Jingren Zhou, Hongxia Yang

Mixture-of-Experts (MoE) models can achieve promising results with outrageous large amount of parameters but constant computation cost, and thus it has become a trend in model scaling.

Playing the Game of 2048

HINet: Half Instance Normalization Network for Image Restoration

2 code implementations13 May 2021 Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, Chengpeng Chen

Specifically, we present a novel block: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks.

Deblurring Image Deblurring +3

Reversible Watermarking in Deep Convolutional Neural Networks for Integrity Authentication

no code implementations9 Apr 2021 Xiquan Guan, Huamin Feng, Weiming Zhang, Hang Zhou, Jie Zhang, Nenghai Yu

Specifically, we present the reversible watermarking problem of deep convolutional neural networks and utilize the pruning theory of model compression technology to construct a host sequence used for embedding watermarking information by histogram shift.

Model Compression

Deep Model Intellectual Property Protection via Deep Watermarking

1 code implementation8 Mar 2021 Jie Zhang, Dongdong Chen, Jing Liao, Weiming Zhang, Huamin Feng, Gang Hua, Nenghai Yu

By jointly training the target model and watermark embedding, the extra barrier can even be absorbed into the target model.

M6: A Chinese Multimodal Pretrainer

no code implementations1 Mar 2021 Junyang Lin, Rui Men, An Yang, Chang Zhou, Ming Ding, Yichang Zhang, Peng Wang, Ang Wang, Le Jiang, Xianyan Jia, Jie Zhang, Jianwei Zhang, Xu Zou, Zhikang Li, Xiaodong Deng, Jie Liu, Jinbao Xue, Huiling Zhou, Jianxin Ma, Jin Yu, Yong Li, Wei Lin, Jingren Zhou, Jie Tang, Hongxia Yang

In this work, we construct the largest dataset for multimodal pretraining in Chinese, which consists of over 1. 9TB images and 292GB texts that cover a wide range of domains.

Image Generation

Meta-Path-Free Representation Learning on Heterogeneous Networks

no code implementations16 Feb 2021 Jie Zhang, Jinru Ding, Suyuan Liu, Hongyan Wu

To the best of our knowledge, this is the first attempt to break out of the confinement of meta-paths for representation learning on heterogeneous networks.

Knowledge Graphs Representation Learning

Dynamic Virtual Graph Significance Networks for Predicting Influenza

1 code implementation16 Feb 2021 Jie Zhang, Pengfei Zhou, Hongyan Wu

In this study, we develop a novel method, Dynamic Virtual Graph Significance Networks (DVGSN), which can supervisedly and dynamically learn from similar "infection situations" in historical timepoints.

Representation Learning Time Series +1

Spatio-Temporal Multi-step Prediction of Influenza Outbreaks

no code implementations16 Feb 2021 Jie Zhang, Kazumitsu Nawata, Hongyan Wu

We compared the MAPEs of SVM, RF, LSTM models of predicting flu data of the 1-4 weeks ahead with and without other countries' flu data.

Spin and polarization effects on the nonlinear Breit-Wheeler pair production in laser-plasma interaction

no code implementations11 Feb 2021 Huai-Hang Song, Wei-Min Wang, Yan-Fei Li, Bing-Jun Li, Yu-Tong Li, Zheng-Ming Sheng, Li-Ming Chen, Jie Zhang

The spin effect of electrons/positrons ($e^-$/$e^+$) and polarization effect of $\gamma$ photons are investigated in the interaction of two counter-propagating linearly polarized 10-PW-class laser pulses with a thin foil target.

Plasma Physics

An Improvement for Quantum Tunneling Radiation of Fermions in a Stationary Kerr-Newman Black Hole Spacetime

no code implementations9 Feb 2021 Jie Zhang, Menquan Liu, Zhie Liu, Shuzheng Yang

By introducing a specific etheric-like vector in the Dirac equation with Lorentz Invariance Violation (LIV) in the curved spacetime, an improved method for quantum tunneling radiation of fermions is proposed.

General Relativity and Quantum Cosmology High Energy Physics - Theory

BridgeDPI: A Novel Graph Neural Network for Predicting Drug-Protein Interactions

1 code implementation29 Jan 2021 Yifan Wu, Min Gao, Min Zeng, Feiyang Chen, Min Li, Jie Zhang

Therefore, we hope to develop a novel supervised learning method to learn the PPAs and DDAs effectively and thereby improve the prediction performance of the specific task of DPI.

Drug Discovery

Optical manipulation of electronic dimensionality in a quantum material

no code implementations21 Jan 2021 Shaofeng Duan, Yun Cheng, Wei Xia, Yuanyuan Yang, Fengfeng Qi, Tianwei Tang, Yanfeng Guo, Dong Qian, Dao Xiang, Jie Zhang, Wentao Zhang

Exotic phenomenon can be achieved in quantum materials by confining electronic states into two dimensions.

Strongly Correlated Electrons Materials Science Superconductivity

Cannot find the paper you are looking for? You can Submit a new open access paper.