Search Results for author: Lei Ma

Found 126 papers, 44 papers with code

MAGIC: Mastering Physical Adversarial Generation in Context through Collaborative LLM Agents

no code implementations11 Dec 2024 Yun Xing, Nhat Chung, Jie Zhang, Yue Cao, Ivor Tsang, Yang Liu, Lei Ma, Qing Guo

We validate our method on both digital and physical level, \ie, nuImage and manually captured real scenes, where both statistical and visual results prove that our MAGIC is powerful and effectively for attacking wide-used object detection systems.

object-detection Object Detection +2

Splats in Splats: Embedding Invisible 3D Watermark within Gaussian Splatting

no code implementations4 Dec 2024 Yijia Guo, Wenkai Huang, Yang Li, Gaolei Li, Hang Zhang, Liwen Hu, Jianhua Li, Tiejun Huang, Lei Ma

3D Gaussian splatting (3DGS) has demonstrated impressive 3D reconstruction performance with explicit scene representations.

3D Reconstruction

Towards Understanding Retrieval Accuracy and Prompt Quality in RAG Systems

no code implementations29 Nov 2024 Shengming Zhao, Yuheng Huang, Jiayang Song, Zhijie Wang, Chengcheng Wan, Lei Ma

Retrieval-Augmented Generation (RAG) is a pivotal technique for enhancing the capability of large language models (LLMs) and has demonstrated promising efficacy across a diverse spectrum of tasks.

RAG Retrieval

RedTest: Towards Measuring Redundancy in Deep Neural Networks Effectively

no code implementations15 Nov 2024 Yao Lu, Peixin Zhang, Jingyi Wang, Lei Ma, Xiaoniu Yang, Qi Xuan

To address the problem, we present a novel testing approach, i. e., RedTest, which proposes a novel testing metric called Model Structural Redundancy Score (MSRS) to quantitatively measure the degree of redundancy in a deep learning model structure.

Deep Learning Model Optimization +1

CTEFM-VC: Zero-Shot Voice Conversion Based on Content-Aware Timbre Ensemble Modeling and Flow Matching

no code implementations4 Nov 2024 Yu Pan, Yuguang Yang, Jixun Yao, Jianhao Ye, Hongbin Zhou, Lei Ma, Jianjun Zhao

Zero-shot voice conversion (VC) aims to transform the timbre of a source speaker into any previously unseen target speaker, while preserving the original linguistic content.

Speaker Verification Voice Conversion

Large Language Model Supply Chain: Open Problems From the Security Perspective

no code implementations3 Nov 2024 Qiang Hu, Xiaofei Xie, Sen Chen, Lei Ma

Large Language Model (LLM) is changing the software development paradigm and has gained huge attention from both academia and industry.

Language Modelling Large Language Model

Towards Unifying Understanding and Generation in the Era of Vision Foundation Models: A Survey from the Autoregression Perspective

1 code implementation29 Oct 2024 Shenghao Xie, Wenqiang Zu, Mingyang Zhao, Duo Su, Shilong Liu, Ruohua Shi, Guoqi Li, Shanghang Zhang, Lei Ma

First, we present the trend for next generation of vision foundation models, i. e., unifying both understanding and generation in vision tasks.

Survey

Takin-VC: Zero-shot Voice Conversion via Jointly Hybrid Content and Memory-Augmented Context-Aware Timbre Modeling

no code implementations2 Oct 2024 Yuguang Yang, Yu Pan, Jixun Yao, Xiang Zhang, Jianhao Ye, Hongbin Zhou, Lei Xie, Lei Ma, Jianjun Zhao

Zero-shot voice conversion (VC) aims to transform the source speaker timbre into an arbitrary unseen one without altering the original speech content. While recent advancements in zero-shot VC methods have shown remarkable progress, there still remains considerable potential for improvement in terms of improving speaker similarity and speech naturalness. In this paper, we propose Takin-VC, a novel zero-shot VC framework based on jointly hybrid content and memory-augmented context-aware timbre modeling to tackle this challenge.

Voice Conversion

Learning from Pattern Completion: Self-supervised Controllable Generation

1 code implementation27 Sep 2024 Zhiqiang Chen, Guofan Fan, Jinying Gao, Lei Ma, Bo Lei, Tiejun Huang, Shan Yu

Inspired by the neural mechanisms that may contribute to the brain's associative power, specifically the cortical modularization and hippocampal pattern completion, here we propose a self-supervised controllable generation (SCG) framework.

Edge Detection Semantic Segmentation

ShapeMamba-EM: Fine-Tuning Foundation Model with Local Shape Descriptors and Mamba Blocks for 3D EM Image Segmentation

no code implementations26 Aug 2024 Ruohua Shi, Qiufan Pang, Lei Ma, Lingyu Duan, Tiejun Huang, Tingting Jiang

Electron microscopy (EM) imaging offers unparalleled resolution for analyzing neural tissues, crucial for uncovering the intricacies of synaptic connections and neural processes fundamental to understanding behavioral mechanisms.

Image Segmentation Mamba +3

LeCov: Multi-level Testing Criteria for Large Language Models

no code implementations20 Aug 2024 Xuan Xie, Jiayang Song, Yuheng Huang, Da Song, Fuyuan Zhang, Felix Juefei-Xu, Lei Ma

Large Language Models (LLMs) are widely used in many different domains, but because of their limited interpretability, there are questions about how trustworthy they are in various perspectives, e. g., truthfulness and toxicity.

I-SHEEP: Self-Alignment of LLM from Scratch through an Iterative Self-Enhancement Paradigm

1 code implementation15 Aug 2024 Yiming Liang, Ge Zhang, Xingwei Qu, Tianyu Zheng, Jiawei Guo, Xinrun Du, Zhenzhu Yang, Jiaheng Liu, Chenghua Lin, Lei Ma, Wenhao Huang, Jiajun Zhang

Large Language Models (LLMs) have achieved significant advancements, however, the common learning paradigm treats LLMs as passive information repositories, neglecting their potential for active learning and alignment.

Active Learning Code Generation

Active Testing of Large Language Model via Multi-Stage Sampling

no code implementations7 Aug 2024 Yuheng Huang, Jiayang Song, Qiang Hu, Felix Juefei-Xu, Lei Ma

Given that LLMs' diverse task-handling abilities stem from large volumes of training data, a comprehensive evaluation also necessitates abundant, well-annotated, and representative test data to assess LLM performance across various downstream tasks.

Language Modelling Large Language Model

PRTGS: Precomputed Radiance Transfer of Gaussian Splats for Real-Time High-Quality Relighting

no code implementations7 Aug 2024 Yijia Guo, Yuanxi Bai, Liwen Hu, Ziyi Guo, Mianzhi Liu, Yu Cai, Tiejun Huang, Lei Ma

We proposed Precomputed RadianceTransfer of GaussianSplats (PRTGS), a real-time high-quality relighting method for Gaussian splats in low-frequency lighting environments that captures soft shadows and interreflections by precomputing 3D Gaussian splats' radiance transfer.

Deep Learning Meets OBIA: Tasks, Challenges, Strategies, and Perspectives

no code implementations2 Aug 2024 Lei Ma, Ziyun Yan, Mengmeng Li, Tao Liu, Liqin Tan, Xuan Wang, Weiqiang He, Ruikun Wang, Guangjun He, Heng Lu, Thomas Blaschke

Deep learning has gained significant attention in remote sensing, especially in pixel- or patch-level applications.

Deep Learning

Multilingual Blending: LLM Safety Alignment Evaluation with Language Mixture

no code implementations10 Jul 2024 Jiayang Song, Yuheng Huang, Zhehua Zhou, Lei Ma

As safety remains a crucial concern throughout the development lifecycle of Large Language Models (LLMs), researchers and industrial practitioners have increasingly focused on safeguarding and aligning LLM behaviors with human preferences and ethical standards.

Safety Alignment

Vortex under Ripplet: An Empirical Study of RAG-enabled Applications

no code implementations6 Jul 2024 Yuchen Shao, Yuheng Huang, Jiawei Shen, Lei Ma, Ting Su, Chengcheng Wan

Large language models (LLMs) enhanced by retrieval-augmented generation (RAG) provide effective solutions in various application scenarios.

Management RAG +1

SpikeGS: Reconstruct 3D scene via fast-moving bio-inspired sensors

no code implementations4 Jul 2024 Yijia Guo, Liwen Hu, Yuanxi Bai, Lei Ma, Tiejun Huang

3D Gaussian Splatting (3DGS) demonstrates unparalleled superior performance in 3D scene reconstruction.

3D Scene Reconstruction

Embedded Prompt Tuning: Towards Enhanced Calibration of Pretrained Models for Medical Images

1 code implementation1 Jul 2024 Wenqiang Zu, Shenghao Xie, Qing Zhao, Guoqi Li, Lei Ma

In this work, we facilitate the study of the performance of PEFT when adapting foundation models to medical image classification tasks.

Cross-Domain Few-Shot Image Classification +3

GenSafe: A Generalizable Safety Enhancer for Safe Reinforcement Learning Algorithms Based on Reduced Order Markov Decision Process Model

no code implementations6 Jun 2024 Zhehua Zhou, Xuan Xie, Jiayang Song, Zhan Shu, Lei Ma

To improve system safety during the learning process, a variety of Safe Reinforcement Learning (SRL) algorithms have been proposed, which usually incorporate safety constraints within the Constrained Markov Decision Process (CMDP) framework.

Autonomous Vehicles Deep Reinforcement Learning +2

SCSim: A Realistic Spike Cameras Simulator

1 code implementation27 May 2024 Liwen Hu, Lei Ma, Yijia Guo, Tiejun Huang

Large-scale synthetic datasets have significantly accelerated the development of these cameras, particularly in reconstruction and optical flow.

Optical Flow Estimation

CodeHalu: Investigating Code Hallucinations in LLMs via Execution-based Verification

1 code implementation30 Apr 2024 Yuchen Tian, Weixiang Yan, Qian Yang, Xuandong Zhao, Qian Chen, Wen Wang, Ziyang Luo, Lei Ma, Dawn Song

By evaluating 17 popular LLMs using this benchmark, we reveal significant differences in their accuracy and reliability in code generation, offering detailed insights for further improving the code generation capabilities of LLMs.

Code Generation Hallucination

DPGAN: A Dual-Path Generative Adversarial Network for Missing Data Imputation in Graphs

no code implementations26 Apr 2024 Xindi Zheng, Yuwei Wu, Yu Pan, WanYu Lin, Lei Ma, Jianjun Zhao

The crux of our work is that it admits both global and local representations of the input graph signal, which can capture the long-range dependencies.

Generative Adversarial Network Graph Neural Network +1

Evaluation and Improvement of Fault Detection for Large Language Models

no code implementations14 Apr 2024 Qiang Hu, Jin Wen, Maxime Cordy, Yuheng Huang, Wei Ma, Xiaofei Xie, Lei Ma

Experimental results on four different tasks~(including both code tasks and natural language processing tasks) and four LLMs~(e. g., LLaMA3 and GPT4) demonstrated that simple methods such as Margin perform well on LLMs but there is still a big room for improvement.

Autonomous Driving Fault Detection

Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward

no code implementations12 Apr 2024 Xuan Xie, Jiayang Song, Zhehua Zhou, Yuheng Huang, Da Song, Lei Ma

To bridge this gap, we conduct in this work a comprehensive evaluation of the effectiveness of existing online safety analysis methods on LLMs.

Fairness

LRR: Language-Driven Resamplable Continuous Representation against Adversarial Tracking Attacks

1 code implementation9 Apr 2024 Jianlang Chen, Xuhong Ren, Qing Guo, Felix Juefei-Xu, Di Lin, Wei Feng, Lei Ma, Jianjun Zhao

To achieve high accuracy on both clean and adversarial data, we propose building a spatial-temporal continuous representation using the semantic text guidance of the object of interest.

Object Visual Object Tracking

PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics

1 code implementation6 Apr 2024 Derui Zhu, Dingfan Chen, Qing Li, Zongxiong Chen, Lei Ma, Jens Grossklags, Mario Fritz

Despite tremendous advancements in large language models (LLMs) over recent years, a notably urgent challenge for their practical deployment is the phenomenon of hallucination, where the model fabricates facts and produces non-factual statements.

Benchmarking Hallucination +1

PSCodec: A Series of High-Fidelity Low-bitrate Neural Speech Codecs Leveraging Prompt Encoders

no code implementations3 Apr 2024 Yu Pan, Xiang Zhang, Yuguang Yang, Jixun Yao, Yanni Hu, Jianhao Ye, Hongbin Zhou, Lei Ma, Jianjun Zhao

In this paper, we propose PSCodec, a series of neural speech codecs based on prompt encoders, comprising PSCodec-Base, PSCodec-DRL-ICT, and PSCodec-CasAN, which are capable of delivering high-performance speech reconstruction with low bandwidths.

Representation Learning Speaker Verification +5

Spike-NeRF: Neural Radiance Field Based On Spike Camera

no code implementations25 Mar 2024 Yijia Guo, Yuanxi Bai, Liwen Hu, Mianzhi Liu, Ziyi Guo, Lei Ma, Tiejun Huang

As a neuromorphic sensor with high temporal resolution, spike cameras offer notable advantages over traditional cameras in high-speed vision applications such as high-speed optical estimation, depth estimation, and object tracking.

3D Reconstruction Depth Estimation +1

DEEP-ICL: Definition-Enriched Experts for Language Model In-Context Learning

no code implementations7 Mar 2024 Xingwei Qu, Yiming Liang, Yucheng Wang, Tianyu Zheng, Tommy Yue, Lei Ma, Stephen W. Huang, Jiajun Zhang, Yinan Shi, Chenghua Lin, Jie Fu, Ge Zhang

It has long been assumed that the sheer number of parameters in large language models (LLMs) drives in-context learning (ICL) capabilities, enabling remarkable performance improvements by leveraging task-specific demonstrations.

Few-Shot Learning In-Context Learning +1

PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement

1 code implementation6 Mar 2024 Zhijie Wang, Yuheng Huang, Da Song, Lei Ma, Tianyi Zhang

However, prompting remains challenging for novice users due to the complexity of the stable diffusion model and the non-trivial efforts required for iteratively editing and refining the text prompts.

Image Inpainting Prompt Engineering +1

m2mKD: Module-to-Module Knowledge Distillation for Modular Transformers

1 code implementation26 Feb 2024 Ka Man Lo, Yiming Liang, Wenyu Du, Yuantao Fan, Zili Wang, Wenhao Huang, Lei Ma, Jie Fu

Additionally, the V-MoE-Base model trained with m2mKD achieves 3. 5% higher accuracy than end-to-end training on ImageNet-1k.

Knowledge Distillation

CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models

no code implementations20 Feb 2024 Yizhi Li, Ge Zhang, Xingwei Qu, Jiali Li, Zhaoqun Li, Zekun Wang, Hao Li, Ruibin Yuan, Yinghao Ma, Kai Zhang, Wangchunshu Zhou, Yiming Liang, Lei Zhang, Lei Ma, Jiajun Zhang, Zuowen Li, Stephen W. Huang, Chenghua Lin, Jie Fu

The advancement of large language models (LLMs) has enhanced the ability to generalize across a wide range of unseen natural language processing (NLP) tasks through instruction-following.

Instruction Following

Large Language Models Based Fuzzing Techniques: A Survey

1 code implementation1 Feb 2024 Linghan Huang, Peizhou Zhao, Huaming Chen, Lei Ma

In the modern era where software plays a pivotal role, software security and vulnerability analysis have become essential for software development.

software testing Survey

Learning to Robustly Reconstruct Low-light Dynamic Scenes from Spike Streams

1 code implementation19 Jan 2024 Liwen Hu, Ziluo Ding, Mianzhi Liu, Lei Ma, Tiejun Huang

In this paper, we propose a bidirectional recurrent-based reconstruction framework, including a Light-Robust Representation (LR-Rep) and a fusion module, to better handle such extreme conditions.

Code Ownership in Open-Source AI Software Security

1 code implementation18 Dec 2023 Jiawen Wen, Dong Yuan, Lei Ma, Huaming Chen

As open-source AI software projects become an integral component in the AI software development, it is critical to develop a novel methods to ensure and measure the security of the open-source projects for developers.

Benchmarking

PathoDuet: Foundation Models for Pathological Slide Analysis of H&E and IHC Stains

1 code implementation15 Dec 2023 Shengyi Hua, Fang Yan, Tianle Shen, Lei Ma, Xiaofan Zhang

To validate the efficacy of our models, we evaluate the performance over a wide variety of downstream tasks, including patch-level colorectal cancer subtyping and whole slide image (WSI)-level classification in H&E field, together with expression level prediction of IHC marker, tumor identification and slide-level qualitative analysis in IHC field.

Self-Supervised Learning

SynFundus-1M: A High-quality Million-scale Synthetic fundus images Dataset with Fifteen Types of Annotation

1 code implementation1 Dec 2023 Fangxin Shang, Jie Fu, Yehui Yang, Haifeng Huang, Junwei Liu, Lei Ma

Large-scale public datasets with high-quality annotations are rarely available for intelligent medical imaging research, due to data privacy concerns and the cost of annotations.

Denoising

Domain Generalization of 3D Object Detection by Density-Resampling

1 code implementation17 Nov 2023 Shuangzhi Li, Lei Ma, Xingyu Li

Specifically, from the perspective of data augmentation, we design a universal physical-aware density-based data augmentation (PDDA) method to mitigate the performance loss stemming from diverse point densities.

3D Object Detection Data Augmentation +6

LUNA: A Model-Based Universal Analysis Framework for Large Language Models

no code implementations22 Oct 2023 Da Song, Xuan Xie, Jiayang Song, Derui Zhu, Yuheng Huang, Felix Juefei-Xu, Lei Ma

the trustworthiness perspective, is bound to and enriches the abstract model with semantics, which enables more detailed analysis applications for diverse purposes.

ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning

1 code implementation26 Aug 2023 Zhehua Zhou, Jiayang Song, Kunpeng Yao, Zhan Shu, Lei Ma

Motivated by the substantial achievements observed in Large Language Models (LLMs) in the field of natural language processing, recent research has commenced investigations into the application of LLMs for complex, long-horizon sequential task planning challenges in robotics.

Language Modelling Large Language Model

MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition

no code implementations8 Aug 2023 Yu Pan, Yuguang Yang, Yuheng Huang, Jixun Yao, JingJing Yin, Yanni Hu, Heng Lu, Lei Ma, Jianjun Zhao

Despite notable progress, speech emotion recognition (SER) remains challenging due to the intricate and ambiguous nature of speech emotion, particularly in wild world.

Attribute Cross-corpus +2

Accurate 3D Prediction of Missing Teeth in Diverse Patterns for Precise Dental Implant Planning

no code implementations16 Jul 2023 Lei Ma, Peng Xue, Yuning Gu, Yue Zhao, Min Zhu, Zhongxiang Ding, Dinggang Shen

This study presents a novel framework for accurate prediction of missing teeth in different patterns, facilitating digital implant planning.

Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models

no code implementations16 Jul 2023 Yuheng Huang, Jiayang Song, Zhijie Wang, Shengming Zhao, Huaming Chen, Felix Juefei-Xu, Lei Ma

In particular, we experiment with twelve uncertainty estimation methods and four LLMs on four prominent natural language processing (NLP) tasks to investigate to what extent uncertainty estimation techniques could help characterize the prediction risks of LLMs.

Code Generation Hallucination +1

CephGPT-4: An Interactive Multimodal Cephalometric Measurement and Diagnostic System with Visual Large Language Model

no code implementations1 Jul 2023 Lei Ma, Jincong Han, Zhaoxin Wang, Dian Zhang

Firstly, a multimodal orthodontic medical dataset is constructed, comprising cephalometric images and doctor-patient dialogue data, with automatic analysis of cephalometric landmarks using U-net and generation of diagnostic reports.

Language Modelling Large Language Model

Mitigating Communication Costs in Neural Networks: The Role of Dendritic Nonlinearity

no code implementations21 Jun 2023 Xundong Wu, Pengfei Zhao, Zilin Yu, Lei Ma, Ka-Wa Yip, Huajin Tang, Gang Pan, Tiejun Huang

Our comprehension of biological neuronal networks has profoundly influenced the evolution of artificial neural networks (ANNs).

GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition

no code implementations13 Jun 2023 Yu Pan, Yanni Hu, Yuguang Yang, Wen Fei, Jixun Yao, Heng Lu, Lei Ma, Jianjun Zhao

Contrastive cross-modality pretraining has recently exhibited impressive success in diverse fields, whereas there is limited research on their merits in speech emotion recognition (SER).

Attribute Contrastive Learning +3

Benchmarking Robustness of AI-Enabled Multi-sensor Fusion Systems: Challenges and Opportunities

no code implementations6 Jun 2023 Xinyu Gao, Zhijie Wang, Yang Feng, Lei Ma, Zhenyu Chen, Baowen Xu

Multi-Sensor Fusion (MSF) based perception systems have been the foundation in supporting many industrial applications and domains, such as self-driving cars, robotic arms, and unmanned aerial vehicles.

Benchmarking Depth Completion +5

Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization

1 code implementation5 Jun 2023 Yibing Liu, Chris Xing Tian, Haoliang Li, Lei Ma, Shiqi Wang

The out-of-distribution (OOD) problem generally arises when neural networks encounter data that significantly deviates from the training data distribution, i. e., in-distribution (InD).

Out-of-Distribution Detection

Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias

1 code implementation NeurIPS 2023 Zhongwei Wan, Che Liu, Mi Zhang, Jie Fu, Benyou Wang, Sibo Cheng, Lei Ma, César Quilodrán-Casas, Rossella Arcucci

Med-UniC reaches superior performance across 5 medical image tasks and 10 datasets encompassing over 30 diseases, offering a versatile framework for unifying multi-modal medical data within diverse linguistic communities.

Disentanglement

Evading DeepFake Detectors via Adversarial Statistical Consistency

no code implementations CVPR 2023 Yang Hou, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Jianjun Zhao

Second, we find that the statistical differences between natural and DeepFake images are positively associated with the distribution shifting between the two kinds of images, and we propose to use a distribution-aware loss to guide the optimization of different degradations.

DeepFake Detection Face Swapping

Construction of unbiased dental template and parametric dental model for precision digital dentistry

no code implementations7 Apr 2023 Lei Ma, Jingyang Zhang, Ke Deng, Peng Xue, Zhiming Cui, Yu Fang, Minhui Tang, Yue Zhao, Min Zhu, Zhongxiang Ding, Dinggang Shen

In this study, we develop an unbiased dental template by constructing an accurate dental atlas from CBCT images with guidance of teeth segmentation.

Image Cropping Segmentation

Spike Stream Denoising via Spike Camera Simulation

no code implementations6 Apr 2023 Liwen Hu, Lei Ma, Zhaofei Yu, Boxin Shi, Tiejun Huang

Based on our noise model, the first benchmark for spike stream denoising is proposed which includes clear (noisy) spike stream.

Denoising

DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction

1 code implementation2 Mar 2023 Zhijie Wang, Yuheng Huang, Da Song, Lei Ma, Tianyi Zhang

The core of DeepSeer is a state abstraction method that bundles semantically similar hidden states in an RNN model and abstracts the model as a finite state machine.

Explainable Artificial Intelligence (XAI)

DeepLens: Interactive Out-of-distribution Data Detection in NLP Models

1 code implementation2 Mar 2023 Da Song, Zhijie Wang, Yuheng Huang, Lei Ma, Tianyi Zhang

In this work, we propose DeepLens, an interactive system that helps users detect and explore OOD issues in massive text corpora.

Text Clustering

Neural Episodic Control with State Abstraction

no code implementations27 Jan 2023 Zhuo Li, Derui Zhu, Yujing Hu, Xiaofei Xie, Lei Ma, Yan Zheng, Yan Song, Yingfeng Chen, Jianjun Zhao

Generally, episodic control-based approaches are solutions that leverage highly-rewarded past experiences to improve sample efficiency of DRL algorithms.

Deep Reinforcement Learning OpenAI Gym

An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty

no code implementations13 Dec 2022 Zhijie Wang, Yuheng Huang, Lei Ma, Haruki Yokoyama, Susumu Tokumoto, Kazuki Munakata

More importantly, it also lacks systematic investigation on how to perform the risk assessment for AI systems from unit level to system level.

An Empirical Study of AI Techniques in Mobile Applications

1 code implementation3 Dec 2022 Yinghua Li, Xueqi Dang, Haoye Tian, Tiezhu Sun, Zhijie Wang, Lei Ma, Jacques Klein, Tegawendé F. Bissyandé

Our study encompasses 56, 682 real-world AI applications, focusing on three crucial perspectives: 1) Application analysis, where we analyze the popularity of AI apps and investigate the update states of AI apps; 2) Framework and model analysis, where we analyze AI framework usage and AI model protection; 3) User analysis, where we examine user privacy protection and user review attitudes.

FAF: A novel multimodal emotion recognition approach integrating face, body and text

no code implementations20 Nov 2022 Zhongyu Fang, Aoyun He, Qihui Yu, Baopeng Gao, Weiping Ding, Tong Zhang, Lei Ma

In this paper, we developed a large multimodal emotion dataset, named "HED" dataset, to facilitate the emotion recognition task, and accordingly propose a multimodal emotion recognition method.

Multimodal Emotion Recognition

Common Corruption Robustness of Point Cloud Detectors: Benchmark and Enhancement

no code implementations12 Oct 2022 Shuangzhi Li, Zhijie Wang, Felix Juefei-Xu, Qing Guo, Xingyu Li, Lei Ma

Then, for the first attempt, we construct a benchmark based on the physical-aware common corruptions for point cloud detectors, which contains a total of 1, 122, 150 examples covering 7, 481 scenes, 25 common corruption types, and 6 severities.

Autonomous Driving Cloud Detection +4

Decompiling x86 Deep Neural Network Executables

no code implementations3 Oct 2022 Zhibo Liu, Yuanyuan Yuan, Shuai Wang, Xiaofei Xie, Lei Ma

BTD takes DNN executables and outputs full model specifications, including types of DNN operators, network topology, dimensions, and parameters that are (nearly) identical to those of the input models.

DARTSRepair: Core-failure-set Guided DARTS for Network Robustness to Common Corruptions

no code implementations21 Sep 2022 Xuhong Ren, Jianlang Chen, Felix Juefei-Xu, Wanli Xue, Qing Guo, Lei Ma, Jianjun Zhao, ShengYong Chen

Then, we propose a novel core-failure-set guided DARTS that embeds a K-center-greedy algorithm for DARTS to select suitable corrupted failure examples to refine the model architecture.

Data Augmentation

Uncertainty Guided Depth Fusion for Spike Camera

no code implementations26 Aug 2022 Jianing Li, Jiaming Liu, Xiaobao Wei, Jiyuan Zhang, Ming Lu, Lei Ma, Li Du, Tiejun Huang, Shanghang Zhang

In this paper, we propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse the predictions of monocular and stereo depth estimation networks for spike camera.

Autonomous Driving Stereo Depth Estimation

Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation

1 code implementation22 Jul 2022 Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon

Recent studies show that test selection for DNN is a promising direction that tackles this issue by selecting minimal representative data to label and using these data to assess the model.

Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites

no code implementations14 Jun 2022 Jingyang Zhang, Peng Xue, Ran Gu, Yuning Gu, Mianxin Liu, Yongsheng Pan, Zhiming Cui, Jiawei Huang, Lei Ma, Dinggang Shen

In clinical practice, a segmentation network is often required to continually learn on a sequential data stream from multiple sites rather than a consolidated set, due to the storage cost and privacy restriction.

Continual Learning

CodeS: Towards Code Model Generalization Under Distribution Shift

2 code implementations11 Jun 2022 Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Lei Ma, Mike Papadakis, Yves Le Traon

Distribution shift has been a longstanding challenge for the reliable deployment of deep learning (DL) models due to unexpected accuracy degradation.

Benchmarking Code Classification

NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks

no code implementations24 Mar 2022 Xiaofei Xie, Tianlin Li, Jian Wang, Lei Ma, Qing Guo, Felix Juefei-Xu, Yang Liu

Inspired by software testing, a number of structural coverage criteria are designed and proposed to measure the test adequacy of DNNs.

Defect Detection DNN Testing +2

1000x Faster Camera and Machine Vision with Ordinary Devices

no code implementations23 Jan 2022 Tiejun Huang, Yajing Zheng, Zhaofei Yu, Rui Chen, Yuan Li, Ruiqin Xiong, Lei Ma, Junwei Zhao, Siwei Dong, Lin Zhu, Jianing Li, Shanshan Jia, Yihua Fu, Boxin Shi, Si Wu, Yonghong Tian

By treating vidar as spike trains in biological vision, we have further developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision, achieving high-speed object detection and tracking 1, 000x faster than human vision.

object-detection Object Detection

Uncertainty-Aware Cascaded Dilation Filtering for High-Efficiency Deraining

1 code implementation7 Jan 2022 Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Di Lin, Wei Feng, Song Wang

First, we propose the uncertainty-aware cascaded predictive filtering (UC-PFilt) that can identify the difficulties of reconstructing clean pixels via predicted kernels and remove the residual rain traces effectively.

Data Augmentation Single Image Deraining +2

Towards Understanding Quality Challenges of the Federated Learning for Neural Networks: A First Look from the Lens of Robustness

1 code implementation5 Jan 2022 Amin Eslami Abyane, Derui Zhu, Roberto Souza, Lei Ma, Hadi Hemmati

Therefore, to better understand the current quality status and challenges of these SOTA FL techniques in the presence of attacks and faults, we perform a large-scale empirical study to investigate the SOTA FL's quality from multiple angles of attacks, simulated faults (via mutation operators), and aggregation (defense) methods.

Data Poisoning Federated Learning +2

A Robust Visual Sampling Model Inspired by Receptive Field

no code implementations4 Jan 2022 Liwen Hu, Lei Ma, Dawei Weng, Tiejun Huang

More importantly, due to mimicking receptive field mechanism to collect regional information, RVSM can filter high intensity noise effectively and improves the problem that Spike camera is sensitive to noise largely.

Quantization

ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks

no code implementations26 Nov 2021 Hua Qi, Zhijie Wang, Qing Guo, Jianlang Chen, Felix Juefei-Xu, Lei Ma, Jianjun Zhao

In this work, as the first attempt, we initiate to repair DNNs by jointly optimizing the architecture and weights at a higher (i. e., block) level.

PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing

no code implementations19 Nov 2021 Yuheng Huang, Lei Ma, Yuanchun Li

Vision Transformer (ViT) is known to be highly nonlinear like other classical neural networks and could be easily fooled by both natural and adversarial patch perturbations.

GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search

1 code implementation4 Nov 2021 Shangqing Liu, Xiaofei Xie, JingKai Siow, Lei Ma, Guozhu Meng, Yang Liu

Specifically, we propose to construct graphs for the source code and queries with bidirectional GGNN (BiGGNN) to capture the local structural information of the source code and queries.

Code Search Code Summarization +3

Optical Flow Estimation for Spiking Camera

1 code implementation CVPR 2022 Liwen Hu, Rui Zhao, Ziluo Ding, Lei Ma, Boxin Shi, Ruiqin Xiong, Tiejun Huang

Further, for training SCFlow, we synthesize two sets of optical flow data for the spiking camera, SPIkingly Flying Things and Photo-realistic High-speed Motion, denoted as SPIFT and PHM respectively, corresponding to random high-speed and well-designed scenes.

Event-based vision Motion Estimation +1

SkullEngine: A Multi-stage CNN Framework for Collaborative CBCT Image Segmentation and Landmark Detection

no code implementations7 Oct 2021 Qin Liu, Han Deng, Chunfeng Lian, Xiaoyang Chen, Deqiang Xiao, Lei Ma, Xu Chen, Tianshu Kuang, Jaime Gateno, Pew-Thian Yap, James J. Xia

We propose a multi-stage coarse-to-fine CNN-based framework, called SkullEngine, for high-resolution segmentation and large-scale landmark detection through a collaborative, integrated, and scalable JSD model and three segmentation and landmark detection refinement models.

Image Segmentation Segmentation +1

CarveNet: Carving Point-Block for Complex 3D Shape Completion

no code implementations28 Jul 2021 Qing Guo, Zhijie Wang, Felix Juefei-Xu, Di Lin, Lei Ma, Wei Feng, Yang Liu

3D point cloud completion is very challenging because it heavily relies on the accurate understanding of the complex 3D shapes (e. g., high-curvature, concave/convex, and hollowed-out 3D shapes) and the unknown & diverse patterns of the partially available point clouds.

Data Augmentation Point Cloud Completion

Learning to Adversarially Blur Visual Object Tracking

1 code implementation ICCV 2021 Qing Guo, Ziyi Cheng, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yang Liu, Jianjun Zhao

In this work, we explore the robustness of visual object trackers against motion blur from a new angle, i. e., adversarial blur attack (ABA).

Object Visual Object Tracking +1

AdvFilter: Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning

no code implementations14 Jul 2021 Yihao Huang, Qing Guo, Felix Juefei-Xu, Lei Ma, Weikai Miao, Yang Liu, Geguang Pu

To this end, we first comprehensively investigate two kinds of pixel denoising methods for adversarial robustness enhancement (i. e., existing additive-based and unexplored filtering-based methods) under the loss functions of image-level and semantic-level, respectively, showing that pixel-wise filtering can obtain much higher image quality (e. g., higher PSNR) as well as higher robustness (e. g., higher accuracy on adversarial examples) than existing pixel-wise additive-based method.

Adversarial Attack Adversarial Robustness +1

DeepMix: Online Auto Data Augmentation for Robust Visual Object Tracking

no code implementations23 Apr 2021 Ziyi Cheng, Xuhong Ren, Felix Juefei-Xu, Wanli Xue, Qing Guo, Lei Ma, Jianjun Zhao

Online updating of the object model via samples from historical frames is of great importance for accurate visual object tracking.

Data Augmentation Object +1

Countering Malicious DeepFakes: Survey, Battleground, and Horizon

1 code implementation27 Feb 2021 Felix Juefei-Xu, Run Wang, Yihao Huang, Qing Guo, Lei Ma, Yang Liu

To fill this gap, in this paper, we provide a comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of DeepFake detection, with more than 318 research papers carefully surveyed.

DeepFake Detection Face Swapping +2

AttackDist: Characterizing Zero-day Adversarial Samples by Counter Attack

no code implementations1 Jan 2021 Simin Chen, Zihe Song, Lei Ma, Cong Liu, Wei Yang

We first theoretically clarify under which condition AttackDist can provide a certified detecting performance, then show that a potential application of AttackDist is distinguishing zero-day adversarial examples without knowing the mechanisms of new attacks.

Sparta: Spatially Attentive and Adversarially Robust Activations

no code implementations1 Jan 2021 Qing Guo, Felix Juefei-Xu, Changqing Zhou, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu

Moreover, comprehensive evaluations have demonstrated two important properties of our method: First, superior transferability across DNNs.

Denoising

Adversarial Rain Attack and Defensive Deraining for DNN Perception

no code implementations19 Sep 2020 Liming Zhai, Felix Juefei-Xu, Qing Guo, Xiaofei Xie, Lei Ma, Wei Feng, Shengchao Qin, Yang Liu

To defend the DNNs from the negative rain effect, we also present a defensive deraining strategy, for which we design an adversarial rain augmentation that uses mixed adversarial rain layers to enhance deraining models for downstream DNN perception.

Adversarial Attack Autonomous Driving +5

EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining

2 code implementations19 Sep 2020 Qing Guo, Jingyang Sun, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Wei Feng, Yang Liu

To fill this gap, in this paper, we regard the single-image deraining as a general image-enhancing problem and originally propose a model-free deraining method, i. e., EfficientDeRain, which is able to process a rainy image within 10~ms (i. e., around 6~ms on average), over 80 times faster than the state-of-the-art method (i. e., RCDNet), while achieving similar de-rain effects.

Data Augmentation Single Image Deraining

FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction

1 code implementation13 Jun 2020 Yihao Huang, Felix Juefei-Xu, Run Wang, Qing Guo, Lei Ma, Xiaofei Xie, Jianwen Li, Weikai Miao, Yang Liu, Geguang Pu

At this moment, GAN-based image generation methods are still imperfect, whose upsampling design has limitations in leaving some certain artifact patterns in the synthesized image.

DeepFake Detection Face Swapping +2

DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms

no code implementations13 Jun 2020 Hua Qi, Qing Guo, Felix Juefei-Xu, Xiaofei Xie, Lei Ma, Wei Feng, Yang Liu, Jianjun Zhao

As the GAN-based face image and video generation techniques, widely known as DeepFakes, have become more and more matured and realistic, there comes a pressing and urgent demand for effective DeepFakes detectors.

DeepFake Detection Face Swapping +2

Satellite-Terrestrial Channel Characterization in High-Speed Railway Environment at 22.6 GHz

no code implementations11 Jun 2020 Lei Ma, Ke Guan, Dong Yan, Danping He, Nuno R. Leonor, Bo Ai, Junhyeong Kim

In this paper, the satellite-terrestrial channel at 22. 6 GHz is characterized for a typical high-speed railway (HSR) environment.

Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning

no code implementations14 May 2020 Jianwen Sun, Tianwei Zhang, Xiaofei Xie, Lei Ma, Yan Zheng, Kangjie Chen, Yang Liu

Adversarial attacks against conventional Deep Learning (DL) systems and algorithms have been widely studied, and various defenses were proposed.

Adversarial Attack Deep Reinforcement Learning +2

Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty

no code implementations24 Apr 2020 Xiyue Zhang, Xiaofei Xie, Lei Ma, Xiaoning Du, Qiang Hu, Yang Liu, Jianjun Zhao, Meng Sun

Based on this, we propose an automated testing technique to generate multiple types of uncommon AEs and BEs that are largely missed by existing techniques.

Adversarial Attack

Byzantine-resilient Decentralized Stochastic Gradient Descent

no code implementations20 Feb 2020 Shangwei Guo, Tianwei Zhang, Han Yu, Xiaofei Xie, Lei Ma, Tao Xiang, Yang Liu

It guarantees that each benign node in a decentralized system can train a correct model under very strong Byzantine attacks with an arbitrary number of faulty nodes.

Edge-computing Image Classification

Amora: Black-box Adversarial Morphing Attack

no code implementations9 Dec 2019 Run Wang, Felix Juefei-Xu, Qing Guo, Yihao Huang, Xiaofei Xie, Lei Ma, Yang Liu

In this paper, we investigate and introduce a new type of adversarial attack to evade FR systems by manipulating facial content, called \textbf{\underline{a}dversarial \underline{mor}phing \underline{a}ttack} (a. k. a.

Adversarial Attack Dictionary Learning +3

SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking

1 code implementation ECCV 2020 Qing Guo, Xiaofei Xie, Felix Juefei-Xu, Lei Ma, Zhongguo Li, Wanli Xue, Wei Feng, Yang Liu

We identify that online object tracking poses two new challenges: 1) it is difficult to generate imperceptible perturbations that can transfer across frames, and 2) real-time trackers require the attack to satisfy a certain level of efficiency.

Adversarial Attack Video Object Tracking +2

An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms

no code implementations15 Sep 2019 Qianyu Guo, Sen Chen, Xiaofei Xie, Lei Ma, Qiang Hu, Hongtao Liu, Yang Liu, Jianjun Zhao, Xiaohong Li

However, the differences in architecture designs and implementations of existing frameworks and platforms bring new challenges for DL software development and deployment.

Adversarial Attack Adversarial Robustness +1

FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces

no code implementations13 Sep 2019 Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, Yang Liu

In recent years, generative adversarial networks (GANs) and its variants have achieved unprecedented success in image synthesis.

Face Detection Face Recognition +2

Hierarchy Neighborhood Discriminative Hashing for An Unified View of Single-Label and Multi-Label Image retrieval

no code implementations10 Jan 2019 Lei Ma, Hongliang Li, Qingbo Wu, Fanman Meng, King Ngi Ngan

Finally, we propose a hierarchy neighborhood discriminative hashing loss to unify the single-label and multilabel image retrieval problem with a one-stream deep neural network architecture.

Multi-Label Image Retrieval Retrieval +3

DeepCruiser: Automated Guided Testing for Stateful Deep Learning Systems

no code implementations13 Dec 2018 Xiaoning Du, Xiaofei Xie, Yi Li, Lei Ma, Jianjun Zhao, Yang Liu

Our in-depth evaluation on a state-of-the-art speech-to-text DL system demonstrates the effectiveness of our technique in improving quality and reliability of stateful DL systems.

Deep Learning

An Orchestrated Empirical Study on Deep Learning Frameworks and Platforms

no code implementations13 Nov 2018 Qianyu Guo, Xiaofei Xie, Lei Ma, Qiang Hu, Ruitao Feng, Li Li, Yang Liu, Jianjun Zhao, Xiaohong Li

Up to the present, it still lacks a comprehensive study on how current diverse DL frameworks and platforms influence the DL software development process.

Autonomous Driving Deep Learning

Secure Deep Learning Engineering: A Software Quality Assurance Perspective

no code implementations10 Oct 2018 Lei Ma, Felix Juefei-Xu, Minhui Xue, Qiang Hu, Sen Chen, Bo Li, Yang Liu, Jianjun Zhao, Jianxiong Yin, Simon See

Over the past decades, deep learning (DL) systems have achieved tremendous success and gained great popularity in various applications, such as intelligent machines, image processing, speech processing, and medical diagnostics.

Deep Learning

Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer

no code implementations7 Sep 2018 Alvin Chan, Lei Ma, Felix Juefei-Xu, Xiaofei Xie, Yang Liu, Yew Soon Ong

Deep neural networks (DNN), while becoming the driving force of many novel technology and achieving tremendous success in many cutting-edge applications, are still vulnerable to adversarial attacks.

Question Answering Relation

DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing

no code implementations4 Sep 2018 Xiaofei Xie, Lei Ma, Felix Juefei-Xu, Hongxu Chen, Minhui Xue, Bo Li, Yang Liu, Jianjun Zhao, Jianxiong Yin, Simon See

In company with the data explosion over the past decade, deep neural network (DNN) based software has experienced unprecedented leap and is becoming the key driving force of many novel industrial applications, including many safety-critical scenarios such as autonomous driving.

Autonomous Driving Quantization

Combinatorial Testing for Deep Learning Systems

no code implementations20 Jun 2018 Lei Ma, Fuyuan Zhang, Minhui Xue, Bo Li, Yang Liu, Jianjun Zhao, Yadong Wang

Deep learning (DL) has achieved remarkable progress over the past decade and been widely applied to many safety-critical applications.

Deep Learning Defect Detection

DeepLaser: Practical Fault Attack on Deep Neural Networks

no code implementations15 Jun 2018 Jakub Breier, Xiaolu Hou, Dirmanto Jap, Lei Ma, Shivam Bhasin, Yang Liu

As deep learning systems are widely adopted in safety- and security-critical applications, such as autonomous vehicles, banking systems, etc., malicious faults and attacks become a tremendous concern, which potentially could lead to catastrophic consequences.

Autonomous Vehicles

DeepMutation: Mutation Testing of Deep Learning Systems

4 code implementations14 May 2018 Lei Ma, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Felix Juefei-Xu, Chao Xie, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang

To do this, by sharing the same spirit of mutation testing in traditional software, we first define a set of source-level mutation operators to inject faults to the source of DL (i. e., training data and training programs).

Software Engineering

DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems

no code implementations20 Mar 2018 Lei Ma, Felix Juefei-Xu, Fuyuan Zhang, Jiyuan Sun, Minhui Xue, Bo Li, Chunyang Chen, Ting Su, Li Li, Yang Liu, Jianjun Zhao, Yadong Wang

Deep learning (DL) defines a new data-driven programming paradigm that constructs the internal system logic of a crafted neuron network through a set of training data.

Adversarial Attack Deep Learning +1

Cannot find the paper you are looking for? You can Submit a new open access paper.