Search Results for author:
Found 95 papers, 31 papers with code
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward
To bridge this gap, we conduct in this work a comprehensive evaluation of the effectiveness of existing online safety analysis methods on LLMs.
LRR: Language-Driven Resamplable Continuous Representation against Adversarial Tracking Attacks
To achieve high accuracy on both clean and adversarial data, we propose building a spatial-temporal continuous representation using the semantic text guidance of the object of interest.
PoLLMgraph: Unraveling Hallucinations in Large Language Models via State Transition Dynamics
Despite tremendous advancements in large language models (LLMs) over recent years, a notably urgent challenge for their practical deployment is the phenomenon of hallucination, where the model fabricates facts and produces non-factual statements.
PromptCodec: High-Fidelity Neural Speech Codec using Disentangled Representation Learning based Adaptive Feature-aware Prompt Encoders
Neural speech codec has recently gained widespread attention in generative speech modeling domains, like voice conversion, text-to-speech synthesis, etc.
Spike-NeRF: Neural Radiance Field Based On Spike Camera
As a neuromorphic sensor with high temporal resolution, spike cameras offer notable advantages over traditional cameras in high-speed vision applications such as high-speed optical estimation, depth estimation, and object tracking.
DEEP-ICL: Definition-Enriched Experts for Language Model In-Context Learning
It has long been assumed that the sheer number of parameters in large language models (LLMs) drives in-context learning (ICL) capabilities, enabling remarkable performance improvements by leveraging task-specific demonstrations.
PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinement
However, prompting remains challenging for novice users due to the complexity of the stable diffusion model and the non-trivial efforts required for iteratively editing and refining the text prompts.
m2mKD: Module-to-Module Knowledge Distillation for Modular Transformers
Leveraging the knowledge from monolithic models, using techniques such as knowledge distillation, is likely to facilitate the training of modular models and enable them to integrate knowledge from multiple models pretrained on diverse sources.
CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models
The advancement of large language models (LLMs) has enhanced the ability to generalize across a wide range of unseen natural language processing (NLP) tasks through instruction-following.
Large Language Models Based Fuzzing Techniques: A Survey
In the modern era where software plays a pivotal role, software security and vulnerability analysis have become essential for software development.
Learning to Robustly Reconstruct Low-light Dynamic Scenes from Spike Streams
In this paper, we propose a bidirectional recurrent-based reconstruction framework, including a Light-Robust Representation (LR-Rep) and a fusion module, to better handle such extreme conditions.
Beyond Fidelity: Explaining Vulnerability Localization of Learning-based Detectors
Vulnerability detectors based on deep learning (DL) models have proven their effectiveness in recent years.
Code Ownership in Open-Source AI Software Security
As open-source AI software projects become an integral component in the AI software development, it is critical to develop a novel methods to ensure and measure the security of the open-source projects for developers.
T3D: Towards 3D Medical Image Understanding through Vision-Language Pre-training
This underlines T3D's potential in representation learning for 3D medical image analysis.
SynFundus-1M: A High-quality Million-scale Synthetic fundus images Dataset with Fifteen Types of Annotation
Large-scale public datasets with high-quality annotations are rarely available for intelligent medical imaging research, due to data privacy concerns and the cost of annotations.
Domain Generalization of 3D Object Detection by Density-Resampling
Specifically, from the perspective of data augmentation, we design a universal physical-aware density-based data augmentation (PDDA) method to mitigate the performance loss stemming from diverse point densities.
LUNA: A Model-Based Universal Analysis Framework for Large Language Models
the trustworthiness perspective, is bound to and enriches the abstract model with semantics, which enables more detailed analysis applications for diverse purposes.
Self-Refined Large Language Model as Automated Reward Function Designer for Deep Reinforcement Learning in Robotics
Then, the performance of the reward function is assessed, and the results are presented back to the LLM for guiding its self-refinement process.
ASTER: Automatic Speech Recognition System Accessibility Testing for Stutterers
To improve the accessibility of ASR systems for stutterers, we need to expose and analyze the failures of ASR systems on stuttering speech.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
ISR-LLM: Iterative Self-Refined Large Language Model for Long-Horizon Sequential Task Planning
Motivated by the substantial achievements observed in Large Language Models (LLMs) in the field of natural language processing, recent research has commenced investigations into the application of LLMs for complex, long-horizon sequential task planning challenges in robotics.
MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognition
Despite notable progress, speech emotion recognition (SER) remains challenging due to the intricate and ambiguous nature of speech emotion, particularly in wild world.
One-shot Joint Extraction, Registration and Segmentation of Neuroimaging Data
Brain extraction, registration and segmentation are indispensable preprocessing steps in neuroimaging studies.
Accurate 3D Prediction of Missing Teeth in Diverse Patterns for Precise Dental Implant Planning
This study presents a novel framework for accurate prediction of missing teeth in different patterns, facilitating digital implant planning.
Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models
In particular, we experiment with twelve uncertainty estimation methods and four LLMs on four prominent natural language processing (NLP) tasks to investigate to what extent uncertainty estimation techniques could help characterize the prediction risks of LLMs.
CephGPT-4: An Interactive Multimodal Cephalometric Measurement and Diagnostic System with Visual Large Language Model
Firstly, a multimodal orthodontic medical dataset is constructed, comprising cephalometric images and doctor-patient dialogue data, with automatic analysis of cephalometric landmarks using U-net and generation of diagnostic reports.
Mitigating Communication Costs in Neural Networks: The Role of Dendritic Nonlinearity
Our comprehension of biological neuronal networks has profoundly influenced the evolution of artificial neural networks (ANNs).
GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition
Contrastive cross-modality pretraining has recently exhibited impressive success in diverse fields, whereas there is limited research on their merits in speech emotion recognition (SER).
Benchmarking Robustness of AI-Enabled Multi-sensor Fusion Systems: Challenges and Opportunities
Multi-Sensor Fusion (MSF) based perception systems have been the foundation in supporting many industrial applications and domains, such as self-driving cars, robotic arms, and unmanned aerial vehicles.
Neuron Activation Coverage: Rethinking Out-of-distribution Detection and Generalization
The out-of-distribution (OOD) problem generally arises when neural networks encounter data that significantly deviates from the training data distribution, i. e., in-distribution (InD).
Ranked #2 on
Out-of-Distribution Detection
on ImageNet-1k vs Textures
(AUROC metric)
Is Model Attention Aligned with Human Attention? An Empirical Study on Large Language Models for Code Generation
Through a quantitative experiment and a user study, we confirmed that, among twelve different attention computation methods, attention computed by the perturbation-based method is most aligned with human attention and is constantly favored by human programmers.
Med-UniC: Unifying Cross-Lingual Medical Vision-Language Pre-Training by Diminishing Bias
Med-UniC reaches superior performance across 5 medical image tasks and 10 datasets encompassing over 30 diseases, offering a versatile framework for unifying multi-modal medical data within diverse linguistic communities.
Deep Learning based Forecasting: a case study from the online fashion industry
These include the volume of data, the irregularity, the high amount of turn-over in the catalog and the fixed inventory assumption.
Evading DeepFake Detectors via Adversarial Statistical Consistency
Second, we find that the statistical differences between natural and DeepFake images are positively associated with the distribution shifting between the two kinds of images, and we propose to use a distribution-aware loss to guide the optimization of different degradations.
Construction of unbiased dental template and parametric dental model for precision digital dentistry
In this study, we develop an unbiased dental template by constructing an accurate dental atlas from CBCT images with guidance of teeth segmentation.
Spike Stream Denoising via Spike Camera Simulation
Based on our noise model, the first benchmark for spike stream denoising is proposed which includes clear (noisy) spike stream.
HYBRIDFORMER: improving SqueezeFormer with hybrid attention and NSR mechanism
SqueezeFormer has recently shown impressive performance in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Structure-aware registration network for liver DCE-CT images
In addition, such paired organ segmentations are not always available in DCE-CT images due to the flow of contrast agents.
DeepLens: Interactive Out-of-distribution Data Detection in NLP Models
In this work, we propose DeepLens, an interactive system that helps users detect and explore OOD issues in massive text corpora.
DeepSeer: Interactive RNN Explanation and Debugging via State Abstraction
The core of DeepSeer is a state abstraction method that bundles semantically similar hidden states in an RNN model and abstracts the model as a finite state machine.
Neural Episodic Control with State Abstraction
Generally, episodic control-based approaches are solutions that leverage highly-rewarded past experiences to improve sample efficiency of DRL algorithms.
An Exploratory Study of AI System Risk Assessment from the Lens of Data Distribution and Uncertainty
More importantly, it also lacks systematic investigation on how to perform the risk assessment for AI systems from unit level to system level.
AI-driven Mobile Apps: an Explorative Study
In this paper, we conduct the most extensive empirical study on 56, 682 published AI apps from three perspectives: dataset characteristics, development issues, and user feedback and privacy.
FAF: A novel multimodal emotion recognition approach integrating face, body and text
In this paper, we developed a large multimodal emotion dataset, named "HED" dataset, to facilitate the emotion recognition task, and accordingly propose a multimodal emotion recognition method.
Common Corruption Robustness of Point Cloud Detectors: Benchmark and Enhancement
Then, for the first attempt, we construct a benchmark based on the physical-aware common corruptions for point cloud detectors, which contains a total of 1, 122, 150 examples covering 7, 481 scenes, 25 common corruption types, and 6 severities.
Decompiling x86 Deep Neural Network Executables
BTD takes DNN executables and outputs full model specifications, including types of DNN operators, network topology, dimensions, and parameters that are (nearly) identical to those of the input models.
DARTSRepair: Core-failure-set Guided DARTS for Network Robustness to Common Corruptions
Then, we propose a novel core-failure-set guided DARTS that embeds a K-center-greedy algorithm for DARTS to select suitable corrupted failure examples to refine the model architecture.
Uncertainty Guided Depth Fusion for Spike Camera
In this paper, we propose a novel Uncertainty-Guided Depth Fusion (UGDF) framework to fuse the predictions of monocular and stereo depth estimation networks for spike camera.
Aries: Efficient Testing of Deep Neural Networks via Labeling-Free Accuracy Estimation
Recent studies show that test selection for DNN is a promising direction that tackles this issue by selecting minimal representative data to label and using these data to assess the model.
Learning towards Synchronous Network Memorizability and Generalizability for Continual Segmentation across Multiple Sites
In clinical practice, a segmentation network is often required to continually learn on a sequential data stream from multiple sites rather than a consolidated set, due to the storage cost and privacy restriction.
CodeS: Towards Code Model Generalization Under Distribution Shift
Distribution shift has been a longstanding challenge for the reliable deployment of deep learning (DL) models due to unexpected accuracy degradation.
NPC: Neuron Path Coverage via Characterizing Decision Logic of Deep Neural Networks
Inspired by software testing, a number of structural coverage criteria are designed and proposed to measure the test adequacy of DNNs.
1000x Faster Camera and Machine Vision with Ordinary Devices
By treating vidar as spike trains in biological vision, we have further developed a spiking neural network-based machine vision system that combines the speed of the machine and the mechanism of biological vision, achieving high-speed object detection and tracking 1, 000x faster than human vision.
Uncertainty-Aware Cascaded Dilation Filtering for High-Efficiency Deraining
First, we propose the uncertainty-aware cascaded predictive filtering (UC-PFilt) that can identify the difficulties of reconstructing clean pixels via predicted kernels and remove the residual rain traces effectively.
Towards Understanding Quality Challenges of the Federated Learning for Neural Networks: A First Look from the Lens of Robustness
Therefore, to better understand the current quality status and challenges of these SOTA FL techniques in the presence of attacks and faults, we perform a large-scale empirical study to investigate the SOTA FL's quality from multiple angles of attacks, simulated faults (via mutation operators), and aggregation (defense) methods.
A Robust Visual Sampling Model Inspired by Receptive Field
More importantly, due to mimicking receptive field mechanism to collect regional information, RVSM can filter high intensity noise effectively and improves the problem that Spike camera is sensitive to noise largely.
ArchRepair: Block-Level Architecture-Oriented Repairing for Deep Neural Networks
In this work, as the first attempt, we initiate to repair DNNs by jointly optimizing the architecture and weights at a higher (i. e., block) level.
PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing
Vision Transformer (ViT) is known to be highly nonlinear like other classical neural networks and could be easily fooled by both natural and adversarial patch perturbations.
GraphSearchNet: Enhancing GNNs via Capturing Global Dependencies for Semantic Code Search
Specifically, we propose to construct graphs for the source code and queries with bidirectional GGNN (BiGGNN) to capture the local structural information of the source code and queries.
Optical Flow Estimation for Spiking Camera
Further, for training SCFlow, we synthesize two sets of optical flow data for the spiking camera, SPIkingly Flying Things and Photo-realistic High-speed Motion, denoted as SPIFT and PHM respectively, corresponding to random high-speed and well-designed scenes.
SkullEngine: A Multi-stage CNN Framework for Collaborative CBCT Image Segmentation and Landmark Detection
We propose a multi-stage coarse-to-fine CNN-based framework, called SkullEngine, for high-resolution segmentation and large-scale landmark detection through a collaborative, integrated, and scalable JSD model and three segmentation and landmark detection refinement models.
A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning
In the training stage, the simulator maps jaw deformities of a patient bone to a normal bone to generate a simulated deformed bone.
CarveNet: Carving Point-Block for Complex 3D Shape Completion
3D point cloud completion is very challenging because it heavily relies on the accurate understanding of the complex 3D shapes (e. g., high-curvature, concave/convex, and hollowed-out 3D shapes) and the unknown & diverse patterns of the partially available point clouds.
Learning to Adversarially Blur Visual Object Tracking
In this work, we explore the robustness of visual object trackers against motion blur from a new angle, i. e., adversarial blur attack (ABA).
AdvFilter: Predictive Perturbation-aware Filtering against Adversarial Attack via Multi-domain Learning
To this end, we first comprehensively investigate two kinds of pixel denoising methods for adversarial robustness enhancement (i. e., existing additive-based and unexplored filtering-based methods) under the loss functions of image-level and semantic-level, respectively, showing that pixel-wise filtering can obtain much higher image quality (e. g., higher PSNR) as well as higher robustness (e. g., higher accuracy on adversarial examples) than existing pixel-wise additive-based method.
DeepMix: Online Auto Data Augmentation for Robust Visual Object Tracking
Online updating of the object model via samples from historical frames is of great importance for accurate visual object tracking.
Countering Malicious DeepFakes: Survey, Battleground, and Horizon
To fill this gap, in this paper, we provide a comprehensive overview and detailed analysis of the research work on the topic of DeepFake generation, DeepFake detection as well as evasion of DeepFake detection, with more than 318 research papers carefully surveyed.
Sparta: Spatially Attentive and Adversarially Robust Activations
Moreover, comprehensive evaluations have demonstrated two important properties of our method: First, superior transferability across DNNs.
AttackDist: Characterizing Zero-day Adversarial Samples by Counter Attack
We first theoretically clarify under which condition AttackDist can provide a certified detecting performance, then show that a potential application of AttackDist is distinguishing zero-day adversarial examples without knowing the mechanisms of new attacks.
DeepRepair: Style-Guided Repairing for DNNs in the Real-world Operational Environment
In this paper, we propose a style-guided data augmentation for repairing DNN in the operational environment.
Human Perception-based Evaluation Criterion for Ultra-high Resolution Cell Membrane Segmentation
Computer vision technology is widely used in biological and medical data analysis and understanding.
Adversarial Rain Attack and Defensive Deraining for DNN Perception
To defend the DNNs from the negative rain effect, we also present a defensive deraining strategy, for which we design an adversarial rain augmentation that uses mixed adversarial rain layers to enhance deraining models for downstream DNN perception.
EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining
To fill this gap, in this paper, we regard the single-image deraining as a general image-enhancing problem and originally propose a model-free deraining method, i. e., EfficientDeRain, which is able to process a rainy image within 10~ms (i. e., around 6~ms on average), over 80 times faster than the state-of-the-art method (i. e., RCDNet), while achieving similar de-rain effects.
DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms
As the GAN-based face image and video generation techniques, widely known as DeepFakes, have become more and more matured and realistic, there comes a pressing and urgent demand for effective DeepFakes detectors.
FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction
At this moment, GAN-based image generation methods are still imperfect, whose upsampling design has limitations in leaving some certain artifact patterns in the synthesized image.
Satellite-Terrestrial Channel Characterization in High-Speed Railway Environment at 22.6 GHz
In this paper, the satellite-terrestrial channel at 22. 6 GHz is characterized for a typical high-speed railway (HSR) environment.
An Auto-Context Deformable Registration Network for Infant Brain MRI
Deformable image registration is fundamental to longitudinal and population analysis.
Stealthy and Efficient Adversarial Attacks against Deep Reinforcement Learning
Adversarial attacks against conventional Deep Learning (DL) systems and algorithms have been widely studied, and various defenses were proposed.
Towards Characterizing Adversarial Defects of Deep Learning Software from the Lens of Uncertainty
Based on this, we propose an automated testing technique to generate multiple types of uncommon AEs and BEs that are largely missed by existing techniques.
Byzantine-resilient Decentralized Stochastic Gradient Descent
It guarantees that each benign node in a decentralized system can train a correct model under very strong Byzantine attacks with an arbitrary number of faulty nodes.
Watch out! Motion is Blurring the Vision of Your Deep Neural Networks
Besides, the attack is further enhanced by adaptively tuning the translations of object and background.
Amora: Black-box Adversarial Morphing Attack
In this paper, we investigate and introduce a new type of adversarial attack to evade FR systems by manipulating facial content, called \textbf{\underline{a}dversarial \underline{mor}phing \underline{a}ttack} (a. k. a.
SPARK: Spatial-aware Online Incremental Attack Against Visual Tracking
We identify that online object tracking poses two new challenges: 1) it is difficult to generate imperceptible perturbations that can transfer across frames, and 2) real-time trackers require the attack to satisfy a certain level of efficiency.
An Empirical Study towards Characterizing Deep Learning Development and Deployment across Different Frameworks and Platforms
However, the differences in architecture designs and implementations of existing frameworks and platforms bring new challenges for DL software development and deployment.
FakeSpotter: A Simple yet Robust Baseline for Spotting AI-Synthesized Fake Faces
In recent years, generative adversarial networks (GANs) and its variants have achieved unprecedented success in image synthesis.
Machine Learning Testing: Survey, Landscapes and Horizons
This paper provides a comprehensive survey of Machine Learning Testing (ML testing) research.
Hierarchy Neighborhood Discriminative Hashing for An Unified View of Single-Label and Multi-Label Image retrieval
Finally, we propose a hierarchy neighborhood discriminative hashing loss to unify the single-label and multilabel image retrieval problem with a one-stream deep neural network architecture.
DeepCruiser: Automated Guided Testing for Stateful Deep Learning Systems
Our in-depth evaluation on a state-of-the-art speech-to-text DL system demonstrates the effectiveness of our technique in improving quality and reliability of stateful DL systems.
An Orchestrated Empirical Study on Deep Learning Frameworks and Platforms
Up to the present, it still lacks a comprehensive study on how current diverse DL frameworks and platforms influence the DL software development process.
Secure Deep Learning Engineering: A Software Quality Assurance Perspective
Over the past decades, deep learning (DL) systems have achieved tremendous success and gained great popularity in various applications, such as intelligent machines, image processing, speech processing, and medical diagnostics.
Metamorphic Relation Based Adversarial Attacks on Differentiable Neural Computer
Deep neural networks (DNN), while becoming the driving force of many novel technology and achieving tremendous success in many cutting-edge applications, are still vulnerable to adversarial attacks.
DeepHunter: Hunting Deep Neural Network Defects via Coverage-Guided Fuzzing
In company with the data explosion over the past decade, deep neural network (DNN) based software has experienced unprecedented leap and is becoming the key driving force of many novel industrial applications, including many safety-critical scenarios such as autonomous driving.
Combinatorial Testing for Deep Learning Systems
Deep learning (DL) has achieved remarkable progress over the past decade and been widely applied to many safety-critical applications.
DeepLaser: Practical Fault Attack on Deep Neural Networks
As deep learning systems are widely adopted in safety- and security-critical applications, such as autonomous vehicles, banking systems, etc., malicious faults and attacks become a tremendous concern, which potentially could lead to catastrophic consequences.
DeepMutation: Mutation Testing of Deep Learning Systems
To do this, by sharing the same spirit of mutation testing in traditional software, we first define a set of source-level mutation operators to inject faults to the source of DL (i. e., training data and training programs).
Software Engineering
DeepGauge: Multi-Granularity Testing Criteria for Deep Learning Systems
Deep learning (DL) defines a new data-driven programming paradigm that constructs the internal system logic of a crafted neuron network through a set of training data.
