Search Results for author:
Found 80 papers, 36 papers with code
An End-to-End Compression Framework Based on Convolutional Neural Networks
The second CNN, named reconstruction convolutional neural network (RecCNN), is used to reconstruct the decoded image with high-quality in the decoding end.
Active Learning for Wireless IoT Intrusion Detection
This paper discusses the human-in-the-loop active learning approach for wireless intrusion detection.
To Compress, or Not to Compress: Characterizing Deep Learning Model Compression for Embedded Inference
We experimentally show that how two mainstream compression techniques, data quantization and pruning, perform on these network architectures and the implications of compression techniques to the model storage size, inference time, energy consumption and performance metrics.
Explaining Neural Networks Semantically and Quantitatively
This paper presents a method to explain the knowledge encoded in a convolutional neural network (CNN) quantitatively and semantically.
Interpretable Complex-Valued Neural Networks for Privacy Protection
Previous studies have found that an adversary attacker can often infer unintended input information from intermediate-layer features.
Semi-parametric Bayesian variable selection for gene-environment interactions
Existing Bayesian methods for G$\times$E interaction studies are challenged by the high-dimensional nature of the study and the complexity of environmental influences.
Methodology
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive {\em uncertainty}.
Likelihood Ratios for Out-of-Distribution Detection
We propose a likelihood ratio method for deep generative models which effectively corrects for these confounding background statistics.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders
We show results for extractive and human baselines to demonstrate a large abstractive gap in performance.
Interpreting and Disentangling Feature Components of Various Complexity from DNNs
This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN.
Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
Accurate estimation of predictive uncertainty in modern neural networks is critical to achieve well calibrated predictions and detect out-of-distribution (OOD) inputs.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
A Unified Approach to Interpreting and Boosting Adversarial Transferability
We discover and prove the negative correlation between the adversarial transferability and the interaction inside adversarial perturbations.
HM-ANN: Efficient Billion-Point Nearest Neighbor Search on Heterogeneous Memory
The emergence of heterogeneous memory (HM) brings a solution to significantly increase memory capacity and break the above tradeoff: Using HM, billions of data points can be placed in the main memory on a single machine without using any data compression.
Receptivity and stability of hypersonic leading-edge sweep flows around a blunt body
We establish an adjoint-based bi-orthogonal eigenfunction system to address the receptivity problem of such flows to any external forces and boundary perturbations.
Fluid Dynamics
Analytic critical points of charged Renyi entropies from hyperbolic black holes
The first system is the Reissner-Nordstrom-AdS$_5$ black hole, which has finite entropy at zero temperature.
High Energy Physics - Theory General Relativity and Quantum Cosmology
Understanding, Analyzing, and Optimizing the Complexity of Deep Models
Based on the proposed metrics, we analyze two typical phenomena of the change of the transformation complexity during the training process, and explore the ceiling of a DNN’s complexity.
Disentanglement, Visualization and Analysis of Complex Features in DNNs
This paper aims to define, visualize, and analyze the feature complexity that is learned by a DNN.
Towards A Unified Understanding and Improving of Adversarial Transferability
We discover and prove the negative correlation between the adversarial transferability and the interaction inside adversarial perturbations.
ZeRO-Offload: Democratizing Billion-Scale Model Training
By combining compute and memory efficiency with ease-of-use, ZeRO-Offload democratizes large-scale model training making it accessible to even data scientists with access to just a single GPU.
Identifying Gene-environment interactions with robust marginal Bayesian variable selection
In this study, we propose a novel marginal Bayesian variable selection method for G$\times$E studies.
Variable Selection
Methodology
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
Uniting Heterogeneity, Inductiveness, and Efficiency for Graph Representation Learning
In WIDEN, we propose a novel inductive, meta path-free message passing scheme that packs up heterogeneous node features with their associated edges from both low- and high-order neighbor nodes.
Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier.
Learning to Remove: Towards Isotropic Pre-trained BERT Embedding
We train the weights on word similarity tasks and show that processed embedding is more isotropic.
Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning
However, grasping distinguishable skills for some tasks with non-unique optima can be essential for further improving its learning efficiency and performance, which may lead to a multimodal policy represented as a mixture-of-experts (MOE).
Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
Masking some input variables of a deep neural network (DNN) and computing output changes on the masked input sample represent a typical way to compute attributions of input variables in the sample.
Exploring the Limits of Out-of-Distribution Detection
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks.
Ranked #2 on
Out-of-Distribution Detection
on CIFAR-10 vs CIFAR-100
(using extra training data)
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
In this paper we introduce Uncertainty Baselines: high-quality implementations of standard and state-of-the-art deep learning methods on a variety of tasks.
A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks.
A Game-Theoretic Taxonomy of Visual Concepts in DNNs
In this paper, we rethink how a DNN encodes visual concepts of different complexities from a new perspective, i. e. the game-theoretic multi-order interactions between pixels in an image.
Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network
However, they often assume in the prior that the topics at each layer are independently drawn from the Dirichlet distribution, ignoring the dependencies between the topics both at the same layer and across different layers.
Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI
This workshop pays a special interest in theoretic foundations, limitations, and new application trends in the scope of XAI.
Towards a Game-Theoretic View of Baseline Values in the Shapley Value
In the computation of Shapley values, people usually set an input variable to its baseline value to represent the absence of this variable.
Cellular Network Radio Propagation Modeling with Deep Convolutional Neural Networks
Deterministic models, such as ray tracing based on physical laws of wave propagation, are more accurate and site specific.
ESOD:Edge-based Task Scheduling for Object Detection
In detail, we train a DNN model (termed as pre-model) to predict which object detection model to use for the coming task and offloads to which edge servers by physical characteristics of the image task (e. g., brightness, saturation).
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, \emph{i. e.} the view of multi-order interactions between input variables of DNNs.
Defining and Quantifying the Emergence of Sparse Concepts in DNNs
This paper aims to illustrate the concept-emerging phenomenon in a trained DNN.
Towards a Unified Game-Theoretic View of Adversarial Perturbations and Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
Trap of Feature Diversity in the Learning of MLPs
In this paper, we focus on a typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs), and we aim to explain the reason for the decrease of feature diversity in the first phase.
MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment
Large-scale Bundle Adjustment (BA) requires massive memory and computation resources which are difficult to be fulfilled by existing BA libraries.
A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning
Gradient-based Meta-RL (GMRL) refers to methods that maintain two-level optimisation procedures wherein the outer-loop meta-learner guides the inner-loop gradient-based reinforcement learner to achieve fast adaptations.
DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations
AI-aided drug discovery (AIDD) is gaining increasing popularity due to its promise of making the search for new pharmaceuticals quicker, cheaper and more efficient.
Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding
We define a maximum traceable distance metric, through which we learn to what extent the text contrastive learning benefits from the historical information of negative samples.
A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness
The most popular approaches to estimate predictive uncertainty in deep learning are methods that combine predictions from multiple neural networks, such as Bayesian neural networks (BNNs) and deep ensembles.
Towards Theoretical Analysis of Transformation Complexity of ReLU DNNs
Based on the proposed metrics, we analyze two typical phenomena of the change of the transformation complexity during the training process, and explore the ceiling of a DNN's complexity.
Pluralistic Image Completion with Probabilistic Mixture-of-Experts
Second, the constraints for diversity are designed to be task-agnostic, which causes the constraints to not work well.
Why Adversarial Training of ReLU Networks Is Difficult?
We also prove that adversarial training tends to strengthen the influence of unconfident input samples with large gradient norms in an exponential manner.
Defense Against Gradient Leakage Attacks via Learning to Obscure Data
However, federated learning is still under the risk of privacy leakage because of the existence of attackers who deliberately conduct gradient leakage attacks to reconstruct the client data.
Plex: Towards Reliability using Pretrained Large Model Extensions
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures.
Proving Common Mechanisms Shared by Twelve Methods of Boosting Adversarial Transferability
This paper summarizes the common mechanism shared by twelve previous transferability-boosting methods in a unified view, i. e., these methods all reduce game-theoretic interactions between regional adversarial perturbations.
Color Image Edge Detection using Multi-scale and Multi-directional Gabor filter
Second, a set of Gabor filters are used to smooth the input images and the color edge strength maps are extracted, which are fused into a new ESM with the noise robustness and accurate edge extraction.
Exemplar-Based Image Colorization with A Learning Framework
This method can be viewed as a hybrid of exemplar-based and learning-based method, and it decouples the colorization process and learning process so as to generate various color styles for the same gray image.
Out-of-Distribution Detection and Selective Generation for Conditional Language Models
Furthermore, the space of potential low-quality outputs is larger as arbitrary text can be generated and it is important to know when to trust the generated output.
Abstractive Text Summarization
Out-of-Distribution Detection
+1
Probabilistic Categorical Adversarial Attack & Adversarial Training
To tackle this problem, we propose Probabilistic Categorical Adversarial Attack (PCAA), which transfers the discrete optimization problem to a continuous problem that can be solved efficiently by Projected Gradient Descent.
Transferable Unlearnable Examples
The unlearnable strategies have been introduced to prevent third parties from training on the data without permission.
TorchOpt: An Efficient Library for Differentiable Optimization
TorchOpt further provides a high-performance distributed execution runtime.
Improving Zero-shot Generalization and Robustness of Multi-modal Models
We also show that our method improves across ImageNet shifted datasets, four other datasets, and other model architectures such as LiT.
Improving the Robustness of Summarization Models by Detecting and Removing Input Noise
We present a large empirical study quantifying the sometimes severe loss in performance (up to 12 ROUGE-1 points) from different types of input noise for a range of datasets and model sizes.
A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models
In particular, we ask "Given a large pool of prompts, can we automatically score the prompts and ensemble those that are most suitable for a particular downstream dataset, without needing access to labeled validation data?".
On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Study
Modern deep models for summarization attains impressive benchmark performance, but they are prone to generating miscalibrated predictive uncertainty.
Second-order Anisotropic Gaussian Directional Derivative Filters for Blob Detection
Interest point detection methods have received increasing attention and are widely used in computer vision tasks such as image retrieval and 3D reconstruction.
DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models
By detecting the watermark from generated images, copyright infringement can be exposed with evidence.
Building One-class Detector for Anything: Open-vocabulary Zero-shot OOD Detection Using Text-image Models
Despite considerable effort, the problem remains significantly challenging in deep learning models due to their propensity to output over-confident predictions for OOD inputs.
Inflated 3D Convolution-Transformer for Weakly-supervised Carotid Stenosis Grading with Ultrasound Videos
First, to avoid the requirement of laborious and unreliable annotation, we propose a novel and effective video classification network for weakly-supervised CSG.
Morse Neural Networks for Uncertainty Quantification
We introduce a new deep generative model useful for uncertainty quantification: the Morse neural network, which generalizes the unnormalized Gaussian densities to have modes of high-dimensional submanifolds instead of just discrete points.
Retrieve-Rewrite-Answer: A KG-to-Text Enhanced LLMs Framework for Knowledge Graph Question Answering
To this end, we propose an answer-sensitive KG-to-Text approach that can transform KG knowledge into well-textualized statements most informative for KGQA.
On the Generalization of Training-based ChatGPT Detection Methods
ChatGPT is one of the most popular language models which achieve amazing performance on various natural language tasks.
FT-Shield: A Watermark Against Unauthorized Fine-tuning in Text-to-Image Diffusion Models
Text-to-image generative models based on latent diffusion models (LDM) have demonstrated their outstanding ability in generating high-quality and high-resolution images according to language prompt.
Exploring Memorization in Fine-tuned Language Models
In this work, we conduct the first comprehensive analysis to explore language models' (LMs) memorization during fine-tuning across tasks.
QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Models
We show, for the first time, that the majority of inference computations for large generative models such as LLaMA, OPT, and Falcon can be performed with both weights and activations being cast to 4 bits, in a way that leads to practical speedups, while at the same time maintaining good accuracy.
Universal Self-Consistency for Large Language Model Generation
Self-consistency with chain-of-thought prompting (CoT) has demonstrated remarkable performance gains on various challenging tasks, by utilizing multiple reasoning paths sampled from large language models (LLMs).
Self-Evaluation Improves Selective Generation in Large Language Models
Safe deployment of large language models (LLMs) may benefit from a reliable method for assessing their generated content to determine when to abstain or to selectively generate.
Gemini: A Family of Highly Capable Multimodal Models
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding.
Ranked #1 on
Multi-task Language Understanding
on MMLU
(using extra training data)
Superiority of Multi-Head Attention in In-Context Linear Regression
We present a theoretical analysis of the performance of transformer with softmax attention in in-context learning with linear regression tasks.
Copyright Protection in Generative AI: A Technical Perspective
We examine from two distinct viewpoints: the copyrights pertaining to the source data held by the data owners and those of the generative models maintained by the model builders.
A novel spatial-frequency domain network for zero-shot incremental learning
The proposed SFDNet has the ability to effectively extract spatial-frequency feature representation from input images, improve the accuracy of image classification, and fundamentally alleviate catastrophic forgetting.
Identifying Semantic Induction Heads to Understand In-Context Learning
Although large language models (LLMs) have demonstrated remarkable performance, the lack of transparency in their inference logic raises concerns about their trustworthiness.
The Good and The Bad: Exploring Privacy Issues in Retrieval-Augmented Generation (RAG)
In this work, we conduct extensive empirical studies with novel attack methods, which demonstrate the vulnerability of RAG systems on leaking the private retrieval database.
Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention
Recent advancements in text-to-image diffusion models have demonstrated their remarkable capability to generate high-quality images from textual prompts.
Construction of Functional Materials Knowledge Graph in Multidisciplinary Materials Science via Large Language Model
The convergence of materials science and artificial intelligence has unlocked new opportunities for gathering, analyzing, and generating novel materials sourced from extensive scientific literature.
