Search Results for author:
Found 59 papers, 26 papers with code
A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models
In particular, we ask "Given a large pool of prompts, can we automatically score the prompts and ensemble those that are most suitable for a particular downstream dataset, without needing access to labeled validation data?".
Improving the Robustness of Summarization Models by Detecting and Removing Input Noise
We present a large empirical study quantifying the sometimes severe loss in performance (up to 12 ROUGE-1 points) from different types of input noise for a range of datasets and model sizes.
Improving Zero-shot Generalization and Robustness of Multi-modal Models
We also show that our method improves across ImageNet shifted datasets and other model architectures such as LiT.
TorchOpt: An Efficient Library for Differentiable Optimization
TorchOpt further provides a high-performance distributed execution runtime.
Transferable Unlearnable Examples
The unlearnable strategies have been introduced to prevent third parties from training on the data without permission.
Probabilistic Categorical Adversarial Attack & Adversarial Training
To tackle this problem, we propose Probabilistic Categorical Adversarial Attack (PCAA), which transfers the discrete optimization problem to a continuous problem that can be solved efficiently by Projected Gradient Descent.
Out-of-Distribution Detection and Selective Generation for Conditional Language Models
Furthermore, the space of potential low-quality outputs is larger as arbitrary text can be generated and it is important to know when to trust the generated output.
Abstractive Text Summarization
Out-of-Distribution Detection
+1
Exemplar-Based Image Colorization with A Learning Framework
This method can be viewed as a hybrid of exemplar-based and learning-based method, and it decouples the colorization process and learning process so as to generate various color styles for the same gray image.
Color Image Edge Detection using Multi-scale and Multi-directional Gabor filter
Second, a set of Gabor filters are used to smooth the input images and the color edge strength maps are extracted, which are fused into a new ESM with the noise robustness and accurate edge extraction.
Proving Common Mechanisms Shared by Twelve Methods of Boosting Adversarial Transferability
This paper summarizes the common mechanism shared by twelve previous transferability-boosting methods in a unified view, i. e., these methods all reduce game-theoretic interactions between regional adversarial perturbations.
Plex: Towards Reliability using Pretrained Large Model Extensions
A recent trend in artificial intelligence is the use of pretrained models for language and vision tasks, which have achieved extraordinary performance but also puzzling failures.
Defense Against Gradient Leakage Attacks via Learning to Obscure Data
However, federated learning is still under the risk of privacy leakage because of the existence of attackers who deliberately conduct gradient leakage attacks to reconstruct the client data.
Why Adversarial Training of ReLU Networks Is Difficult?
We also prove that adversarial training tends to strengthen the influence of unconfident input samples with large gradient norms in an exponential manner.
Pluralistic Image Completion with Probabilistic Mixture-of-Experts
Second, the constraints for diversity are designed to be task-agnostic, which causes the constraints to not work well.
Towards Theoretical Analysis of Transformation Complexity of ReLU DNNs
Based on the proposed metrics, we analyze two typical phenomena of the change of the transformation complexity during the training process, and explore the ceiling of a DNN's complexity.
A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awareness
The most popular approaches to estimate predictive uncertainty in deep learning are methods that combine predictions from multiple neural networks, such as Bayesian neural networks (BNNs) and deep ensembles.
Exploring the Impact of Negative Samples of Contrastive Learning: A Case Study of Sentence Embedding
We define a maximum traceable distance metric, through which we learn to what extent the text contrastive learning benefits from the historical information of negative samples.
DrugOOD: Out-of-Distribution (OOD) Dataset Curator and Benchmark for AI-aided Drug Discovery -- A Focus on Affinity Prediction Problems with Noise Annotations
AI-aided drug discovery (AIDD) is gaining increasing popularity due to its promise of making the search for new pharmaceuticals quicker, cheaper and more efficient.
A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning
Gradient-based Meta-RL (GMRL) refers to methods that maintain two-level optimisation procedures wherein the outer-loop meta-learner guides the inner-loop gradient-based reinforcement learner to achieve fast adaptations.
MegBA: A GPU-Based Distributed Library for Large-Scale Bundle Adjustment
Large-scale Bundle Adjustment (BA) requires massive memory and computation resources which are difficult to be fulfilled by existing BA libraries.
Trap of Feature Diversity in the Learning of MLPs
In this paper, we focus on a typical two-phase phenomenon in the learning of multi-layer perceptrons (MLPs), and we aim to explain the reason for the decrease of feature diversity in the first phase.
Towards a Unified Game-Theoretic View of Adversarial Perturbations and Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
Towards Axiomatic, Hierarchical, and Symbolic Explanation for Deep Models
This paper aims to show that the inference logic of a deep model can be faithfully approximated as a sparse, symbolic causal graph.
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, \emph{i. e.} the view of multi-order interactions between input variables of DNNs.
ESOD:Edge-based Task Scheduling for Object Detection
In detail, we train a DNN model (termed as pre-model) to predict which object detection model to use for the coming task and offloads to which edge servers by physical characteristics of the image task (e. g., brightness, saturation).
Cellular Network Radio Propagation Modeling with Deep Convolutional Neural Networks
Deterministic models, such as ray tracing based on physical laws of wave propagation, are more accurate and site specific.
Towards a Game-Theoretic View of Baseline Values in the Shapley Value
In the computation of Shapley values, people usually set an input variable to its baseline value to represent the absence of this variable.
Proceedings of ICML 2021 Workshop on Theoretic Foundation, Criticism, and Application Trend of Explainable AI
This workshop pays a special interest in theoretic foundations, limitations, and new application trends in the scope of XAI.
Sawtooth Factorial Topic Embeddings Guided Gamma Belief Network
However, they often assume in the prior that the topics at each layer are independently drawn from the Dirichlet distribution, ignoring the dependencies between the topics both at the same layer and across different layers.
A Game-Theoretic Taxonomy of Visual Concepts in DNNs
In this paper, we rethink how a DNN encodes visual concepts of different complexities from a new perspective, i. e. the game-theoretic multi-order interactions between pixels in an image.
A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks.
Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learning
In this paper we introduce Uncertainty Baselines: high-quality implementations of standard and state-of-the-art deep learning methods on a variety of tasks.
Exploring the Limits of Out-of-Distribution Detection
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks.
Ranked #2 on
Out-of-Distribution Detection
on CIFAR-10 vs CIFAR-100
(using extra training data)
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Can We Faithfully Represent Absence States to Compute Shapley Values on a DNN?
Therefore, we revisit the feature representation of a DNN in terms of causality, and propose to use causal patterns to examine whether the masking method faithfully removes information encoded in input variables.
Probabilistic Mixture-of-Experts for Efficient Deep Reinforcement Learning
However, grasping distinguishable skills for some tasks with non-unique optima can be essential for further improving its learning efficiency and performance, which may lead to a multimodal policy represented as a mixture-of-experts (MOE).
Learning to Remove: Towards Isotropic Pre-trained BERT Embedding
We train the weights on word similarity tasks and show that processed embedding is more isotropic.
Does Your Dermatology Classifier Know What It Doesn't Know? Detecting the Long-Tail of Unseen Conditions
We develop and rigorously evaluate a deep learning based system that can accurately classify skin conditions while detecting rare conditions for which there is not enough data available for training a confident classifier.
Uniting Heterogeneity, Inductiveness, and Efficiency for Graph Representation Learning
In WIDEN, we propose a novel inductive, meta path-free message passing scheme that packs up heterogeneous node features with their associated edges from both low- and high-order neighbor nodes.
A Unified Game-Theoretic Interpretation of Adversarial Robustness
This paper provides a unified view to explain different adversarial attacks and defense methods, i. e. the view of multi-order interactions between input variables of DNNs.
Identifying Gene-environment interactions with robust marginal Bayesian variable selection
In this study, we propose a novel marginal Bayesian variable selection method for G$\times$E studies.
Variable Selection
Methodology
ZeRO-Offload: Democratizing Billion-Scale Model Training
By combining compute and memory efficiency with ease-of-use, ZeRO-Offload democratizes large-scale model training making it accessible to even data scientists with access to just a single GPU.
Disentanglement, Visualization and Analysis of Complex Features in DNNs
This paper aims to define, visualize, and analyze the feature complexity that is learned by a DNN.
Understanding, Analyzing, and Optimizing the Complexity of Deep Models
Based on the proposed metrics, we analyze two typical phenomena of the change of the transformation complexity during the training process, and explore the ceiling of a DNN’s complexity.
Towards A Unified Understanding and Improving of Adversarial Transferability
We discover and prove the negative correlation between the adversarial transferability and the interaction inside adversarial perturbations.
Analytic critical points of charged Renyi entropies from hyperbolic black holes
The first system is the Reissner-Nordstrom-AdS$_5$ black hole, which has finite entropy at zero temperature.
High Energy Physics - Theory General Relativity and Quantum Cosmology
Receptivity and stability of hypersonic leading-edge sweep flows around a blunt body
We establish an adjoint-based bi-orthogonal eigenfunction system to address the receptivity problem of such flows to any external forces and boundary perturbations.
Fluid Dynamics
HM-ANN: Efficient Billion-Point Nearest Neighbor Search on Heterogeneous Memory
The emergence of heterogeneous memory (HM) brings a solution to significantly increase memory capacity and break the above tradeoff: Using HM, billions of data points can be placed in the main memory on a single machine without using any data compression.
A Unified Approach to Interpreting and Boosting Adversarial Transferability
We discover and prove the negative correlation between the adversarial transferability and the interaction inside adversarial perturbations.
Revisiting One-vs-All Classifiers for Predictive Uncertainty and Out-of-Distribution Detection in Neural Networks
Accurate estimation of predictive uncertainty in modern neural networks is critical to achieve well calibrated predictions and detect out-of-distribution (OOD) inputs.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Interpreting and Disentangling Feature Components of Various Complexity from DNNs
This paper aims to define, quantify, and analyze the feature complexity that is learned by a DNN.
SummAE: Zero-Shot Abstractive Text Summarization using Length-Agnostic Auto-Encoders
We show results for extractive and human baselines to demonstrate a large abstractive gap in performance.
Likelihood Ratios for Out-of-Distribution Detection
We propose a likelihood ratio method for deep generative models which effectively corrects for these confounding background statistics.
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Modern machine learning methods including deep learning have achieved great success in predictive accuracy for supervised learning tasks, but may still fall short in giving useful estimates of their predictive {\em uncertainty}.
Semi-parametric Bayesian variable selection for gene-environment interactions
Existing Bayesian methods for G$\times$E interaction studies are challenged by the high-dimensional nature of the study and the complexity of environmental influences.
Methodology
Interpretable Complex-Valued Neural Networks for Privacy Protection
Previous studies have found that an adversary attacker can often infer unintended input information from intermediate-layer features.
Explaining Neural Networks Semantically and Quantitatively
This paper presents a method to explain the knowledge encoded in a convolutional neural network (CNN) quantitatively and semantically.
To Compress, or Not to Compress: Characterizing Deep Learning Model Compression for Embedded Inference
We experimentally show that how two mainstream compression techniques, data quantization and pruning, perform on these network architectures and the implications of compression techniques to the model storage size, inference time, energy consumption and performance metrics.
Active Learning for Wireless IoT Intrusion Detection
This paper discusses the human-in-the-loop active learning approach for wireless intrusion detection.
An End-to-End Compression Framework Based on Convolutional Neural Networks
The second CNN, named reconstruction convolutional neural network (RecCNN), is used to reconstruct the decoded image with high-quality in the decoding end.
