Search Results for author:
Found 48 papers, 20 papers with code
MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness
However, these methods still follow the design of traditional supervised adversarial training, limiting the potential of adversarial training on ViTs.
Aberrant High-Order Dependencies in Schizophrenia Resting-State Functional MRI Networks
The human brain has a complex, intricate functional architecture.
Continual Invariant Risk Minimization
Invariant risk minimization (IRM) is a recent proposal for discovering environment-invariant representations.
Revisiting the Robustness of the Minimum Error Entropy Criterion: A Transfer Learning Case Study
In this paper, we revisit the robustness of the minimum error entropy (MEE) criterion, a widely used objective in statistical signal processing to deal with non-Gaussian noises, and investigate its feasibility and usefulness in real-life transfer learning regression tasks, where distributional shifts are common.
Coping with Change: Learning Invariant and Minimum Sufficient Representations for Fine-Grained Visual Categorization
Fine-grained visual categorization (FGVC) is a challenging task due to similar visual appearances between various species.
Higher-order Organization in the Human Brain from Matrix-Based Rényi's Entropy
To address this problem, we applied multivariate mutual information, specifically, Total Correlation and Dual Total Correlation to reveal higher-order information in the brain.
The Conditional Cauchy-Schwarz Divergence with Applications to Time-Series Data and Sequential Decision Making
In this paper, we extend the classic CS divergence to quantify the closeness between two conditional distributions and show that the developed conditional CS divergence can be simply estimated by a kernel density estimator from given samples.
Causal Recurrent Variational Autoencoder for Medical Time Series Generation
We propose causal recurrent variational autoencoder (CR-VAE), a novel generative model that is able to learn a Granger causal graph from a multivariate time series x and incorporates the underlying causal mechanism into its data generation process.
CI-GNN: A Granger Causality-Inspired Graph Neural Network for Interpretable Brain Network-Based Psychiatric Diagnosis
There is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which, in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs.
Robust and Fast Measure of Information via Low-rank Representation
The matrix-based R\'enyi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution.
Functional Connectome of the Human Brain with Total Correlation
In this work we build on this idea to infer a large scale (whole brain) connectivity network based on Total Correlation and show the possibility of using this kind of networks as biomarkers of brain alterations.
Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval
In this paper, instead of using an extra NLP model to define a common space beforehand, we consider a totally different way to construct (or learn) a common hamming space from an information-theoretic perspective.
Principle of Relevant Information for Graph Sparsification
Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties.
Optimal Randomized Approximations for Matrix based Renyi's Entropy
The Matrix-based Renyi's entropy enables us to directly measure information quantities from given data without the costly probability density estimation of underlying distributions, thus has been widely adopted in numerous statistical learning and inference tasks.
BrainIB: Interpretable Brain Network-based Psychiatric Diagnosis with Graph Information Bottleneck
Developing a new diagnostic models based on the underlying biological mechanisms rather than subjective symptoms for psychiatric disorders is an emerging consensus.
Multi-view Information Bottleneck Without Variational Approximation
By "intelligently" fusing the complementary information across different views, multi-view learning is able to improve the performance of classification tasks.
R2-Trans:Fine-Grained Visual Categorization with Redundancy Reduction
In this paper, we present a novel approach for FGVC, which can simultaneously make use of partial yet sufficient discriminative information in environmental cues and also compress the redundant information in class-token with respect to the target.
Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing
We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components.
Computationally Efficient Approximations for Matrix-based Renyi's Entropy
The recently developed matrix based Renyi's entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the underlying data distribution.
Information Theoretic Structured Generative Modeling
We test the framework for estimation of mutual information and compare the results with the mutual information neural estimation (MINE), for density estimation, for conditional probability estimation in Markov models as well as for training adversarial networks.
Gated Information Bottleneck for Generalization in Sequential Environments
By learning minimum sufficient representations from training data, the information bottleneck (IB) approach has demonstrated its effectiveness to improve generalization in different AI applications.
Adversarial Robustness
Out of Distribution (OOD) Detection
+1
Learning to Transfer with von Neumann Conditional Divergence
Feature similarity includes both the invariance of marginal distributions and the closeness of conditional distributions given the desired response $y$ (e. g., class labels).
Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional
We introduce the matrix-based Renyi's $\alpha$-order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network.
Measuring Dependence with Matrix-based Entropy Functional
Measuring the dependence of data plays a central role in statistics and machine learning.
Modular-Relatedness for Continual Learning
In this paper, we propose a continual learning (CL) technique that is beneficial to sequential task learners by improving their retained accuracy and reducing catastrophic forgetting.
Bilevel Continual Learning
This paper presents Bilevel Continual Learning (BiCL), a general framework for continual learning that fuses bilevel optimization and recent advances in meta-learning for deep neural networks.
Towards Interpretable Multi-Task Learning Using Bilevel Programming
Interpretable Multi-Task Learning can be expressed as learning a sparse graph of the task relationship based on the prediction performance of the learned models.
Learning an Interpretable Graph Structure in Multi-Task Learning
We present a novel methodology to jointly perform multi-task learning and infer intrinsic relationship among tasks by an interpretable and sparse graph.
Interpretable Fault Detection using Projections of Mutual Information Matrix
This paper presents a novel mutual information (MI) matrix based method for fault detection.
PRI-VAE: Principle-of-Relevant-Information Variational Autoencoders
Although substantial efforts have been made to learn disentangled representations under the variational autoencoder (VAE) framework, the fundamental properties to the dynamics of learning of most VAE models still remain unknown and under-investigated.
Modularizing Deep Learning via Pairwise Learning With Kernels
By redefining the conventional notions of layers, we present an alternative view on finitely wide, fully trainable deep neural networks as stacked linear models in feature spaces, leading to a kernel machine interpretation.
Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications
We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions.
Information Plane Analysis of Deep Neural Networks via Matrix--Based Renyi's Entropy and Tensor Kernels
In this paper, we propose an IP analysis using the new matrix--based R\'enyi's entropy coupled with tensor kernels over convolutional layers, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data.
Information Plane Analysis of Deep Neural Networks via Matrix-Based Renyi's Entropy and Tensor Kernels
In this paper, we propose an IP analysis using the new matrix--based R\'enyi's entropy coupled with tensor kernels over convolutional layers, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data.
Multiscale Principle of Relevant Information for Hyperspectral Image Classification
This paper proposes a novel architecture, termed multiscale principle of relevant information (MPRI), to learn discriminative spectral-spatial features for hyperspectral image (HSI) classification.
Closed-Loop Adaptation for Weakly-Supervised Semantic Segmentation
Weakly-supervised semantic segmentation aims to assign each pixel a semantic category under weak supervisions, such as image-level tags.
Learning Backpropagation-Free Deep Architectures with Kernels
Moreover, unlike backpropagation, which turns models into black boxes, the optimal hidden representation enjoys an intuitive geometric interpretation, making the dynamics of learning in a deep kernel network simple to understand.
Simple stopping criteria for information theoretic feature selection
Feature selection aims to select the smallest feature subset that yields the minimum generalization error.
Robust Visual Tracking using Multi-Frame Multi-Feature Joint Modeling
It remains a huge challenge to design effective and efficient trackers under complex scenarios, including occlusions, illumination changes and pose variations.
Multivariate Extension of Matrix-based Renyi's α-order Entropy Functional
The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).
Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels
In this paper, a novel Hierarchical Hypothesis Testing framework with Request-and-Reverify strategy is developed to detect concept drifts by requesting labels only when necessary.
Coarse-to-Fine Salient Object Detection with Low-Rank Matrix Recovery
Low-Rank Matrix Recovery (LRMR) has recently been applied to saliency detection by decomposing image features into a low-rank component associated with background and a sparse component associated with visual salient regions.
Multi-view Hybrid Embedding: A Divide-and-Conquer Approach
We present a novel cross-view classification algorithm where the gallery and probe data come from different views.
Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration
The matrix-based Renyi's \alpha-entropy functional and its multivariate extension were recently developed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).
Understanding Autoencoders with Information Theoretic Concepts
Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks.
On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation
With this method, we obtain a counterpart of any given NN that is powered by kernel machines instead of neurons.
Concept Drift Detection and Adaptation with Hierarchical Hypothesis Testing
A fundamental issue for statistical classification models in a streaming environment is that the joint distribution between predictor and response variables changes over time (a phenomenon also known as concept drifts), such that their classification performance deteriorates dramatically.
Marine Animal Classification with Correntropy Loss Based Multi-view Learning
Depending on the quantity and properties of acquired imagery, the animals are characterized as either features (shape, color, texture, etc.
