Search Results for author: Shujian Yu

Found 48 papers, 20 papers with code

MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustness

1 code implementation8 Dec 2023 Xiaoyun Xu, Shujian Yu, Jingzheng Wu, Stjepan Picek

However, these methods still follow the design of traditional supervised adversarial training, limiting the potential of adversarial training on ViTs.

Adversarial Robustness

Continual Invariant Risk Minimization

no code implementations21 Oct 2023 Francesco Alesiani, Shujian Yu, Mathias Niepert

Invariant risk minimization (IRM) is a recent proposal for discovering environment-invariant representations.

Continual Learning

Revisiting the Robustness of the Minimum Error Entropy Criterion: A Transfer Learning Case Study

1 code implementation17 Jul 2023 Luis Pedro Silvestrin, Shujian Yu, Mark Hoogendoorn

In this paper, we revisit the robustness of the minimum error entropy (MEE) criterion, a widely used objective in statistical signal processing to deal with non-Gaussian noises, and investigate its feasibility and usefulness in real-life transfer learning regression tasks, where distributional shifts are common.

Time Series Transfer Learning

Higher-order Organization in the Human Brain from Matrix-Based Rényi's Entropy

no code implementations21 Mar 2023 Qiang Li, Shujian Yu, Kristoffer H Madsen, Vince D Calhoun, Armin Iraji

To address this problem, we applied multivariate mutual information, specifically, Total Correlation and Dual Total Correlation to reveal higher-order information in the brain.

Time Series

The Conditional Cauchy-Schwarz Divergence with Applications to Time-Series Data and Sequential Decision Making

no code implementations21 Jan 2023 Shujian Yu, Hongming Li, Sigurd Løkse, Robert Jenssen, José C. Príncipe

In this paper, we extend the classic CS divergence to quantify the closeness between two conditional distributions and show that the developed conditional CS divergence can be simply estimated by a kernel density estimator from given samples.

Decision Making Time Series +1

Causal Recurrent Variational Autoencoder for Medical Time Series Generation

1 code implementation16 Jan 2023 Hongming Li, Shujian Yu, Jose Principe

We propose causal recurrent variational autoencoder (CR-VAE), a novel generative model that is able to learn a Granger causal graph from a multivariate time series x and incorporates the underlying causal mechanism into its data generation process.

Causal Inference EEG +3

CI-GNN: A Granger Causality-Inspired Graph Neural Network for Interpretable Brain Network-Based Psychiatric Diagnosis

1 code implementation4 Jan 2023 Kaizhong Zheng, Shujian Yu, Badong Chen

There is a recent trend to leverage the power of graph neural networks (GNNs) for brain-network based psychiatric diagnosis, which, in turn, also motivates an urgent need for psychiatrists to fully understand the decision behavior of the used GNNs.

Robust and Fast Measure of Information via Low-rank Representation

1 code implementation30 Nov 2022 Yuxin Dong, Tieliang Gong, Shujian Yu, Hong Chen, Chen Li

The matrix-based R\'enyi's entropy allows us to directly quantify information measures from given data, without explicit estimation of the underlying probability distribution.

Computational Efficiency

Functional Connectome of the Human Brain with Total Correlation

no code implementations6 Oct 2022 Qiang Li, Greg Ver Steeg, Shujian Yu, Jesus Malo

In this work we build on this idea to infer a large scale (whole brain) connectivity network based on Total Correlation and show the possibility of using this kind of networks as biomarkers of brain alterations.

Information-Theoretic Hashing for Zero-Shot Cross-Modal Retrieval

no code implementations26 Sep 2022 Yufeng Shi, Shujian Yu, Duanquan Xu, Xinge You

In this paper, instead of using an extra NLP model to define a common space beforehand, we consider a totally different way to construct (or learn) a common hamming space from an information-theoretic perspective.

Cross-Modal Retrieval Retrieval +1

Principle of Relevant Information for Graph Sparsification

1 code implementation31 May 2022 Shujian Yu, Francesco Alesiani, Wenzhe Yin, Robert Jenssen, Jose C. Principe

Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties.

Multi-Task Learning

Optimal Randomized Approximations for Matrix based Renyi's Entropy

no code implementations16 May 2022 Yuxin Dong, Tieliang Gong, Shujian Yu, Chen Li

The Matrix-based Renyi's entropy enables us to directly measure information quantities from given data without the costly probability density estimation of underlying distributions, thus has been widely adopted in numerous statistical learning and inference tasks.

Density Estimation

BrainIB: Interpretable Brain Network-based Psychiatric Diagnosis with Graph Information Bottleneck

no code implementations7 May 2022 Kaizhong Zheng, Shujian Yu, Baojuan Li, Robert Jenssen, Badong Chen

Developing a new diagnostic models based on the underlying biological mechanisms rather than subjective symptoms for psychiatric disorders is an emerging consensus.

Multi-view Information Bottleneck Without Variational Approximation

1 code implementation22 Apr 2022 Qi Zhang, Shujian Yu, Jingmin Xin, Badong Chen

By "intelligently" fusing the complementary information across different views, multi-view learning is able to improve the performance of classification tasks.


R2-Trans:Fine-Grained Visual Categorization with Redundancy Reduction

no code implementations21 Apr 2022 Yu Wang, Shuo Ye, Shujian Yu, Xinge You

In this paper, we present a novel approach for FGVC, which can simultaneously make use of partial yet sufficient discriminative information in environmental cues and also compress the redundant information in class-token with respect to the target.

Fine-Grained Visual Categorization

Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing

1 code implementation7 Feb 2022 Hongming Li, Shujian Yu, Jose C. Principe

We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components.

Hyperspectral Unmixing

Computationally Efficient Approximations for Matrix-based Renyi's Entropy

no code implementations27 Dec 2021 Tieliang Gong, Yuxin Dong, Shujian Yu, Bo Dong

The recently developed matrix based Renyi's entropy enables measurement of information in data simply using the eigenspectrum of symmetric positive semi definite (PSD) matrices in reproducing kernel Hilbert space, without estimation of the underlying data distribution.

Information Theoretic Structured Generative Modeling

1 code implementation12 Oct 2021 Bo Hu, Shujian Yu, Jose C. Principe

We test the framework for estimation of mutual information and compare the results with the mutual information neural estimation (MINE), for density estimation, for conditional probability estimation in Markov models as well as for training adversarial networks.

Density Estimation

Gated Information Bottleneck for Generalization in Sequential Environments

1 code implementation12 Oct 2021 Francesco Alesiani, Shujian Yu, Xi Yu

By learning minimum sufficient representations from training data, the information bottleneck (IB) approach has demonstrated its effectiveness to improve generalization in different AI applications.

Adversarial Robustness Out of Distribution (OOD) Detection +1

Learning to Transfer with von Neumann Conditional Divergence

no code implementations7 Aug 2021 Ammar Shaker, Shujian Yu, Daniel Oñoro-Rubio

Feature similarity includes both the invariance of marginal distributions and the closeness of conditional distributions given the desired response $y$ (e. g., class labels).

Domain Adaptation

Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional

1 code implementation31 Jan 2021 Xi Yu, Shujian Yu, Jose C. Principe

We introduce the matrix-based Renyi's $\alpha$-order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network.

Variational Inference

Modular-Relatedness for Continual Learning

no code implementations2 Nov 2020 Ammar Shaker, Shujian Yu, Francesco Alesiani

In this paper, we propose a continual learning (CL) technique that is beneficial to sequential task learners by improving their retained accuracy and reducing catastrophic forgetting.

Continual Learning

Bilevel Continual Learning

no code implementations2 Nov 2020 Ammar Shaker, Francesco Alesiani, Shujian Yu, Wenzhe Yin

This paper presents Bilevel Continual Learning (BiCL), a general framework for continual learning that fuses bilevel optimization and recent advances in meta-learning for deep neural networks.

Bilevel Optimization Continual Learning +1

Towards Interpretable Multi-Task Learning Using Bilevel Programming

no code implementations11 Sep 2020 Francesco Alesiani, Shujian Yu, Ammar Shaker, Wenzhe Yin

Interpretable Multi-Task Learning can be expressed as learning a sparse graph of the task relationship based on the prediction performance of the learned models.

Multi-Task Learning

Learning an Interpretable Graph Structure in Multi-Task Learning

no code implementations11 Sep 2020 Shujian Yu, Francesco Alesiani, Ammar Shaker, Wenzhe Yin

We present a novel methodology to jointly perform multi-task learning and infer intrinsic relationship among tasks by an interpretable and sparse graph.

Multi-Task Learning

PRI-VAE: Principle-of-Relevant-Information Variational Autoencoders

no code implementations13 Jul 2020 Yanjun Li, Shujian Yu, Jose C. Principe, Xiaolin Li, Dapeng Wu

Although substantial efforts have been made to learn disentangled representations under the variational autoencoder (VAE) framework, the fundamental properties to the dynamics of learning of most VAE models still remain unknown and under-investigated.

Modularizing Deep Learning via Pairwise Learning With Kernels

1 code implementation12 May 2020 Shiyu Duan, Shujian Yu, Jose Principe

By redefining the conventional notions of layers, we present an alternative view on finitely wide, fully trainable deep neural networks as stacked linear models in feature spaces, leading to a kernel machine interpretation.

Binary Classification General Classification +1

Information Plane Analysis of Deep Neural Networks via Matrix-Based Renyi's Entropy and Tensor Kernels

no code implementations25 Sep 2019 Kristoffer Wickstrøm, Sigurd Løkse, Michael Kampffmeyer, Shujian Yu, Jose Principe, Robert Jenssen

In this paper, we propose an IP analysis using the new matrix--based R\'enyi's entropy coupled with tensor kernels over convolutional layers, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data.

Information Plane

Information Plane Analysis of Deep Neural Networks via Matrix--Based Renyi's Entropy and Tensor Kernels

no code implementations25 Sep 2019 Kristoffer Wickstrøm, Sigurd Løkse, Michael Kampffmeyer, Shujian Yu, Jose Principe, Robert Jenssen

In this paper, we propose an IP analysis using the new matrix--based R\'enyi's entropy coupled with tensor kernels over convolutional layers, leveraging the power of kernel methods to represent properties of the probability distribution independently of the dimensionality of the data.

Information Plane

Multiscale Principle of Relevant Information for Hyperspectral Image Classification

1 code implementation13 Jul 2019 Yantao Wei, Shujian Yu, Luis Sanchez Giraldo, Jose C. Principe

This paper proposes a novel architecture, termed multiscale principle of relevant information (MPRI), to learn discriminative spectral-spatial features for hyperspectral image (HSI) classification.

Classification Dimensionality Reduction +2

Closed-Loop Adaptation for Weakly-Supervised Semantic Segmentation

no code implementations29 May 2019 Zhengqiang Zhang, Shujian Yu, Shi Yin, Qinmu Peng, Xinge You

Weakly-supervised semantic segmentation aims to assign each pixel a semantic category under weak supervisions, such as image-level tags.

Segmentation Superpixels +2

Learning Backpropagation-Free Deep Architectures with Kernels

no code implementations ICLR 2019 Shiyu Duan, Shujian Yu, Yun-Mei Chen, Jose Principe

Moreover, unlike backpropagation, which turns models into black boxes, the optimal hidden representation enjoys an intuitive geometric interpretation, making the dynamics of learning in a deep kernel network simple to understand.

Simple stopping criteria for information theoretic feature selection

no code implementations29 Nov 2018 Shujian Yu, Jose C. Principe

Feature selection aims to select the smallest feature subset that yields the minimum generalization error.

feature selection

Robust Visual Tracking using Multi-Frame Multi-Feature Joint Modeling

1 code implementation19 Nov 2018 Peng Zhang, Shujian Yu, Jiamiao Xu, Xinge You, Xiubao Jiang, Xiao-Yuan Jing, DaCheng Tao

It remains a huge challenge to design effective and efficient trackers under complex scenarios, including occlusions, illumination changes and pose variations.

Multi-Task Learning MULTI-VIEW LEARNING +2

Multivariate Extension of Matrix-based Renyi's α-order Entropy Functional

1 code implementation23 Aug 2018 Shujian Yu, Luis Gonzalo Sanchez Giraldo, Robert Jenssen, Jose C. Principe

The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).

feature selection

Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels

no code implementations25 Jun 2018 Shujian Yu, Xiaoyang Wang, Jose C. Principe

In this paper, a novel Hierarchical Hypothesis Testing framework with Request-and-Reverify strategy is developed to detect concept drifts by requesting labels only when necessary.

Attribute General Classification +1

Coarse-to-Fine Salient Object Detection with Low-Rank Matrix Recovery

no code implementations21 May 2018 Qi Zheng, Shujian Yu, Xinge You, Qinmu Peng

Low-Rank Matrix Recovery (LRMR) has recently been applied to saliency detection by decomposing image features into a low-rank component associated with background and a sparse component associated with visual salient regions.

object-detection RGB Salient Object Detection +2

Multi-view Hybrid Embedding: A Divide-and-Conquer Approach

no code implementations19 Apr 2018 Jiamiao Xu, Shujian Yu, Xinge You, Mengjun Leng, Xiao-Yuan Jing, C. L. Philip Chen

We present a novel cross-view classification algorithm where the gallery and probe data come from different views.

Classification General Classification

Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration

no code implementations18 Apr 2018 Shujian Yu, Kristoffer Wickstrøm, Robert Jenssen, Jose C. Principe

The matrix-based Renyi's \alpha-entropy functional and its multivariate extension were recently developed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).

Understanding Autoencoders with Information Theoretic Concepts

no code implementations30 Mar 2018 Shujian Yu, Jose C. Principe

Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks.

Information Plane

On Kernel Method-Based Connectionist Models and Supervised Deep Learning Without Backpropagation

1 code implementation ICLR 2019 Shiyu Duan, Shujian Yu, Yun-Mei Chen, Jose Principe

With this method, we obtain a counterpart of any given NN that is powered by kernel machines instead of neurons.

Concept Drift Detection and Adaptation with Hierarchical Hypothesis Testing

no code implementations25 Jul 2017 Shujian Yu, Zubin Abraham, Heng Wang, Mohak Shah, Yantao Wei, José C. Príncipe

A fundamental issue for statistical classification models in a streaming environment is that the joint distribution between predictor and response variables changes over time (a phenomenon also known as concept drifts), such that their classification performance deteriorates dramatically.

General Classification Two-sample testing

Marine Animal Classification with Correntropy Loss Based Multi-view Learning

no code implementations3 May 2017 Zheng Cao, Shujian Yu, Bing Ouyang, Fraser Dalgleish, Anni Vuorenkoski, Gabriel Alsenas, Jose Principe

Depending on the quantity and properties of acquired imagery, the animals are characterized as either features (shape, color, texture, etc.

General Classification MULTI-VIEW LEARNING

Cannot find the paper you are looking for? You can Submit a new open access paper.