Search Results for author: Jose C. Principe

Found 52 papers, 9 papers with code

Principle of Relevant Information for Graph Sparsification

1 code implementation31 May 2022 Shujian Yu, Francesco Alesiani, Wenzhe Yin, Robert Jenssen, Jose C. Principe

Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties.

Multi-Task Learning

Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing

1 code implementation7 Feb 2022 Hongming Li, Shujian Yu, Jose C. Principe

We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components.

Hyperspectral Unmixing

Information Theoretic Structured Generative Modeling

1 code implementation12 Oct 2021 Bo Hu, Shujian Yu, Jose C. Principe

We test the framework for estimation of mutual information and compare the results with the mutual information neural estimation (MINE), for density estimation, for conditional probability estimation in Markov models as well as for training adversarial networks.

Density Estimation

Estimating Rényi's $α$-Cross-Entropies in a Matrix-Based Way

1 code implementation24 Sep 2021 Isaac J. Sledge, Jose C. Principe

This yields matrix-based estimators of R\'enyi's $\alpha$-cross-entropies.

A Physics inspired Functional Operator for Model Uncertainty Quantification in the RKHS

no code implementations22 Sep 2021 Rishabh Singh, Jose C. Principe

The RKHS projection of model weights yields a potential field based interpretation of model weight PDF which consequently allows the definition of a functional operator, inspired by perturbation theory in physics, that performs a moment decomposition of the model weight PDF (the potential field) at a specific model output to quantify its uncertainty.

Analysis of Intra-Operative Physiological Responses Through Complex Higher-Order SVD for Long-Term Post-Operative Pain Prediction

no code implementations2 Sep 2021 Raheleh Baharloo, Jose C. Principe, Parisa Rashidi, Patrick J. Tighe

The dynamics of patients' physiological responses to these surgical events are linked to long-term post-operative pain development.

An Information-Theoretic Approach for Automatically Determining the Number of States when Aggregating Markov Chains

no code implementations5 Jul 2021 Isaac J. Sledge, Jose C. Principe

A fundamental problem when aggregating Markov chains is the specification of the number of state groups.

Labels, Information, and Computation: Efficient Learning Using Sufficient Labels

no code implementations19 Apr 2021 Shiyu Duan, Spencer Chang, Jose C. Principe

We call this statistic "sufficiently-labeled data" and prove its sufficiency and efficiency for finding the optimal hidden representations, on which competent classifier heads can be trained using as few as a single randomly-chosen fully-labeled example per class.

Privacy Preserving

A Kernel Framework to Quantify a Model's Local Predictive Uncertainty under Data Distributional Shifts

no code implementations2 Mar 2021 Rishabh Singh, Jose C. Principe

We therefore propose a framework for predictive uncertainty quantification of a trained neural network that explicitly estimates the PDF of its raw prediction space (before activation), p(y'|x, w), which we refer to as the model PDF, in a Gaussian reproducing kernel Hilbert space (RKHS).

Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning

no code implementations24 Feb 2021 Isaac J. Sledge, Darshan W. Bryner, Jose C. Principe

We have previously developed a means of constraining, and hence speeding up, the search process through the use of motion primitives; motion primitives are sequences of pre-specified actions taken across a state series.


Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional

1 code implementation31 Jan 2021 Xi Yu, Shujian Yu, Jose C. Principe

We introduce the matrix-based Renyi's $\alpha$-order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network.

Variational Inference

Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations

no code implementations18 Jan 2021 Isaac J. Sledge, Jose C. Principe

It yields unsupervised object recognition that surpass convolutional autoencoders and are on par with convolutional networks trained in a supervised manner.

Object Recognition

Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods

no code implementations9 Jan 2021 Shiyu Duan, Jose C. Principe

This tutorial paper surveys provably optimal alternatives to end-to-end backpropagation (E2EBP) -- the de facto standard for training deep architectures.

Local power estimation of neuromodulations using point process modeling

no code implementations16 Nov 2020 Shailaja Akella, Ali Mohebi, Kiersten Riels, Andreas Keil, Karim Oweiss, Jose C. Principe

A detailed analysis of the power - specific marked features of neuromodulations confirm high correlation between power spectral density and power in neuromodulations establishing the aptness of MPP spectrogram as a finer measure of power where it is able to track local variations in power while preserving the global structure of signal power distribution.

Unsupervised Foveal Vision Neural Networks with Top-Down Attention

no code implementations18 Oct 2020 Ryan Burt, Nina N. Thigpen, Andreas Keil, Jose C. Principe

The results in foveated vision show that Gamma saliency is comparable to the best and computationally faster.

Object Recognition Scene Understanding

PRI-VAE: Principle-of-Relevant-Information Variational Autoencoders

no code implementations13 Jul 2020 Yanjun Li, Shujian Yu, Jose C. Principe, Xiaolin Li, Dapeng Wu

Although substantial efforts have been made to learn disentangled representations under the variational autoencoder (VAE) framework, the fundamental properties to the dynamics of learning of most VAE models still remain unknown and under-investigated.

Towards a Kernel based Uncertainty Decomposition Framework for Data and Models

no code implementations30 Jan 2020 Rishabh Singh, Jose C. Principe

This paper introduces a new framework for quantifying predictive uncertainty for both data and models that relies on projecting the data into a Gaussian reproducing kernel Hilbert space (RKHS) and transforming the data probability density function (PDF) in a way that quantifies the flow of its gradient as a topological potential field quantified at all points in the sample space.

Time Series

Fast Estimation of Information Theoretic Learning Descriptors using Explicit Inner Product Spaces

no code implementations1 Jan 2020 Kan Li, Jose C. Principe

The inner product defined by the feature mapping corresponds to a positive-definite finite-rank kernel that induces a finite-dimensional reproducing kernel Hilbert space (RKHS).

No-Trick (Treat) Kernel Adaptive Filtering using Deterministic Features

no code implementations10 Dec 2019 Kan Li, Jose C. Principe

Without loss of generality, we apply this approach to classical adaptive filtering algorithms and validate the methodology to show that deterministic features are faster to generate and outperform state-of-the-art kernel methods based on random Fourier features.

Functional Bayesian Filter

no code implementations24 Nov 2019 Kan Li, Jose C. Principe

We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS).

Arithmetic Time Series

Algorithmic Design and Implementation of Unobtrusive Multistatic Serial LiDAR Image

no code implementations8 Nov 2019 Chi Ding, Zheng Cao, Matthew S. Emigh, Jose C. Principe, Bing Ouyang, Anni Vuorenkoski, Fraser Dalgleish, Brian Ramos, Yanjun Li

To fully understand interactions between marine hydrokinetic (MHK) equipment and marine animals, a fast and effective monitoring system is required to capture relevant information whenever underwater animals appear.

Scene Understanding

Multiscale Principle of Relevant Information for Hyperspectral Image Classification

no code implementations13 Jul 2019 Yantao Wei, Shujian Yu, Luis Sanchez Giraldo, Jose C. Principe

This paper proposes a novel architecture, termed multiscale principle of relevant information (MPRI), to learn discriminative spectral-spatial features for hyperspectral image (HSI) classification.

Classification Dimensionality Reduction +2

Maximum Correntropy Criterion with Variable Center

no code implementations13 Apr 2019 Badong Chen, Xin Wang, Yingsong Li, Jose C. Principe

The kernel function in correntropy is usually restricted to the Gaussian function with center located at zero.

An Exact Reformulation of Feature-Vector-based Radial-Basis-Function Networks for Graph-based Observations

no code implementations22 Jan 2019 Isaac J. Sledge, Jose C. Principe

An important property of our radial basis function networks is that they are guaranteed to yield the same responses as conventional radial-basis networks trained on a corresponding vector realization of the relationships encoded by the adjacency-matrix.

Theory and Algorithms for Pulse Signal Processing

no code implementations31 Dec 2018 Gabriel Nallathambi, Jose C. Principe

The integrate and fire converter transforms an analog signal into train of biphasic pulses.

Simple stopping criteria for information theoretic feature selection

no code implementations29 Nov 2018 Shujian Yu, Jose C. Principe

Feature selection aims to select the smallest feature subset that yields the minimum generalization error.

feature selection

Multivariate Extension of Matrix-based Renyi's α-order Entropy Functional

no code implementations23 Aug 2018 Shujian Yu, Luis Gonzalo Sanchez Giraldo, Robert Jenssen, Jose C. Principe

The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).

feature selection

Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels

no code implementations25 Jun 2018 Shujian Yu, Xiaoyang Wang, Jose C. Principe

In this paper, a novel Hierarchical Hypothesis Testing framework with Request-and-Reverify strategy is developed to detect concept drifts by requesting labels only when necessary.

General Classification Two-sample testing

Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration

no code implementations18 Apr 2018 Shujian Yu, Kristoffer Wickstrøm, Robert Jenssen, Jose C. Principe

The matrix-based Renyi's \alpha-entropy functional and its multivariate extension were recently developed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).

Understanding Autoencoders with Information Theoretic Concepts

no code implementations30 Mar 2018 Shujian Yu, Jose C. Principe

Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks.

Information Plane

Guided Policy Exploration for Markov Decision Processes using an Uncertainty-Based Value-of-Information Criterion

no code implementations5 Feb 2018 Isaac J. Sledge, Matthew S. Emigh, Jose C. Principe

The value of information yields a stochastic routine for choosing actions during learning that can explore the policy space in a coarse to fine manner.


Augmented Space Linear Model

no code implementations1 Feb 2018 Zhengda Qin, Badong Chen, Nanning Zheng, Jose C. Principe

In this paper, we propose a linear model called Augmented Space Linear Model (ASLM), which uses the full joint space of input and desired signal as the projection space and approaches the performance of nonlinear models.

Partitioning Relational Matrices of Similarities or Dissimilarities using the Value of Information

no code implementations28 Oct 2017 Isaac J. Sledge, Jose C. Principe

In this paper, we provide an approach to clustering relational matrices whose entries correspond to either similarities or dissimilarities between objects.

An Analysis of the Value of Information when Exploring Stochastic, Discrete Multi-Armed Bandits

no code implementations8 Oct 2017 Isaac J. Sledge, Jose C. Principe

High amounts of policy information are associated with exploration-dominant searches of the space and yield high rewards.

Multi-Armed Bandits

Robust period estimation using mutual information for multi-band light curves in the synoptic survey era

2 code implementations11 Sep 2017 Pablo Huijse, Pablo A. Estevez, Francisco Forster, Scott F. Daniel, Andrew J. Connolly, Pavlos Protopapas, Rodrigo Carrasco, Jose C. Principe

Robust and efficient methods that can aggregate data from multidimensional sparsely-sampled time series are needed.

Instrumentation and Methods for Astrophysics Information Theory Information Theory

Associations among Image Assessments as Cost Functions in Linear Decomposition: MSE, SSIM, and Correlation Coefficient

no code implementations4 Aug 2017 Jianji Wang, Nanning Zheng, Badong Chen, Jose C. Principe

Moreover, for a target vector, the ratio of the corresponding affine parameters in the MSE-based linear decomposition scheme and the SSIM-based scheme is a constant, which is just the value of PCC between the target vector and its estimated vector.


Group-Wise Point-Set Registration Based on Renyi's Second Order Entropy

no code implementations CVPR 2017 Luis G. Sanchez Giraldo, Erion Hasanbelliu, Murali Rao, Jose C. Principe

In this paper, we describe a set of robust algorithms for group-wise registration using both rigid and non-rigid transformations of multiple unlabelled point-sets with no bias toward a given set.

Robustness of Maximum Correntropy Estimation Against Large Outliers

no code implementations23 Mar 2017 Badong Chen, Lei Xing, Haiquan Zhao, Bin Xu, Jose C. Principe

The maximum correntropy criterion (MCC) has recently been successfully applied in robust regression, classification and adaptive filtering, where the correntropy is maximized instead of minimizing the well-known mean square error (MSE) to improve the robustness with respect to outliers (or impulsive noises).

Analysis of Agent Expertise in Ms. Pac-Man using Value-of-Information-based Policies

no code implementations28 Feb 2017 Isaac J. Sledge, Jose C. Principe

This cost function is the value of information, which provides the optimal trade-off between the expected return of a policy and the policy's complexity; policy complexity is measured by number of bits and controlled by a single hyperparameter on the cost function.


Exploiting Spatio-Temporal Structure with Recurrent Winner-Take-All Networks

no code implementations31 Oct 2016 Eder Santana, Matthew Emigh, Pablo Zegers, Jose C. Principe

We propose a convolutional recurrent neural network, with Winner-Take-All dropout for high dimensional unsupervised feature learning in multi-dimensional time series.

Object Recognition Time Series

Kernel Risk-Sensitive Loss: Definition, Properties and Application to Robust Adaptive Filtering

no code implementations1 Aug 2016 Badong Chen, Lei Xing, Bin Xu, Haiquan Zhao, Nanning Zheng, Jose C. Principe

Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher-order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non-Gaussian signal processing and machine learning.

Information Theoretic-Learning Auto-Encoder

no code implementations22 Mar 2016 Eder Santana, Matthew Emigh, Jose C. Principe

We propose Information Theoretic-Learning (ITL) divergence measures for variational regularization of neural networks.

Kernel Least Mean Square with Adaptive Kernel Size

no code implementations23 Jan 2014 Badong Chen, Junli Liang, Nanning Zheng, Jose C. Principe

Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS).

Time Series Prediction

Rate-Distortion Auto-Encoders

no code implementations28 Dec 2013 Luis G. Sanchez Giraldo, Jose C. Principe

Here, we propose a learning algorithm for auto-encoders based on a rate-distortion objective that minimizes the mutual information between the inputs and the outputs of the auto-encoder subject to a fidelity constraint.

Information Theoretic Learning with Infinitely Divisible Kernels

no code implementations16 Jan 2013 Luis G. Sanchez Giraldo, Jose C. Principe

In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices.

Metric Learning

Measures of Entropy from Data Using Infinitely Divisible Kernels

no code implementations11 Nov 2012 Luis G. Sanchez Giraldo, Murali Rao, Jose C. Principe

In this way, capitalizing on both the axiomatic definition of entropy and on the representation power of positive definite kernels, the proposed measure of entropy avoids the estimation of the probability distribution underlying the data.

Dimensionality Reduction Two-sample testing

Cannot find the paper you are looking for? You can Submit a new open access paper.