Search Results for author:
Found 60 papers, 11 papers with code
Robust period estimation using mutual information for multi-band light curves in the synoptic survey era
Robust and efficient methods that can aggregate data from multidimensional sparsely-sampled time series are needed.
Instrumentation and Methods for Astrophysics Information Theory Information Theory
Deep Deterministic Information Bottleneck with Matrix-based Entropy Functional
We introduce the matrix-based Renyi's $\alpha$-order entropy functional to parameterize Tishby et al. information bottleneck (IB) principle with a neural network.
Interpretable Fault Detection using Projections of Mutual Information Matrix
This paper presents a novel mutual information (MI) matrix based method for fault detection.
Deep Deterministic Independent Component Analysis for Hyperspectral Unmixing
We develop a new neural network based independent component analysis (ICA) method by directly minimizing the dependence amongst all extracted components.
Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications
We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions.
Measuring Dependence with Matrix-based Entropy Functional
Measuring the dependence of data plays a central role in statistics and machine learning.
Principle of Relevant Information for Graph Sparsification
Graph sparsification aims to reduce the number of edges of a graph while maintaining its structural properties.
Estimating Rényi's $α$-Cross-Entropies in a Matrix-Based Way
This yields matrix-based estimators of R\'enyi's $\alpha$-cross-entropies.
Multivariate Extension of Matrix-based Renyi's α-order Entropy Functional
The matrix-based Renyi's \alpha-order entropy functional was recently introduced using the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).
Understanding Convolutional Neural Networks with Information Theory: An Initial Exploration
The matrix-based Renyi's \alpha-entropy functional and its multivariate extension were recently developed in terms of the normalized eigenspectrum of a Hermitian matrix of the projected data in a reproducing kernel Hilbert space (RKHS).
Understanding Autoencoders with Information Theoretic Concepts
Despite their great success in practical applications, there is still a lack of theoretical and systematic methods to analyze deep neural networks.
An Analysis of the Value of Information when Exploring Stochastic, Discrete Multi-Armed Bandits
High amounts of policy information are associated with exploration-dominant searches of the space and yield high rewards.
Guided Policy Exploration for Markov Decision Processes using an Uncertainty-Based Value-of-Information Criterion
The value of information yields a stochastic routine for choosing actions during learning that can explore the policy space in a coarse to fine manner.
Augmented Space Linear Model
In this paper, we propose a linear model called Augmented Space Linear Model (ASLM), which uses the full joint space of input and desired signal as the projection space and approaches the performance of nonlinear models.
Robustness of Maximum Correntropy Estimation Against Large Outliers
The maximum correntropy criterion (MCC) has recently been successfully applied in robust regression, classification and adaptive filtering, where the correntropy is maximized instead of minimizing the well-known mean square error (MSE) to improve the robustness with respect to outliers (or impulsive noises).
Analysis of Agent Expertise in Ms. Pac-Man using Value-of-Information-based Policies
This cost function is the value of information, which provides the optimal trade-off between the expected return of a policy and the policy's complexity; policy complexity is measured by number of bits and controlled by a single hyperparameter on the cost function.
Partitioning Relational Matrices of Similarities or Dissimilarities using the Value of Information
In this paper, we provide an approach to clustering relational matrices whose entries correspond to either similarities or dissimilarities between objects.
Associations among Image Assessments as Cost Functions in Linear Decomposition: MSE, SSIM, and Correlation Coefficient
Moreover, for a target vector, the ratio of the corresponding affine parameters in the MSE-based linear decomposition scheme and the SSIM-based scheme is a constant, which is just the value of PCC between the target vector and its estimated vector.
Exploiting Spatio-Temporal Structure with Recurrent Winner-Take-All Networks
We propose a convolutional recurrent neural network, with Winner-Take-All dropout for high dimensional unsupervised feature learning in multi-dimensional time series.
Kernel Risk-Sensitive Loss: Definition, Properties and Application to Robust Adaptive Filtering
Nonlinear similarity measures defined in kernel space, such as correntropy, can extract higher-order statistics of data and offer potentially significant performance improvement over their linear counterparts especially in non-Gaussian signal processing and machine learning.
Information Theoretic-Learning Auto-Encoder
We propose Information Theoretic-Learning (ITL) divergence measures for variational regularization of neural networks.
Computational Intelligence Challenges and Applications on Large-Scale Astronomical Time Series Databases
In this article we present an overview of machine learning and computational intelligence applications to TDA.
Measures of Entropy from Data Using Infinitely Divisible Kernels
In this way, capitalizing on both the axiomatic definition of entropy and on the representation power of positive definite kernels, the proposed measure of entropy avoids the estimation of the probability distribution underlying the data.
Rate-Distortion Auto-Encoders
Here, we propose a learning algorithm for auto-encoders based on a rate-distortion objective that minimizes the mutual information between the inputs and the outputs of the auto-encoder subject to a fidelity constraint.
Kernel Least Mean Square with Adaptive Kernel Size
Kernel adaptive filters (KAF) are a class of powerful nonlinear filters developed in Reproducing Kernel Hilbert Space (RKHS).
Information Theoretic Learning with Infinitely Divisible Kernels
In this paper, we develop a framework for information theoretic learning based on infinitely divisible matrices.
Request-and-Reverify: Hierarchical Hypothesis Testing for Concept Drift Detection with Expensive Labels
In this paper, a novel Hierarchical Hypothesis Testing framework with Request-and-Reverify strategy is developed to detect concept drifts by requesting labels only when necessary.
Simple stopping criteria for information theoretic feature selection
Feature selection aims to select the smallest feature subset that yields the minimum generalization error.
Theory and Algorithms for Pulse Signal Processing
The integrate and fire converter transforms an analog signal into train of biphasic pulses.
An Exact Reformulation of Feature-Vector-based Radial-Basis-Function Networks for Graph-based Observations
An important property of our radial basis function networks is that they are guaranteed to yield the same responses as conventional radial-basis networks trained on a corresponding vector realization of the relationships encoded by the adjacency-matrix.
Group-Wise Point-Set Registration Based on Renyi's Second Order Entropy
In this paper, we describe a set of robust algorithms for group-wise registration using both rigid and non-rigid transformations of multiple unlabelled point-sets with no bias toward a given set.
Maximum Correntropy Criterion with Variable Center
The kernel function in correntropy is usually restricted to the Gaussian function with center located at zero.
Multiscale Principle of Relevant Information for Hyperspectral Image Classification
This paper proposes a novel architecture, termed multiscale principle of relevant information (MPRI), to learn discriminative spectral-spatial features for hyperspectral image (HSI) classification.
Algorithmic Design and Implementation of Unobtrusive Multistatic Serial LiDAR Image
To fully understand interactions between marine hydrokinetic (MHK) equipment and marine animals, a fast and effective monitoring system is required to capture relevant information whenever underwater animals appear.
Functional Bayesian Filter
We present a general nonlinear Bayesian filter for high-dimensional state estimation using the theory of reproducing kernel Hilbert space (RKHS).
No-Trick (Treat) Kernel Adaptive Filtering using Deterministic Features
Without loss of generality, we apply this approach to classical adaptive filtering algorithms and validate the methodology to show that deterministic features are faster to generate and outperform state-of-the-art kernel methods based on random Fourier features.
Fast Estimation of Information Theoretic Learning Descriptors using Explicit Inner Product Spaces
The inner product defined by the feature mapping corresponds to a positive-definite finite-rank kernel that induces a finite-dimensional reproducing kernel Hilbert space (RKHS).
Towards a Kernel based Uncertainty Decomposition Framework for Data and Models
This paper introduces a new framework for quantifying predictive uncertainty for both data and models that relies on projecting the data into a Gaussian reproducing kernel Hilbert space (RKHS) and transforming the data probability density function (PDF) in a way that quantifies the flow of its gradient as a topological potential field quantified at all points in the sample space.
PRI-VAE: Principle-of-Relevant-Information Variational Autoencoders
Although substantial efforts have been made to learn disentangled representations under the variational autoencoder (VAE) framework, the fundamental properties to the dynamics of learning of most VAE models still remain unknown and under-investigated.
Unsupervised Foveal Vision Neural Networks with Top-Down Attention
The results in foveated vision show that Gamma saliency is comparable to the best and computationally faster.
Local power estimation of neuromodulations using point process modeling
A detailed analysis of the power - specific marked features of neuromodulations confirm high correlation between power spectral density and power in neuromodulations establishing the aptness of MPP spectrogram as a finer measure of power where it is able to track local variations in power while preserving the global structure of signal power distribution.
Target Detection and Segmentation in Circular-Scan Synthetic-Aperture-Sonar Images using Semi-Supervised Convolutional Encoder-Decoders
We show that our framework outperforms supervised, deep-saliency networks designed for natural imagery.
Training Deep Architectures Without End-to-End Backpropagation: A Survey on the Provably Optimal Methods
This tutorial paper surveys provably optimal alternatives to end-to-end backpropagation (E2EBP) -- the de facto standard for training deep architectures.
Faster Convergence in Deep-Predictive-Coding Networks to Learn Deeper Representations
It yields unsupervised object recognition that surpass convolutional autoencoders and are on par with convolutional networks trained in a supervised manner.
Annotating Motion Primitives for Simplifying Action Search in Reinforcement Learning
We have previously developed a means of constraining, and hence speeding up, the search process through the use of motion primitives; motion primitives are sequences of pre-specified actions taken across a state series.
A Kernel Framework to Quantify a Model's Local Predictive Uncertainty under Data Distributional Shifts
We therefore propose a framework for predictive uncertainty quantification of a trained neural network that explicitly estimates the PDF of its raw prediction space (before activation), p(y'|x, w), which we refer to as the model PDF, in a Gaussian reproducing kernel Hilbert space (RKHS).
Labels, Information, and Computation: Efficient Learning Using Sufficient Labels
We call this statistic "sufficiently-labeled data" and prove its sufficiency and efficiency for finding the optimal hidden representations, on which competent classifier heads can be trained using as few as a single randomly-chosen fully-labeled example per class.
An Information-Theoretic Approach for Automatically Determining the Number of States when Aggregating Markov Chains
A fundamental problem when aggregating Markov chains is the specification of the number of state groups.
External-Memory Networks for Low-Shot Learning of Targets in Forward-Looking-Sonar Imagery
We propose a memory-based framework for real-time, data-efficient target analysis in forward-looking-sonar (FLS) imagery.
Analysis of Intra-Operative Physiological Responses Through Complex Higher-Order SVD for Long-Term Post-Operative Pain Prediction
The dynamics of patients' physiological responses to these surgical events are linked to long-term post-operative pain development.
A Physics inspired Functional Operator for Model Uncertainty Quantification in the RKHS
The RKHS projection of model weights yields a potential field based interpretation of model weight PDF which consequently allows the definition of a functional operator, inspired by perturbation theory in physics, that performs a moment decomposition of the model weight PDF (the potential field) at a specific model output to quantify its uncertainty.
Information Theoretic Structured Generative Modeling
We test the framework for estimation of mutual information and compare the results with the mutual information neural estimation (MINE), for density estimation, for conditional probability estimation in Markov models as well as for training adversarial networks.
Quantifying Model Uncertainty for Semantic Segmentation using Operators in the RKHS
We present a simple framework for high-resolution predictive uncertainty quantification of semantic segmentation models that leverages a multi-moment functional definition of uncertainty associated with the model's feature space in the reproducing kernel Hilbert space (RKHS).
Robust Dependence Measure using RKHS based Uncertainty Moments and Optimal Transport
Being based on the Gaussian RKHS, our approach is robust towards outliers and monotone transformations of data, while the multiple moments of uncertainty provide high resolution and interpretability of the type of dependence being quantified.
The Normalized Cross Density Functional: A Framework to Quantify Statistical Dependence for Random Processes
We mathematically prove that FMCA learns the dominant eigenvalues and eigenfunctions of NCD directly from realizations.
Adapting the Exploration Rate for Value-of-Information-Based Reinforcement Learning
We then illustrate that these trends hold for deep, value-of-information-based agents that learn to play ten simple games and over forty more complicated games for the Nintendo GameBoy system.
The Functional Wiener Filter
Because of the lack of congruence between the Gaussian RKHS and the space of time series, we compare performance of two pre-imaging algorithms: a fixed-point optimization (FWFFP) that finds and approximate solution in the RKHS, and a local model implementation named FWFLM.
An Alternate View on Optimal Filtering in an RKHS
Kernel Adaptive Filtering (KAF) are mathematically principled methods which search for a function in a Reproducing Kernel Hilbert Space.
Weakly-Supervised Semantic Segmentation of Circular-Scan, Synthetic-Aperture-Sonar Imagery
We propose a weakly-supervised framework for the semantic segmentation of circular-scan synthetic-aperture-sonar (CSAS) imagery.
An Analytic Solution for Kernel Adaptive Filtering
We provide an extended functional Wiener equation, and present a solution to this equation in an explicit, finite dimensional, data-dependent RKHS.
