Search Results for author:
Found 24 papers, 6 papers with code
Intrinsic Motivation in Dynamical Control Systems
Biological systems often choose actions without an explicit reward signal, a phenomenon known as intrinsic motivation.
Detecting chaos in lineage-trees: A deep learning approach
Estimating the largest Lyapunov exponent from observations of a process is especially challenging in systems affected by dynamical noise, which is the case for many models of real-world processes, in particular models of biological systems.
The Dual Information Bottleneck
The Information Bottleneck (IB) framework is a general characterization of optimal representations obtained using a principled approach for balancing accuracy and complexity.
Semantic categories of artifacts and animals reflect efficient coding
Recently, this idea has been cast in terms of a general information-theoretic principle of efficiency, the Information Bottleneck (IB) principle, and it has been shown that this principle accounts for the emergence and evolution of named color categories across languages, including soft structure and patterns of inconsistent naming.
REPRESENTATION COMPRESSION AND GENERALIZATION IN DEEP NEURAL NETWORKS
Specifically, we show that the training of the network is characterized by a rapid increase in the mutual information (MI) between the layers and the target label, followed by a longer decrease in the MI between the layers and the input variable.
Machine learning and the physical sciences
Machine learning encompasses a broad range of algorithms and modeling tools used for a vast array of data processing tasks, which has entered most scientific disciplines in recent years.
Computational Physics Cosmology and Nongalactic Astrophysics Disordered Systems and Neural Networks High Energy Physics - Theory Quantum Physics
Non-linear Canonical Correlation Analysis: A Compressed Representation Approach
In this work we introduce an information-theoretic compressed representation framework for the non-linear CCA problem (CRCCA), which extends the classical ACE approach.
Efficient human-like semantic representations via the Information Bottleneck principle
This work thus identifies a computational principle that characterizes human semantic systems, and that could usefully inform semantic representations in machines.
Color naming reflects both perceptual structure and communicative need
We show that greater communicative precision for warm than for cool colors, and greater communicative need, may both be explained by perceptual structure.
A General Memory-Bounded Learning Algorithm
Designing bounded-memory algorithms is becoming increasingly important nowadays.
Gaussian Lower Bound for the Information Bottleneck Limit
In this work we introduce a Gaussian lower bound to the IB curve; we find an embedding of the data which maximizes its "Gaussian part", on which we apply the GIB.
Mixing Complexity and its Applications to Neural Networks
We suggest analyzing neural networks through the prism of space constraints.
Opening the Black Box of Deep Neural Networks via Information
Previous work proposed to analyze DNNs in the \textit{Information Plane}; i. e., the plane of the Mutual Information values that each layer preserves on the input and output variables.
Principled Option Learning in Markov Decision Processes
It is well known that options can make planning more efficient, among their many benefits.
Memory shapes time perception and intertemporal choices
There is a consensus that human and non-human subjects experience temporal distortions in many stages of their perceptual and decision-making systems.
Optimal Selective Attention in Reactive Agents
One attempt to deal with this is to focus on reactive policies, that only base their actions on the most recent observation.
Taming the Noise in Reinforcement Learning via Soft Updates
We propose G-learning, a new off-policy learning algorithm that regularizes the value estimates by penalizing deterministic policies in the beginning of the learning process.
Information-Theoretic Bounded Rationality
Bounded rationality, that is, decision-making and planning under resource limitations, is widely regarded as an important open problem in artificial intelligence, reinforcement learning, computational neuroscience and economics.
Deep Learning and the Information Bottleneck Principle
Deep Neural Networks (DNNs) are analyzed via the theoretical framework of the information bottleneck (IB) principle.
Distribution-Dependent Sample Complexity of Large Margin Learning
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the margin-adapted dimension, which is a simple function of the second order statistics of the data distribution, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the margin-adapted dimension of the data distribution.
Tight Sample Complexity of Large-Margin Learning
We obtain a tight distribution-specific characterization of the sample complexity of large-margin classification with L2 regularization: We introduce the gamma-adapted-dimension, which is a simple function of the spectrum of a distribution's covariance matrix, and show distribution-specific upper and lower bounds on the sample complexity, both governed by the gamma-adapted-dimension of the source distribution.
On the Reliability of Clustering Stability in the Large Sample Regime
In this paper, we provide a set of general sufficient conditions, which ensure the reliability of clustering stability estimators in the large sample regime.
Efficient representation as a design principle for neural coding and computation
Does the brain construct an efficient representation of the sensory world?
The information bottleneck method
We define the relevant information in a signal $x\in X$ as being the information that this signal provides about another signal $y\in \Y$.
