Search Results for author: Jorg Bornschein

Found 15 papers, 2 papers with code

Denoising Autoregressive Representation Learning

no code implementations8 Mar 2024 Yazhe Li, Jorg Bornschein, Ting Chen

In this paper, we explore a new generative approach for learning visual representations.

Denoising Image Generation +1

Transformers for Supervised Online Continual Learning

no code implementations3 Mar 2024 Jorg Bornschein, Yazhe Li, Amal Rannen-Triki

Inspired by the in-context learning capabilities of transformers and their connection to meta-learning, we propose a method that leverages these strengths for online continual learning.

Continual Learning Few-Shot Learning +2

Kalman Filter for Online Classification of Non-Stationary Data

no code implementations14 Jun 2023 Michalis K. Titsias, Alexandre Galashov, Amal Rannen-Triki, Razvan Pascanu, Yee Whye Teh, Jorg Bornschein

Non-stationarity over the linear predictor weights is modelled using a parameter drift transition density, parametrized by a coefficient that quantifies forgetting.

Classification Continual Learning +1

Evaluating Representations with Readout Model Switching

no code implementations19 Feb 2023 Yazhe Li, Jorg Bornschein, Marcus Hutter

Although much of the success of Deep Learning builds on learning good representations, a rigorous method to evaluate their quality is lacking.

Model Selection

Bridging the Gap Between Offline and Online Reinforcement Learning Evaluation Methodologies

no code implementations15 Dec 2022 Shivakanth Sujit, Pedro H. M. Braga, Jorg Bornschein, Samira Ebrahimi Kahou

Offline RL algorithms try to address this issue by bootstrapping the learning process from existing logged data without needing to interact with the environment from the very beginning.

Offline RL reinforcement-learning +1

Sequential Learning Of Neural Networks for Prequential MDL

no code implementations14 Oct 2022 Jorg Bornschein, Yazhe Li, Marcus Hutter

In the prequential formulation of MDL, the objective is to minimize the cumulative next-step log-loss when sequentially going through the data and using previous observations for parameter estimation.

Image Classification

Learning to Induce Causal Structure

no code implementations11 Apr 2022 Nan Rosemary Ke, Silvia Chiappa, Jane Wang, Anirudh Goyal, Jorg Bornschein, Melanie Rey, Theophane Weber, Matthew Botvinic, Michael Mozer, Danilo Jimenez Rezende

The fundamental challenge in causal induction is to infer the underlying graph structure given observational and/or interventional data.

Prequential MDL for Causal Structure Learning with Neural Networks

no code implementations2 Jul 2021 Jorg Bornschein, Silvia Chiappa, Alan Malek, Rosemary Nan Ke

Learning the structure of Bayesian networks and causal relationships from observations is a common goal in several areas of science and technology.

A study on the plasticity of neural networks

no code implementations31 May 2021 Tudor Berariu, Wojciech Czarnecki, Soham De, Jorg Bornschein, Samuel Smith, Razvan Pascanu, Claudia Clopath

One aim shared by multiple settings, such as continual learning or transfer learning, is to leverage previously acquired knowledge to converge faster on the current task.

Continual Learning Transfer Learning

Small Data, Big Decisions: Model Selection in the Small-Data Regime

no code implementations ICML 2020 Jorg Bornschein, Francesco Visin, Simon Osindero

Highly overparametrized neural networks can display curiously strong generalization performance - a phenomenon that has recently garnered a wealth of theoretical and empirical research in order to better understand it.

Model Selection

Bidirectional Helmholtz Machines

1 code implementation12 Jun 2015 Jorg Bornschein, Samira Shabanian, Asja Fischer, Yoshua Bengio

We present a lower-bound for the likelihood of this model and we show that optimizing this bound regularizes the model so that the Bhattacharyya distance between the bottom-up and top-down approximate distributions is minimized.

Towards Biologically Plausible Deep Learning

no code implementations14 Feb 2015 Yoshua Bengio, Dong-Hyun Lee, Jorg Bornschein, Thomas Mesnard, Zhouhan Lin

Neuroscientists have long criticised deep learning algorithms as incompatible with current knowledge of neurobiology.

Denoising Representation Learning

Cannot find the paper you are looking for? You can Submit a new open access paper.