Search Results for author: Thomas B. Schön

Found 74 papers, 35 papers with code

Surprises in adversarially-trained linear regression

no code implementations25 May 2022 Antônio H. Ribeiro, Dave Zachariah, Thomas B. Schön

We prove that adversarial training with small disturbances gives the solution with the minimum-norm that interpolates the training data.

regression

Probabilistic Estimation of Chirp Instantaneous Frequency Using Gaussian Processes

1 code implementation12 May 2022 Zheng Zhao, Simo Särkkä, Jens Sjölund, Thomas B. Schön

We present a probabilistic approach for estimating chirp signal and its instantaneous frequency function when the true forms of the chirp and instantaneous frequency are unknown.

Gaussian Processes

Overparameterized Linear Regression under Adversarial Attacks

no code implementations13 Apr 2022 Antônio H. Ribeiro, Thomas B. Schön

This is then used as a tool to study how adversarial training and other regularization methods might affect the robustness of the estimated models.

regression

Incorporating Sum Constraints into Multitask Gaussian Processes

1 code implementation3 Feb 2022 Philipp Pilar, Carl Jidling, Thomas B. Schön, Niklas Wahlström

Machine learning models can be improved by adapting them to respect existing background knowledge.

Gaussian Processes

Efficient Learning of the Parameters of Non-Linear Models using Differentiable Resampling in Particle Filters

no code implementations2 Nov 2021 Conor Rosato, Vincent Beraud, Paul Horridge, Thomas B. Schön, Simon Maskell

It has been widely documented that the sampling and resampling steps in particle filters cannot be differentiated.

Learning Proposals for Practical Energy-Based Regression

1 code implementation22 Oct 2021 Fredrik K. Gustafsson, Martin Danelljan, Thomas B. Schön

Energy-based models (EBMs) have experienced a resurgence within machine learning in recent years, including as a promising alternative for probabilistic regression.

regression

Learning deep autoregressive models for hierarchical data

no code implementations28 Apr 2021 Carl R. Andersson, Niklas Wahlström, Thomas B. Schön

We propose a model for hierarchical structured data as an extension to the stochastic temporal convolutional network.

Unsupervised dynamic modeling of medical image transformation

2 code implementations1 Mar 2021 Niklas Gunnarsson, Peter Kimstrand, Jens Sjölund, Thomas B. Schön

For this, we use a conditional variational auto-encoder (CVAE) to nonlinearly map the higher-dimensional image to a lower-dimensional space, wherein we model the dynamics with a linear Gaussian state-space model (LG-SSM).

Denoising Dimensionality Reduction +3

How Convolutional Neural Networks Deal with Aliasing

1 code implementation15 Feb 2021 Antônio H. Ribeiro, Thomas B. Schön

The convolutional neural network (CNN) remains an essential tool in solving computer vision problems.

Beyond Occam's Razor in System Identification: Double-Descent when Modeling Dynamics

1 code implementation11 Dec 2020 Antônio H. Ribeiro, Johannes N. Hendriks, Adrian G. Wills, Thomas B. Schön

It is typically observed that the model validation performance follows a U-shaped curve as the model complexity increases.

Accurate 3D Object Detection using Energy-Based Models

1 code implementation8 Dec 2020 Fredrik K. Gustafsson, Martin Danelljan, Thomas B. Schön

On the KITTI dataset, our proposed approach consistently outperforms the SA-SSD baseline across all 3DOD metrics, demonstrating the potential of EBM-based regression for highly accurate 3DOD.

2D object detection 3D Object Detection +2

Deep Energy-Based NARX Models

1 code implementation8 Dec 2020 Johannes N. Hendriks, Fredrik K. Gustafsson, Antônio H. Ribeiro, Adrian G. Wills, Thomas B. Schön

This paper is directed towards the problem of learning nonlinear ARX models based on system input--output data.

How to Train Your Energy-Based Model for Regression

1 code implementation4 May 2020 Fredrik K. Gustafsson, Martin Danelljan, Radu Timofte, Thomas B. Schön

While they are commonly employed for generative image modeling, recent work has applied EBMs also for regression tasks, achieving state-of-the-art performance on object detection and visual tracking.

object-detection Object Detection +3

Deep State Space Models for Nonlinear System Identification

1 code implementation31 Mar 2020 Daniel Gedon, Niklas Wahlström, Thomas B. Schön, Lennart Ljung

Deep state space models (SSMs) are an actively researched model class for temporal models developed in the deep learning community which have a close connection to classic SSMs.

Registration by tracking for sequential 2D MRI

no code implementations24 Mar 2020 Niklas Gunnarsson, Jens Sjölund, Thomas B. Schön

Together with a sparse-to-dense interpolation scheme we can then estimate of the displacement field.

Anatomy Image Registration

Gaussian Variational State Estimation for Nonlinear State-Space Models

no code implementations7 Feb 2020 Jarrad Courts, Adrian Wills, Thomas B. Schön

In this paper, the problem of state estimation, in the context of both filtering and smoothing, for nonlinear state-space models is considered.

Variational Inference

Particle filter with rejection control and unbiased estimator of the marginal likelihood

no code implementations21 Oct 2019 Jan Kudlicka, Lawrence M. Murray, Thomas B. Schön, Fredrik Lindsten

While the variance reducing properties of rejection control are known, there has not been (to the best of our knowledge) any work on unbiased estimation of the marginal likelihood (also known as the model evidence or the normalizing constant) in this type of particle filter.

Energy-Based Models for Deep Probabilistic Regression

1 code implementation ECCV 2020 Fredrik K. Gustafsson, Martin Danelljan, Goutam Bhat, Thomas B. Schön

In our proposed approach, we create an energy-based model of the conditional target density p(y|x), using a deep neural network to predict the un-normalized density from (x, y).

 Ranked #1 on Object Detection on COCO test-dev (Hardware Burden metric)

Head Pose Estimation object-detection +4

Deep Convolutional Networks in System Identification

1 code implementation4 Sep 2019 Carl Andersson, Antônio H. Ribeiro, Koen Tiels, Niklas Wahlström, Thomas B. Schön

Recent developments within deep learning are relevant for nonlinear system identification problems.

Deep kernel learning for integral measurements

no code implementations4 Sep 2019 Carl Jidling, Johannes Hendriks, Thomas B. Schön, Adrian Wills

Deep kernel learning refers to a Gaussian process that incorporates neural networks to improve the modelling of complex functions.

Probabilistic programming for birth-death models of evolution using an alive particle filter with delayed sampling

1 code implementation10 Jul 2019 Jan Kudlicka, Lawrence M. Murray, Fredrik Ronquist, Thomas B. Schön

Probabilistic programming languages (PPLs) give phylogeneticists a new and exciting tool: their models can be implemented as probabilistic programs with just a basic knowledge of programming.

Probabilistic Programming

Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness

1 code implementation20 Jun 2019 Antônio H. Ribeiro, Koen Tiels, Luis A. Aguirre, Thomas B. Schön

The exploding and vanishing gradient problem has been the major conceptual principle behind most architecture and training improvements in recurrent neural networks (RNNs) during the last decade.

Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision

1 code implementation4 Jun 2019 Fredrik K. Gustafsson, Martin Danelljan, Thomas B. Schön

We therefore accept this task and propose a comprehensive evaluation framework for scalable epistemic uncertainty estimation methods in deep learning.

Depth Completion Semantic Segmentation

Robust exploration in linear quadratic reinforcement learning

1 code implementation NeurIPS 2019 Jack Umenberger, Mina Ferizbegovic, Thomas B. Schön, Håkan Hjalmarsson

This paper concerns the problem of learning control policies for an unknown linear dynamical system to minimize a quadratic cost function.

reinforcement-learning

On the smoothness of nonlinear system identification

1 code implementation2 May 2019 Antônio H. Ribeiro, Koen Tiels, Jack Umenberger, Thomas B. Schön, Luis A. Aguirre

We shed new light on the \textit{smoothness} of optimization problems arising in prediction error parameter estimation of linear and nonlinear systems.

Elements of Sequential Monte Carlo

no code implementations12 Mar 2019 Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön

A core problem in statistics and probabilistic machine learning is to compute probability distributions and expectations.

Bayesian Inference BIG-bench Machine Learning +1

Nonlinear input design as optimal control of a Hamiltonian system

no code implementations6 Mar 2019 Jack Umenberger, Thomas B. Schön

We propose an input design method for a general class of parametric probabilistic models, including nonlinear dynamical systems with process noise.

Evaluating model calibration in classification

1 code implementation19 Feb 2019 Juozas Vaicenavicius, David Widmann, Carl Andersson, Fredrik Lindsten, Jacob Roll, Thomas B. Schön

Probabilistic classifiers output a probability distribution on target classes rather than just a class prediction.

Classification Decision Making +1

Constructing the Matrix Multilayer Perceptron and its Application to the VAE

no code implementations4 Feb 2019 Jalil Taghia, Maria Bånkestad, Fredrik Lindsten, Thomas B. Schön

However, in certain scenarios we are interested in learning structured parameters (predictions) in the form of symmetric positive definite matrices.

Inferring Heterogeneous Causal Effects in Presence of Spatial Confounding

1 code implementation28 Jan 2019 Muhammad Osama, Dave Zachariah, Thomas B. Schön

We address the problem of inferring the causal effect of an exposure on an outcome across space, using observational data.

Automatic Diagnosis of Short-Duration 12-Lead ECG using a Deep Convolutional Network

no code implementations28 Nov 2018 Antônio H. Ribeiro, Manoel Horta Ribeiro, Gabriela Paixão, Derick Oliveira, Paulo R. Gomes, Jéssica A. Canazart, Milton Pifano, Wagner Meira Jr., Thomas B. Schön, Antonio Luiz Ribeiro

We present a model for predicting electrocardiogram (ECG) abnormalities in short-duration 12-lead ECG signals which outperformed medical doctors on the 4th year of their cardiology residency.

Electrocardiography (ECG)

Reliable Semi-Supervised Learning when Labels are Missing at Random

no code implementations27 Nov 2018 Xiuming Liu, Dave Zachariah, Johan Wågberg, Thomas B. Schön

Semi-supervised learning methods are motivated by the availability of large datasets with unlabeled features in addition to labeled data.

General Classification

Automated learning with a probabilistic programming language: Birch

no code implementations2 Oct 2018 Lawrence M. Murray, Thomas B. Schön

We introduce a formal description of models expressed as programs, and discuss some of the ways in which probabilistic programming languages can reveal the structure and form of these, in order to tailor inference methods.

Multiple Object Tracking Probabilistic Programming

Probabilistic approach to limited-data computed tomography reconstruction

no code implementations11 Sep 2018 Zenith Purisha, Carl Jidling, Niklas Wahlström, Simo Särkkä, Thomas B. Schön

The approach also allows for reformulation of come classical regularization methods as Laplacian and Tikhonov regularization as Gaussian process regression, and hence provides an efficient algorithm and principled means for their parameter tuning.

Numerical Integration

Data Consistency Approach to Model Validation

1 code implementation17 Aug 2018 Andreas Svensson, Dave Zachariah, Petre Stoica, Thomas B. Schön

The contribution in this paper is a general criterion to evaluate the consistency of a set of statistical models with respect to observed data.

Time Series

Learning convex bounds for linear quadratic control policy synthesis

no code implementations NeurIPS 2018 Jack Umenberger, Thomas B. Schön

Learning to make decisions from observed data in dynamic environments remains a problem of fundamental importance in a number of fields, from artificial intelligence and robotics, to medicine and finance.

Conditionally Independent Multiresolution Gaussian Processes

1 code implementation25 Feb 2018 Jalil Taghia, Thomas B. Schön

This in turn results in models which are prone to overfitting in the sense of excessive sensitivity to the chosen resolution, and predictions which are non-smooth at the boundaries.

Bayesian Inference Gaussian Processes

Learning Localized Spatio-Temporal Models From Streaming Data

1 code implementation ICML 2018 Muhammad Osama, Dave Zachariah, Thomas B. Schön

We address the problem of predicting spatio-temporal processes with temporal patterns that vary across spatial regions, when data is obtained as a stream.

Data-Driven Impulse Response Regularization via Deep Learning

no code implementations25 Jan 2018 Carl Andersson, Niklas Wahlström, Thomas B. Schön

We consider the problem of impulse response estimation of stable linear single-input single-output systems.

How consistent is my model with the data? Information-Theoretic Model Check

1 code implementation7 Dec 2017 Andreas Svensson, Dave Zachariah, Thomas B. Schön

The choice of model class is fundamental in statistical learning and system identification, no matter whether the class is derived from physical principles or is a generic black-box.

Delayed Sampling and Automatic Rao-Blackwellization of Probabilistic Programs

1 code implementation25 Aug 2017 Lawrence M. Murray, Daniel Lundén, Jan Kudlicka, David Broman, Thomas B. Schön

For inference with Sequential Monte Carlo, this automatically yields improvements such as locally-optimal proposals and Rao-Blackwellization.

Probabilistic Programming

Using Inertial Sensors for Position and Orientation Estimation

1 code implementation20 Apr 2017 Manon Kok, Jeroen D. Hol, Thomas B. Schön

In this tutorial we focus on the signal processing aspects of position and orientation estimation using inertial sensors.

Robotics Systems and Control

On the construction of probabilistic Newton-type algorithms

no code implementations5 Apr 2017 Adrian G. Wills, Thomas B. Schön

It has recently been shown that many of the existing quasi-Newton algorithms can be formulated as learning algorithms, capable of learning local models of the cost functions.

Online Learning for Distribution-Free Prediction

1 code implementation15 Mar 2017 Dave Zachariah, Petre Stoica, Thomas B. Schön

We develop an online learning method for prediction, which is important in problems with large and/or streaming data sets.

online learning

Probabilistic learning of nonlinear dynamical systems using sequential Monte Carlo

no code implementations7 Mar 2017 Thomas B. Schön, Andreas Svensson, Lawrence Murray, Fredrik Lindsten

We are concerned with the problem of learning probabilistic models of dynamical systems from measured data.

Linearly constrained Gaussian processes

no code implementations NeurIPS 2017 Carl Jidling, Niklas Wahlström, Adrian Wills, Thomas B. Schön

We consider a modification of the covariance function in Gaussian processes to correctly account for known linear constraints.

Gaussian Processes

Learning of state-space models with highly informative observations: a tempered Sequential Monte Carlo solution

no code implementations6 Feb 2017 Andreas Svensson, Thomas B. Schön, Fredrik Lindsten

In particular, for learning of unknown parameters in nonlinear state-space models, methods based on the particle filter (a Monte Carlo method) have proven very useful.

Smoothing with Couplings of Conditional Particle Filters

1 code implementation8 Jan 2017 Pierre E. Jacob, Fredrik Lindsten, Thomas B. Schön

The method combines two recent breakthroughs: the first is a generic debiasing technique for Markov chains due to Rhee and Glynn, and the second is the introduction of a uniformly ergodic Markov chain for smoothing, the conditional particle filter of Andrieu, Doucet and Holenstein.

Methodology Computation

High-dimensional Filtering using Nested Sequential Monte Carlo

no code implementations29 Dec 2016 Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön

Sequential Monte Carlo (SMC) methods comprise one of the most successful approaches to approximate Bayesian filtering.

Prediction performance after learning in Gaussian process regression

no code implementations13 Jun 2016 Johan Wågberg, Dave Zachariah, Thomas B. Schön, Petre Stoica

Starting from a generalization of the Cram\'er-Rao bound, we derive a more accurate MSE bound which provides a measure of uncertainty for prediction of Gaussian processes.

Gaussian Processes regression

A flexible state space model for learning nonlinear dynamical systems

no code implementations17 Mar 2016 Andreas Svensson, Thomas B. Schön

We consider a nonlinear state-space model with the state transition and observation functions expressed as basis function expansions.

Gaussian Processes

System Identification through Online Sparse Gaussian Process Regression with Input Noise

1 code implementation29 Jan 2016 Hildo Bijl, Thomas B. Schön, Jan-Willem van Wingerden, Michel Verhaegen

There has been a growing interest in using non-parametric regression methods like Gaussian Process (GP) regression for system identification.

regression

Accelerating pseudo-marginal Metropolis-Hastings by correlating auxiliary variables

no code implementations17 Nov 2015 Johan Dahlin, Fredrik Lindsten, Joel Kronander, Thomas B. Schön

Pseudo-marginal Metropolis-Hastings (pmMH) is a powerful method for Bayesian inference in models where the posterior distribution is analytical intractable or computationally costly to evaluate directly.

Bayesian Inference

Getting Started with Particle Metropolis-Hastings for Inference in Nonlinear Dynamical Models

1 code implementation5 Nov 2015 Johan Dahlin, Thomas B. Schön

This tutorial provides a gentle introduction to the particle Metropolis-Hastings (PMH) algorithm for parameter inference in nonlinear state-space models together with a software implementation in the statistical programming language R. We employ a step-by-step approach to develop an implementation of the PMH algorithm (and the particle filter within) together with the reader.

Data-Efficient Learning of Feedback Policies from Image Pixels using Deep Dynamical Models

no code implementations8 Oct 2015 John-Alexander M. Assael, Niklas Wahlström, Thomas B. Schön, Marc Peter Deisenroth

We consider a particularly important instance of this challenge, the pixels-to-torques problem, where an RL agent learns a closed-loop control policy ("torques") from pixel information only.

Model-based Reinforcement Learning reinforcement-learning

Bayesian optimisation for fast approximate inference in state-space models with intractable likelihoods

2 code implementations23 Jun 2015 Johan Dahlin, Mattias Villani, Thomas B. Schön

We consider the problem of approximate Bayesian parameter inference in non-linear state-space models with intractable likelihoods.

Bayesian Optimisation

Computationally Efficient Bayesian Learning of Gaussian Process State Space Models

no code implementations7 Jun 2015 Andreas Svensson, Arno Solin, Simo Särkkä, Thomas B. Schön

We present a procedure for efficient Bayesian learning in Gaussian process state space models, where the representation is formed by projecting the problem onto a set of approximate eigenfunctions derived from the prior covariance structure.

Gaussian Processes

Sequential Monte Carlo Methods for System Identification

no code implementations20 Mar 2015 Thomas B. Schön, Fredrik Lindsten, Johan Dahlin, Johan Wågberg, Christian A. Naesseth, Andreas Svensson, Liang Dai

One of the key challenges in identifying nonlinear and possibly non-Gaussian state space models (SSMs) is the intractability of estimating the system state.

Quasi-Newton particle Metropolis-Hastings

1 code implementation12 Feb 2015 Johan Dahlin, Fredrik Lindsten, Thomas B. Schön

A possible application is parameter inference in the challenging class of SSMs with intractable likelihoods.

Newton-based maximum likelihood estimation in nonlinear state space models

1 code implementation12 Feb 2015 Manon Kok, Johan Dahlin, Thomas B. Schön, Adrian Wills

Maximum likelihood (ML) estimation using Newton's method in nonlinear state space models (SSMs) is a challenging problem due to the analytical intractability of the log-likelihood and its gradient and Hessian.

Nested Sequential Monte Carlo Methods

1 code implementation9 Feb 2015 Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön

NSMC generalises the SMC framework by requiring only approximate, properly weighted, samples from the SMC proposal distribution, while still resulting in a correct SMC algorithm.

From Pixels to Torques: Policy Learning with Deep Dynamical Models

no code implementations8 Feb 2015 Niklas Wahlström, Thomas B. Schön, Marc Peter Deisenroth

In this paper, we consider one instance of this challenge, the pixels to torques problem, where an agent must learn a closed-loop control policy from pixel information only.

Model-based Reinforcement Learning reinforcement-learning

Marginalizing Gaussian Process Hyperparameters using Sequential Monte Carlo

no code implementations6 Feb 2015 Andreas Svensson, Johan Dahlin, Thomas B. Schön

Gaussian process regression is a popular method for non-parametric probabilistic modeling of functions.

regression

Learning deep dynamical models from image pixels

no code implementations28 Oct 2014 Niklas Wahlström, Thomas B. Schön, Marc Peter Deisenroth

In particular, we jointly learn a low-dimensional embedding of the observation by means of deep auto-encoders and a predictive transition model in this low-dimensional space.

Identification of jump Markov linear models using particle filters

no code implementations25 Sep 2014 Andreas Svensson, Thomas B. Schön, Fredrik Lindsten

Jump Markov linear models consists of a finite number of linear state space models and a discrete variable encoding the jumps (or switches) between the different linear models.

Divide-and-Conquer with Sequential Monte Carlo

3 code implementations19 Jun 2014 Fredrik Lindsten, Adam M. Johansen, Christian A. Naesseth, Bonnie Kirkpatrick, Thomas B. Schön, John Aston, Alexandre Bouchard-Côté

We propose a novel class of Sequential Monte Carlo (SMC) algorithms, appropriate for inference in probabilistic graphical models.

Sequential Monte Carlo for Graphical Models

no code implementations NeurIPS 2014 Christian A. Naesseth, Fredrik Lindsten, Thomas B. Schön

We propose a new framework for how to use sequential Monte Carlo (SMC) algorithms for inference in probabilistic graphical models (PGM).

Particle Gibbs with Ancestor Sampling

no code implementations3 Jan 2014 Fredrik Lindsten, Michael. I. Jordan, Thomas B. Schön

Particle Markov chain Monte Carlo (PMCMC) is a systematic way of combining the two main tools used for Monte Carlo statistical inference: sequential Monte Carlo (SMC) and Markov chain Monte Carlo (MCMC).

Identification of Gaussian Process State-Space Models with Particle Stochastic Approximation EM

no code implementations17 Dec 2013 Roger Frigola, Fredrik Lindsten, Thomas B. Schön, Carl E. Rasmussen

Gaussian process state-space models (GP-SSMs) are a very flexible family of models of nonlinear dynamical systems.

Particle Metropolis-Hastings using gradient and Hessian information

no code implementations4 Nov 2013 Johan Dahlin, Fredrik Lindsten, Thomas B. Schön

Particle Metropolis-Hastings (PMH) allows for Bayesian parameter inference in nonlinear state space models by combining Markov chain Monte Carlo (MCMC) and particle filtering.

Cannot find the paper you are looking for? You can Submit a new open access paper.