Search Results for author: Thomas Unterthiner

Found 21 papers, 16 papers with code

GradMax: Growing Neural Networks using Gradient Information

1 code implementation ICLR 2022 Utku Evci, Bart van Merriënboer, Thomas Unterthiner, Max Vladymyrov, Fabian Pedregosa

The architecture and the parameters of neural networks are often optimized independently, which requires costly retraining of the parameters whenever the architecture is modified.

Do Vision Transformers See Like Convolutional Neural Networks?

3 code implementations NeurIPS 2021 Maithra Raghu, Thomas Unterthiner, Simon Kornblith, Chiyuan Zhang, Alexey Dosovitskiy

Finally, we study the effect of (pretraining) dataset scale on intermediate features and transfer learning, and conclude with a discussion on connections to new architectures such as the MLP-Mixer.

Image Classification Transfer Learning

Differentiable Patch Selection for Image Recognition

no code implementations CVPR 2021 Jean-Baptiste Cordonnier, Aravindh Mahendran, Alexey Dosovitskiy, Dirk Weissenborn, Jakob Uszkoreit, Thomas Unterthiner

Neural Networks require large amounts of memory and compute to process high resolution images, even when only a small part of the image is actually informative for the task at hand.

Traffic Sign Recognition

Understanding Robustness of Transformers for Image Classification

no code implementations ICCV 2021 Srinadh Bhojanapalli, Ayan Chakrabarti, Daniel Glasner, Daliang Li, Thomas Unterthiner, Andreas Veit

We find that when pre-trained with a sufficient amount of data, ViT models are at least as robust as the ResNet counterparts on a broad range of perturbations.

General Classification Image Classification

Object-Centric Learning with Slot Attention

3 code implementations NeurIPS 2020 Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, Thomas Kipf

Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features.

Object Discovery

Predicting Neural Network Accuracy from Weights

no code implementations26 Feb 2020 Thomas Unterthiner, Daniel Keysers, Sylvain Gelly, Olivier Bousquet, Ilya Tolstikhin

Furthermore, the predictors are able to rank networks trained on different, unobserved datasets and with different architectures.

FVD: A new Metric for Video Generation

no code implementations ICLR Workshop DeepGenStruct 2019 Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, Sylvain Gelly

While recent generative models of video have had some success, current progress is hampered by the lack of qualitative metrics that consider visual quality, temporal coherence, and diversity of samples.

Representation Learning Video Generation

Interpretable Deep Learning in Drug Discovery

1 code implementation7 Mar 2019 Kristina Preuer, Günter Klambauer, Friedrich Rippmann, Sepp Hochreiter, Thomas Unterthiner

Without any means of interpretation, neural networks that predict molecular properties and bioactivities are merely black boxes.

Drug Discovery

Towards Accurate Generative Models of Video: A New Metric & Challenges

2 code implementations3 Dec 2018 Thomas Unterthiner, Sjoerd van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, Sylvain Gelly

To this extent we propose Fr\'{e}chet Video Distance (FVD), a new metric for generative models of video, and StarCraft 2 Videos (SCV), a benchmark of game play from custom starcraft 2 scenarios that challenge the current capabilities of generative models of video.

Representation Learning Starcraft +1

Fréchet ChemNet Distance: A metric for generative models for molecules in drug discovery

2 code implementations26 Mar 2018 Kristina Preuer, Philipp Renz, Thomas Unterthiner, Sepp Hochreiter, Günter Klambauer

We propose a novel distance measure between two sets of molecules, called Fr\'echet ChemNet distance (FCD), that can be used as an evaluation metric for generative models.

Drug Discovery

First Order Generative Adversarial Networks

1 code implementation ICML 2018 Calvin Seward, Thomas Unterthiner, Urs Bergmann, Nikolay Jetchev, Sepp Hochreiter

To formally describe an optimal update direction, we introduce a theoretical framework which allows the derivation of requirements on both the divergence and corresponding method for determining an update direction, with these requirements guaranteeing unbiased mini-batch updates in the direction of steepest descent.

Image Generation Text Generation

Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields

1 code implementation ICLR 2018 Thomas Unterthiner, Bernhard Nessler, Calvin Seward, Günter Klambauer, Martin Heusel, Hubert Ramsauer, Sepp Hochreiter

We prove that Coulomb GANs possess only one Nash equilibrium which is optimal in the sense that the model distribution equals the target distribution.

Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

14 code implementations23 Nov 2015 Djork-Arné Clevert, Thomas Unterthiner, Sepp Hochreiter

In contrast to ReLUs, ELUs have negative values which allows them to push mean unit activations closer to zero like batch normalization but with lower computational complexity.

General Classification Image Classification

Toxicity Prediction using Deep Learning

1 code implementation4 Mar 2015 Thomas Unterthiner, Andreas Mayr, Günter Klambauer, Sepp Hochreiter

The goal of this challenge was to assess the performance of computational methods in predicting the toxicity of chemical compounds.

Cannot find the paper you are looking for? You can Submit a new open access paper.