Universal Physics Transformers

no code implementations • 19 Feb 2024 • Benedikt Alkin, Andreas Fürst, Simon Schmid, Lukas Gruber, Markus Holzleitner, Johannes Brandstetter

Deep neural network based surrogates for partial differential equations have recently gained increased interest.

Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget

1 code implementation • 20 Apr 2023 • Johannes Lehner, Benedikt Alkin, Andreas Fürst, Elisabeth Rumetshofer, Lukas Miklautz, Sepp Hochreiter

In this work, we study how to combine the efficiency and scalability of MIM with the ability of ID to perform downstream classification in the absence of large amounts of labeled data.

Clustering Contrastive Learning +2

CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

1 code implementation • 21 Oct 2021 • Andreas Fürst, Elisabeth Rumetshofer, Johannes Lehner, Viet Tran, Fei Tang, Hubert Ramsauer, David Kreil, Michael Kopp, Günter Klambauer, Angela Bitto-Nemling, Sepp Hochreiter

We suggest to use modern Hopfield networks to tackle the problem of explaining away.

Contrastive Learning Self-Supervised Learning +1