Search Results for author: Thomas Kipf

Found 35 papers, 23 papers with code

Video OWL-ViT: Temporally-consistent open-world localization in video

no code implementations22 Aug 2023 Georg Heigold, Matthias Minderer, Alexey Gritsenko, Alex Bewley, Daniel Keysers, Mario Lučić, Fisher Yu, Thomas Kipf

Our model is end-to-end trainable on video data and enjoys improved temporal consistency compared to tracking-by-detection baselines, while retaining the open-world capabilities of the backbone detector.

Object Localization

DORSal: Diffusion for Object-centric Representations of Scenes $\textit{et al.}$

no code implementations13 Jun 2023 Allan Jabri, Sjoerd van Steenkiste, Emiel Hoogeboom, Mehdi S. M. Sajjadi, Thomas Kipf

In this paper, we leverage recent progress in diffusion models to equip 3D scene representation learning models with the ability to render high-fidelity novel views, while retaining benefits such as object-level scene editing to a large degree.

Neural Rendering Representation Learning +2

Sensitivity of Slot-Based Object-Centric Models to their Number of Slots

no code implementations30 May 2023 Roland S. Zimmermann, Sjoerd van Steenkiste, Mehdi S. M. Sajjadi, Thomas Kipf, Klaus Greff

Self-supervised methods for learning object-centric representations have recently been applied successfully to various datasets.

AudioSlots: A slot-centric generative model for audio separation

no code implementations9 May 2023 Pradyumna Reddy, Scott Wisdom, Klaus Greff, John R. Hershey, Thomas Kipf

We discuss the results and limitations of our approach in detail, and further outline potential ways to overcome the limitations and directions for future work.

Speech Separation

Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames

1 code implementation9 Feb 2023 Ondrej Biza, Sjoerd van Steenkiste, Mehdi S. M. Sajjadi, Gamaleldin F. Elsayed, Aravindh Mahendran, Thomas Kipf

Automatically discovering composable abstractions from raw perceptual data is a long-standing challenge in machine learning.

Object Discovery

RUST: Latent Neural Scene Representations from Unposed Imagery

no code implementations CVPR 2023 Mehdi S. M. Sajjadi, Aravindh Mahendran, Thomas Kipf, Etienne Pot, Daniel Duckworth, Mario Lucic, Klaus Greff

Our main insight is that one can train a Pose Encoder that peeks at the target image and learns a latent pose embedding which is used by the decoder for view synthesis.

Novel View Synthesis

Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks

1 code implementation1 Nov 2022 Sadegh Mahdavi, Kevin Swersky, Thomas Kipf, Milad Hashemi, Christos Thrampoulidis, Renjie Liao

In this paper, we study the OOD generalization of neural algorithmic reasoning tasks, where the goal is to learn an algorithm (e. g., sorting, breadth-first search, and depth-first search) from input-output pairs using deep neural networks.

Data Augmentation Out-of-Distribution Generalization

SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models

1 code implementation12 Oct 2022 Ziyi Wu, Nikita Dvornik, Klaus Greff, Thomas Kipf, Animesh Garg

While recent object-centric models can successfully decompose a scene into objects, modeling their dynamics effectively still remains a challenge.

Question Answering Video Prediction +1

Object Scene Representation Transformer

no code implementations14 Jun 2022 Mehdi S. M. Sajjadi, Daniel Duckworth, Aravindh Mahendran, Sjoerd van Steenkiste, Filip Pavetić, Mario Lučić, Leonidas J. Guibas, Klaus Greff, Thomas Kipf

A compositional understanding of the world in terms of objects and their geometry in 3D space is considered a cornerstone of human cognition.

Novel View Synthesis Representation Learning

Binding Actions to Objects in World Models

1 code implementation27 Apr 2022 Ondrej Biza, Robert Platt, Jan-Willem van de Meent, Lawson L. S. Wong, Thomas Kipf

We study the problem of binding actions to objects in object-factored world models using action-attention mechanisms.

Hard Attention

Test-time Adaptation with Slot-Centric Models

1 code implementation21 Mar 2022 Mihir Prabhudesai, Anirudh Goyal, Sujoy Paul, Sjoerd van Steenkiste, Mehdi S. M. Sajjadi, Gaurav Aggarwal, Thomas Kipf, Deepak Pathak, Katerina Fragkiadaki

In our work, we find evidence that these losses are insufficient for the task of scene decomposition, without also considering architectural inductive biases.

Image Classification Image Segmentation +6

Symmetry Group Equivariant Architectures for Physics

no code implementations11 Mar 2022 Alexander Bogatskiy, Sanmay Ganguly, Thomas Kipf, Risi Kondor, David W. Miller, Daniel Murnane, Jan T. Offermann, Mariel Pettee, Phiala Shanahan, Chase Shimmin, Savannah Thais

Physical theories grounded in mathematical symmetries are an essential component of our understanding of a wide range of properties of the universe.

BIG-bench Machine Learning

Factored World Models for Zero-Shot Generalization in Robotic Manipulation

1 code implementation10 Feb 2022 Ondrej Biza, Thomas Kipf, David Klee, Robert Platt, Jan-Willem van de Meent, Lawson L. S. Wong

In this paper, we learn to generalize over robotic pick-and-place tasks using object-factored world models, which combat the combinatorial explosion by ensuring that predictions are equivariant to permutations of objects.

Conditional Object-Centric Learning from Video

2 code implementations ICLR 2022 Thomas Kipf, Gamaleldin F. Elsayed, Aravindh Mahendran, Austin Stone, Sara Sabour, Georg Heigold, Rico Jonschkowski, Alexey Dosovitskiy, Klaus Greff

Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built.

Instance Segmentation Optical Flow Estimation +2

The Impact of Negative Sampling on Contrastive Structured World Models

1 code implementation24 Jul 2021 Ondrej Biza, Elise van der Pol, Thomas Kipf

World models trained by contrastive learning are a compelling alternative to autoencoder-based world models, which learn by reconstructing pixel states.

Contrastive Learning

Learning Object-Centric Video Models by Contrasting Sets

no code implementations20 Nov 2020 Sindy Löwe, Klaus Greff, Rico Jonschkowski, Alexey Dosovitskiy, Thomas Kipf

We address this problem by introducing a global, set-based contrastive loss: instead of contrasting individual slot representations against one another, we aggregate the representations and contrast the joined sets against one another.

Future prediction Self-Supervised Learning

Object-Centric Learning with Slot Attention

8 code implementations NeurIPS 2020 Francesco Locatello, Dirk Weissenborn, Thomas Unterthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, Thomas Kipf

Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features.

Object Discovery Property Prediction

Plannable Approximations to MDP Homomorphisms: Equivariance under Actions

1 code implementation27 Feb 2020 Elise van der Pol, Thomas Kipf, Frans A. Oliehoek, Max Welling

We introduce a contrastive loss function that enforces action equivariance on the learned representations.

Representation Learning

Contrastive Learning of Structured World Models

3 code implementations ICLR 2020 Thomas Kipf, Elise van der Pol, Max Welling

Our experiments demonstrate that C-SWMs can overcome limitations of models based on pixel reconstruction and outperform typical representatives of this model class in highly structured environments, while learning interpretable object-based representations.

Atari Games Contrastive Learning +1

Image-Conditioned Graph Generation for Road Network Extraction

4 code implementations31 Oct 2019 Davide Belli, Thomas Kipf

For this, we introduce the Toulouse Road Network dataset, based on real-world publicly-available data.

Graph Generation Semantic Segmentation

Estimating Cardinalities with Deep Sketches

1 code implementation17 Apr 2019 Andreas Kipf, Dimitri Vorona, Jonas Müller, Thomas Kipf, Bernhard Radke, Viktor Leis, Peter Boncz, Thomas Neumann, Alfons Kemper

We introduce Deep Sketches, which are compact models of databases that allow us to estimate the result sizes of SQL queries.


CompILE: Compositional Imitation Learning and Execution

3 code implementations4 Dec 2018 Thomas Kipf, Yujia Li, Hanjun Dai, Vinicius Zambaldi, Alvaro Sanchez-Gonzalez, Edward Grefenstette, Pushmeet Kohli, Peter Battaglia

We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data.

Continuous Control Imitation Learning

Graph Refinement based Airway Extraction using Mean-Field Networks and Graph Neural Networks

1 code implementation21 Nov 2018 Raghavendra Selvan, Thomas Kipf, Max Welling, Antonio Garcia-Uceda Juarez, Jesper H. Pedersen, Jens Petersen, Marleen de Bruijne

Graph refinement, or the task of obtaining subgraphs of interest from over-complete graphs, can have many varied applications.

Towards Sparse Hierarchical Graph Classifiers

1 code implementation3 Nov 2018 Cătălina Cangea, Petar Veličković, Nikola Jovanović, Thomas Kipf, Pietro Liò

Recent advances in representation learning on graphs, mainly leveraging graph convolutional networks, have brought a substantial improvement on many graph-based benchmark tasks.

General Classification Graph Classification +3

MolGAN: An implicit generative model for small molecular graphs

10 code implementations30 May 2018 Nicola De Cao, Thomas Kipf

Deep generative models for graph-structured data offer a new angle on the problem of chemical synthesis: by optimizing differentiable models that directly generate molecular graphs, it is possible to side-step expensive search procedures in the discrete and vast space of chemical structures.

Graph Matching

Extraction of Airways using Graph Neural Networks

no code implementations12 Apr 2018 Raghavendra Selvan, Thomas Kipf, Max Welling, Jesper H. Pedersen, Jens Petersen, Marleen de Bruijne

We present extraction of tree structures, such as airways, from image data as a graph refinement task.

Hyperspherical Variational Auto-Encoders

9 code implementations3 Apr 2018 Tim R. Davidson, Luca Falorsi, Nicola De Cao, Thomas Kipf, Jakub M. Tomczak

But although the default choice of a Gaussian distribution for both the prior and posterior represents a mathematically convenient distribution often leading to competitive results, we show that this parameterization fails to model data with a latent hyperspherical structure.

Link Prediction

Neural Relational Inference for Interacting Systems

9 code implementations ICML 2018 Thomas Kipf, Ethan Fetaya, Kuan-Chieh Wang, Max Welling, Richard Zemel

Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics.

Cannot find the paper you are looking for? You can Submit a new open access paper.