Search Results for author:
Found 39 papers, 24 papers with code
Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Models
By fine-tuning a pre-trained text-to-image diffusion model with this information, our approach enables fine-grained 3D pose and placement control of individual objects in a scene.
SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code
SceneCraft first models a scene graph as a blueprint, detailing the spatial relationships among assets in the scene.
Learning 3D Particle-based Simulators from RGB-D Videos
Realistic simulation is critical for applications ranging from robotics to animation.
DyST: Towards Dynamic Neural Scene Representations on Real-World Videos
Our Dynamic Scene Transformer (DyST) model leverages recent work in neural scene representation to learn a latent decomposition of monocular real-world videos into scene content, per-view scene dynamics, and camera pose.
Video OWL-ViT: Temporally-consistent open-world localization in video
Our model is end-to-end trainable on video data and enjoys improved temporal consistency compared to tracking-by-detection baselines, while retaining the open-world capabilities of the backbone detector.
One-shot Imitation Learning via Interaction Warping
We propose a new method, Interaction Warping, for learning SE(3) robotic manipulation policies from a single demonstration.
DORSal: Diffusion for Object-centric Representations of Scenes et al
In this paper, we leverage recent progress in diffusion models to equip 3D scene representation learning models with the ability to render high-fidelity novel views, while retaining benefits such as object-level scene editing to a large degree.
Sensitivity of Slot-Based Object-Centric Models to their Number of Slots
Self-supervised methods for learning object-centric representations have recently been applied successfully to various datasets.
AudioSlots: A slot-centric generative model for audio separation
We discuss the results and limitations of our approach in detail, and further outline potential ways to overcome the limitations and directions for future work.
Scaling Vision Transformers to 22 Billion Parameters
The scaling of Transformers has driven breakthrough capabilities for language models.
Ranked #1 on
Zero-Shot Transfer Image Classification
on ObjectNet
Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames
Automatically discovering composable abstractions from raw perceptual data is a long-standing challenge in machine learning.
RUST: Latent Neural Scene Representations from Unposed Imagery
Our main insight is that one can train a Pose Encoder that peeks at the target image and learns a latent pose embedding which is used by the decoder for view synthesis.
Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks
In this paper, we study the OOD generalization of neural algorithmic reasoning tasks, where the goal is to learn an algorithm (e. g., sorting, breadth-first search, and depth-first search) from input-output pairs using deep neural networks.
SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Models
While recent object-centric models can successfully decompose a scene into objects, modeling their dynamics effectively still remains a challenge.
SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos
The visual world can be parsimoniously characterized in terms of distinct entities with sparse interactions.
Simple Open-Vocabulary Object Detection with Vision Transformers
Combining simple architectures with large-scale pre-training has led to massive improvements in image classification.
Ranked #1 on
One-Shot Object Detection
on MS COCO
Binding Actions to Objects in World Models
We study the problem of binding actions to objects in object-factored world models using action-attention mechanisms.
Test-time Adaptation with Slot-Centric Models
In our work, we find evidence that these losses are insufficient for the task of scene decomposition, without also considering architectural inductive biases.
Symmetry Group Equivariant Architectures for Physics
Physical theories grounded in mathematical symmetries are an essential component of our understanding of a wide range of properties of the universe.
Kubric: A scalable dataset generator
Data is the driving force of machine learning, with the amount and quality of training data often being more important for the performance of a system than architecture and training details.
Factored World Models for Zero-Shot Generalization in Robotic Manipulation
In this paper, we learn to generalize over robotic pick-and-place tasks using object-factored world models, which combat the combinatorial explosion by ensuring that predictions are equivariant to permutations of objects.
Conditional Object-Centric Learning from Video
Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built.
The Impact of Negative Sampling on Contrastive Structured World Models
World models trained by contrastive learning are a compelling alternative to autoencoder-based world models, which learn by reconstructing pixel states.
Reasoning-Modulated Representations
Neural networks leverage robust internal representations in order to generalise.
Learning Object-Centric Video Models by Contrasting Sets
We address this problem by introducing a global, set-based contrastive loss: instead of contrasting individual slot representations against one another, we aggregate the representations and contrast the joined sets against one another.
Object-Centric Learning with Slot Attention
Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features.
Plannable Approximations to MDP Homomorphisms: Equivariance under Actions
We introduce a contrastive loss function that enforces action equivariance on the learned representations.
Contrastive Learning of Structured World Models
Our experiments demonstrate that C-SWMs can overcome limitations of models based on pixel reconstruction and outperform typical representatives of this model class in highly structured environments, while learning interpretable object-based representations.
Image-Conditioned Graph Generation for Road Network Extraction
For this, we introduce the Toulouse Road Network dataset, based on real-world publicly-available data.
Ranked #1 on
Graph Generation
on Toulouse Road Network
Estimating Cardinalities with Deep Sketches
We introduce Deep Sketches, which are compact models of databases that allow us to estimate the result sizes of SQL queries.
Databases
CompILE: Compositional Imitation Learning and Execution
We introduce Compositional Imitation Learning and Execution (CompILE): a framework for learning reusable, variable-length segments of hierarchically-structured behavior from demonstration data.
Graph Refinement based Airway Extraction using Mean-Field Networks and Graph Neural Networks
Graph refinement, or the task of obtaining subgraphs of interest from over-complete graphs, can have many varied applications.
Towards Sparse Hierarchical Graph Classifiers
Recent advances in representation learning on graphs, mainly leveraging graph convolutional networks, have brought a substantial improvement on many graph-based benchmark tasks.
Learned Cardinalities: Estimating Correlated Joins with Deep Learning
We describe a new deep learning approach to cardinality estimation.
Databases
MolGAN: An implicit generative model for small molecular graphs
Deep generative models for graph-structured data offer a new angle on the problem of chemical synthesis: by optimizing differentiable models that directly generate molecular graphs, it is possible to side-step expensive search procedures in the discrete and vast space of chemical structures.
Ranked #3 on
Molecular Graph Generation
on InterBioScreen
Extraction of Airways using Graph Neural Networks
We present extraction of tree structures, such as airways, from image data as a graph refinement task.
Hyperspherical Variational Auto-Encoders
But although the default choice of a Gaussian distribution for both the prior and posterior represents a mathematically convenient distribution often leading to competitive results, we show that this parameterization fails to model data with a latent hyperspherical structure.
Ranked #6 on
Link Prediction
on Cora
Neural Relational Inference for Interacting Systems
Interacting systems are prevalent in nature, from dynamical systems in physics to complex societal dynamics.
