Search Results for author: Christian Rupprecht

Found 69 papers, 25 papers with code

Recent Trends in 3D Reconstruction of General Non-Rigid Scenes

no code implementations22 Mar 2024 Raza Yunus, Jan Eric Lenssen, Michael Niemeyer, Yiyi Liao, Christian Rupprecht, Christian Theobalt, Gerard Pons-Moll, Jia-Bin Huang, Vladislav Golyanik, Eddy Ilg

Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision.

3D Reconstruction Navigate

DragAPart: Learning a Part-Level Motion Prior for Articulated Objects

no code implementations22 Mar 2024 Ruining Li, Chuanxia Zheng, Christian Rupprecht, Andrea Vedaldi

We introduce DragAPart, a method that, given an image and a set of drags as input, can generate a new image of the same object in a new state, compatible with the action of the drags.

IM-3D: Iterative Multiview Diffusion and Reconstruction for High-Quality 3D Generation

no code implementations13 Feb 2024 Luke Melas-Kyriazi, Iro Laina, Christian Rupprecht, Natalia Neverova, Andrea Vedaldi, Oran Gafni, Filippos Kokkinos

A mitigation is to fine-tune the 2D generator to be multi-view aware, which can help distillation or can be combined with reconstruction networks to output 3D objects directly.

3D Reconstruction Text to 3D

Learning the 3D Fauna of the Web

no code implementations4 Jan 2024 Zizhang Li, Dor Litvak, Ruining Li, Yunzhi Zhang, Tomas Jakab, Christian Rupprecht, Shangzhe Wu, Andrea Vedaldi, Jiajun Wu

We show that prior category-specific attempts fail to generalize to rare species with limited training images.

Splatter Image: Ultra-Fast Single-View 3D Reconstruction

1 code implementation20 Dec 2023 Stanislaw Szymanowicz, Christian Rupprecht, Andrea Vedaldi

We introduce the Splatter Image, an ultra-fast approach for monocular 3D object reconstruction which operates at 38 FPS.

3D Object Reconstruction 3D Reconstruction +2

Scene-Conditional 3D Object Stylization and Composition

no code implementations19 Dec 2023 Jinghao Zhou, Tomas Jakab, Philip Torr, Christian Rupprecht

Recently, 3D generative models have made impressive progress, enabling the generation of almost arbitrary 3D assets from text or image inputs.

Object

Visual Geometry Grounded Deep Structure From Motion

no code implementations7 Dec 2023 Jianyuan Wang, Nikita Karaev, Christian Rupprecht, David Novotny

Finally, we optimise the cameras and triangulate 3D points via a differentiable bundle adjustment layer.

Point Tracking

HowToCaption: Prompting LLMs to Transform Video Annotations at Scale

1 code implementation7 Oct 2023 Nina Shvetsova, Anna Kukleva, Xudong Hong, Christian Rupprecht, Bernt Schiele, Hilde Kuehne

Specifically, we prompt an LLM to create plausible video descriptions based on ASR narrations of the video for a large-scale instructional video dataset.

Automatic Speech Recognition Sentence +3

CoTracker: It is Better to Track Together

1 code implementation14 Jul 2023 Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, Christian Rupprecht

We introduce CoTracker, a transformer-based model that tracks dense points in a frame jointly across a video sequence.

motion prediction Object Tracking +2

PoseDiffusion: Solving Pose Estimation via Diffusion-aided Bundle Adjustment

no code implementations ICCV 2023 Jianyuan Wang, Christian Rupprecht, David Novotny

Camera pose estimation is a long-standing computer vision problem that to date often relies on classical methods, such as handcrafted keypoint matching, RANSAC and bundle adjustment.

Pose Estimation

Diffusion Models for Zero-Shot Open-Vocabulary Segmentation

no code implementations15 Jun 2023 Laurynas Karazija, Iro Laina, Andrea Vedaldi, Christian Rupprecht

This provides a distribution of appearances for a given text circumventing the ambiguity problem.

DynamicStereo: Consistent Dynamic Depth from Stereo Videos

1 code implementation CVPR 2023 Nikita Karaev, Ignacio Rocco, Benjamin Graham, Natalia Neverova, Andrea Vedaldi, Christian Rupprecht

The network learns to pool information from neighboring frames to improve the temporal consistency of its predictions.

Farm3D: Learning Articulated 3D Animals by Distilling 2D Diffusion

no code implementations20 Apr 2023 Tomas Jakab, Ruining Li, Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi

We propose a framework that uses an image generator, such as Stable Diffusion, to generate synthetic training data that are sufficiently clean and do not require further manual curation, enabling the learning of such a reconstruction network from scratch.

Monocular Reconstruction Object

What does CLIP know about a red circle? Visual prompt engineering for VLMs

no code implementations ICCV 2023 Aleksandar Shtedritski, Christian Rupprecht, Andrea Vedaldi

Large-scale Vision-Language Models, such as CLIP, learn powerful image-text representations that have found numerous applications, from zero-shot classification to text-to-image generation.

Prompt Engineering Text-to-Image Generation +1

Continual Detection Transformer for Incremental Object Detection

no code implementations CVPR 2023 Yaoyao Liu, Bernt Schiele, Andrea Vedaldi, Christian Rupprecht

Incremental object detection (IOD) aims to train an object detector in phases, each with annotations for new object categories.

Knowledge Distillation Object +2

Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Data

1 code implementation23 Mar 2023 Anna Kukleva, Moritz Böhle, Bernt Schiele, Hilde Kuehne, Christian Rupprecht

Such a schedule results in a constant `task switching' between an emphasis on instance discrimination and group-wise discrimination and thereby ensures that the model learns both group-wise features, as well as instance-specific details.

Self-Supervised Learning

RealFusion: 360° Reconstruction of Any Object from a Single Image

3 code implementations21 Feb 2023 Luke Melas-Kyriazi, Christian Rupprecht, Iro Laina, Andrea Vedaldi

We consider the problem of reconstructing a full 360{\deg} photographic model of an object from a single image of it.

3D Reconstruction Object

$PC^2$: Projection-Conditioned Point Cloud Diffusion for Single-Image 3D Reconstruction

2 code implementations21 Feb 2023 Luke Melas-Kyriazi, Christian Rupprecht, Andrea Vedaldi

Reconstructing the 3D shape of an object from a single RGB image is a long-standing and highly challenging problem in computer vision.

3D Reconstruction Denoising

Behind the Scenes: Density Fields for Single View Reconstruction

1 code implementation CVPR 2023 Felix Wimbauer, Nan Yang, Christian Rupprecht, Daniel Cremers

By directly sampling color from the available views instead of storing color in the density field, our scene representation becomes significantly less complex compared to NeRFs, and a neural network can predict it in a single forward pass.

Depth Estimation Depth Prediction +1

MagicPony: Learning Articulated 3D Animals in the Wild

no code implementations CVPR 2023 Shangzhe Wu, Ruining Li, Tomas Jakab, Christian Rupprecht, Andrea Vedaldi

We consider the problem of predicting the 3D shape, articulation, viewpoint, texture, and lighting of an articulated animal like a horse given a single test image as input.

Viewpoint Estimation

Nondestructive thermographic detection of internal defects using pixel-pattern based laser excitation and photothermal super resolution reconstruction

no code implementations8 Nov 2022 Julien Lecompagnon, Philipp Daniel Hirsch, Christian Rupprecht, Mathias Ziegler

In this work, we present a novel approach to photothermal super resolution based thermographic resolution of internal defects using two-dimensional pixel pattern-based active photothermal laser heating in conjunction with subsequent numerical reconstruction to achieve a high-resolution reconstruction of internal defect structures.

Super-Resolution

VTC: Improving Video-Text Retrieval with User Comments

1 code implementation19 Oct 2022 Laura Hanu, James Thewlis, Yuki M. Asano, Christian Rupprecht

In this paper, we a) introduce a new dataset of videos, titles and comments; b) present an attention-based mechanism that allows the model to learn from sometimes irrelevant data such as comments; c) show that by using comments, our method is able to learn better, more contextualised, representations for image, video and audio representations.

Representation Learning Retrieval +3

Thermographic detection of internal defects using 2D photothermal super resolution reconstruction with sequential laser heating

no code implementations1 Mar 2022 Julien Lecompagnon, Samim Ahmadi, Philipp Hirsch, Christian Rupprecht, Mathias Ziegler

Based on a combination of the application of special sampling strategies and a subsequent numerical optimization step in post-processing, thermographic super resolution has already proven to be superior to standard thermographic methods in the detection of one-dimensional defect/inhomogeneity structures.

Super-Resolution

De-rendering 3D Objects in the Wild

1 code implementation CVPR 2022 Felix Wimbauer, Shangzhe Wu, Christian Rupprecht

With increasing focus on augmented and virtual reality applications (XR) comes the demand for algorithms that can lift objects from images and videos into representations that are suitable for a wide variety of related 3D tasks.

ClevrTex: A Texture-Rich Benchmark for Unsupervised Multi-Object Segmentation

1 code implementation19 Nov 2021 Laurynas Karazija, Iro Laina, Christian Rupprecht

We benchmark a large set of recent unsupervised multi-object segmentation models on ClevrTex and find all state-of-the-art approaches fail to learn good representations in the textured setting, despite impressive performance on simpler data.

Segmentation Semantic Segmentation +1

The Curious Layperson: Fine-Grained Image Recognition without Expert Labels

1 code implementation5 Nov 2021 Subhabrata Choudhury, Iro Laina, Christian Rupprecht, Andrea Vedaldi

We then train a fine-grained textual similarity model that matches image descriptions with documents on a sentence-level basis.

Cross-Modal Retrieval Fine-Grained Image Recognition +2

Learning Context-Adapted Video-Text Retrieval by Attending to User Comments

no code implementations29 Sep 2021 Laura Hanu, Yuki M Asano, James Thewlis, Christian Rupprecht

Learning strong representations for multi-modal retrieval is an important problem for many applications, such as recommendation and search.

Retrieval Text Retrieval +1

PASS: An ImageNet replacement for self-supervised pretraining without humans

1 code implementation NeurIPS Workshop ImageNet_PPF 2021 Yuki M. Asano, Christian Rupprecht, Andrew Zisserman, Andrea Vedaldi

On the other hand, state-of-the-art pretraining is nowadays obtained with unsupervised methods, meaning that labelled datasets such as ImageNet may not be necessary, or perhaps not even optimal, for model pretraining.

Benchmarking Ethics +2

DOVE: Learning Deformable 3D Objects by Watching Videos

no code implementations22 Jul 2021 Shangzhe Wu, Tomas Jakab, Christian Rupprecht, Andrea Vedaldi

In this paper, we present DOVE, a method that learns textured 3D models of deformable object categories from monocular videos available online, without keypoint, viewpoint or template shape supervision.

Semantic Image Manipulation Using Scene Graphs

1 code implementation CVPR 2020 Helisa Dhamo, Azade Farshad, Iro Laina, Nassir Navab, Gregory D. Hager, Federico Tombari, Christian Rupprecht

In our work, we address the novel problem of image manipulation from scene graphs, in which a user can edit images by merely applying changes in the nodes or edges of a semantic graph that is generated from the image.

Image Inpainting Image Manipulation +1

Improving Feature Attribution through Input-specific Network Pruning

no code implementations25 Nov 2019 Ashkan Khakzar, Soroosh Baselizadeh, Saurabh Khanduja, Christian Rupprecht, Seong Tae Kim, Nassir Navab

Attributing the output of a neural network to the contribution of given input elements is a way of shedding light on the black-box nature of neural networks.

Network Pruning

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

1 code implementation CVPR 2020 Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi

We propose a method to learn 3D deformable object categories from raw single-view images, without external supervision.

Object

Self-labelling via simultaneous clustering and representation learning

4 code implementations ICLR 2020 Yuki Markus Asano, Christian Rupprecht, Andrea Vedaldi

Combining clustering and representation learning is one of the most promising approaches for unsupervised learning of deep neural networks.

Clustering Contrastive Learning +4

Traffic4cast-Traffic Map Movie Forecasting -- Team MIE-Lab

1 code implementation27 Oct 2019 Henry Martin, Ye Hong, Dominik Bucher, Christian Rupprecht, René Buffat

The goal of the IARAI competition traffic4cast was to predict the city-wide traffic status within a 15-minute time window, based on information from the previous hour.

Photo-Geometric Autoencoding to Learn 3D Objects from Unlabelled Images

no code implementations4 Jun 2019 Shangzhe Wu, Christian Rupprecht, Andrea Vedaldi

Specifically, given a single image of the object seen from an arbitrary viewpoint, our model predicts a symmetric canonical view, the corresponding 3D shape and a viewpoint transformation, and trains with the goal of reconstructing the input view, resembling an auto-encoder.

Finding and Visualizing Weaknesses of Deep Reinforcement Learning Agents

no code implementations ICLR 2020 Christian Rupprecht, Cyril Ibrahim, Christopher J. Pal

Further, critical states in which a very high or a very low reward can be achieved are often interesting to understand the situational awareness of the system as they can correspond to risky states.

Autonomous Driving Decision Making +2

Explaining the Ambiguity of Object Detection and 6D Pose From Visual Data

no code implementations ICCV 2019 Fabian Manhardt, Diego Martin Arroyo, Christian Rupprecht, Benjamin Busam, Tolga Birdal, Nassir Navab, Federico Tombari

For each object instance we predict multiple pose and class outcomes to estimate the specific pose distribution generated by symmetries and repetitive textures.

3D Object Detection Object +3

Dealing with Ambiguity in Robotic Grasping via Multiple Predictions

no code implementations2 Nov 2018 Ghazal Ghazaei, Iro Laina, Christian Rupprecht, Federico Tombari, Nassir Navab, Kianoush Nazarpour

Further, we reformulate the problem of robotic grasping by replacing conventional grasp rectangles with grasp belief maps, which hold more precise location information than a rectangle and account for the uncertainty inherent to the task.

Robotic Grasping

Understanding Regularization to Visualize Convolutional Neural Networks

no code implementations20 Apr 2018 Maximilian Baust, Florian Ludwig, Christian Rupprecht, Matthias Kohl, Stefan Braunewell

Variational methods for revealing visual concepts learned by convolutional neural networks have gained significant attention during the last years.

Guide Me: Interacting with Deep Networks

no code implementations CVPR 2018 Christian Rupprecht, Iro Laina, Nassir Navab, Gregory D. Hager, Federico Tombari

Interaction and collaboration between humans and intelligent machines has become increasingly important as machine learning methods move into real-world applications that involve end users.

Image Captioning Image Generation

Learning to Imagine Manipulation Goals for Robot Task Planning

no code implementations8 Nov 2017 Chris Paxton, Kapil Katyal, Christian Rupprecht, Raman Arora, Gregory D. Hager

Ideally, we would combine the ability of machine learning to leverage big data for learning the semantics of a task, while using techniques from task planning to reliably generalize to new environment.

Robot Task Planning

Analyzing and Exploiting NARX Recurrent Neural Networks for Long-Term Dependencies

no code implementations ICLR 2018 Robert DiPietro, Christian Rupprecht, Nassir Navab, Gregory D. Hager

Recurrent neural networks (RNNs) have achieved state-of-the-art performance on many diverse tasks, from machine translation to surgical activity recognition, yet training RNNs to capture long-term dependencies remains difficult.

Activity Recognition Machine Translation +1

Deep Active Contours

no code implementations18 Jul 2016 Christian Rupprecht, Elizabeth Huaroc, Maximilian Baust, Nassir Navab

We propose a method for interactive boundary extraction which combines a deep, patch-based representation with an active contour framework.

Interactive Segmentation

A Taxonomy and Library for Visualizing Learned Features in Convolutional Neural Networks

no code implementations24 Jun 2016 Felix Grün, Christian Rupprecht, Nassir Navab, Federico Tombari

Over the last decade, Convolutional Neural Networks (CNN) saw a tremendous surge in performance.

Semi-Automatic Segmentation of Autosomal Dominant Polycystic Kidneys using Random Forests

no code implementations23 Oct 2015 Kanishka Sharma, Loic Peter, Christian Rupprecht, Anna Caroli, Lichao Wang, Andrea Remuzzi, Maximilian Baust, Nassir Navab

This paper presents a method for 3D segmentation of kidneys from patients with autosomal dominant polycystic kidney disease (ADPKD) and severe renal insufficiency, using computed tomography (CT) data.

Computed Tomography (CT) Segmentation

Robust Optimization for Deep Regression

1 code implementation ICCV 2015 Vasileios Belagiannis, Christian Rupprecht, Gustavo Carneiro, Nassir Navab

Convolutional Neural Networks (ConvNets) have successfully contributed to improve the accuracy of regression-based methods for computer vision tasks such as human pose estimation, landmark localization, and object detection.

Age Estimation object-detection +3

Cannot find the paper you are looking for? You can Submit a new open access paper.