Search Results for author: Bernt Schiele

Found 153 papers, 55 papers with code

Towards Automated Testing and Robustification by Semantic Adversarial Data Generation

no code implementations ECCV 2020 Rakshith Shetty, Mario Fritz, Bernt Schiele

Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector.

Data Augmentation

Revisiting Consistency Regularization for Semi-Supervised Learning

no code implementations10 Dec 2021 Yue Fan, Anna Kukleva, Bernt Schiele

Generally, the aim is to train a model that is invariant to various data augmentations.

CoSSL: Co-Learning of Representation and Classifier for Imbalanced Semi-Supervised Learning

no code implementations8 Dec 2021 Yue Fan, Dengxin Dai, Bernt Schiele

In this paper, we propose a novel co-learning framework (CoSSL) with decoupled representation learning and classifier learning for imbalanced SSL.

Representation Learning

RMM: Reinforced Memory Management for Class-Incremental Learning

no code implementations NeurIPS 2021 Yaoyao Liu, Bernt Schiele, Qianru Sun

Class-Incremental Learning (CIL) [38] trains classifiers under a strict memory budget: in each incremental phase, learning is done for new data, most of which is abandoned to free space for the next phase.

class-incremental learning Incremental Learning

Keypoint Message Passing for Video-based Person Re-Identification

no code implementations16 Nov 2021 Di Chen, Andreas Doering, Shanshan Zhang, Jian Yang, Juergen Gall, Bernt Schiele

Video-based person re-identification (re-ID) is an important technique in visual surveillance systems which aims to match video snippets of people captured by different cameras.

Graph Convolutional Network Representation Learning +1

Optimising for Interpretability: Convolutional Dynamic Alignment Networks

1 code implementation27 Sep 2021 Moritz Böhle, Mario Fritz, Bernt Schiele

As a result, CoDA Nets model the classification prediction through a series of input-dependent linear transformations, allowing for linear decomposition of the output into individual input contributions.

Generalized and Incremental Few-Shot Learning by Explicit Learning and Calibration without Forgetting

no code implementations ICCV 2021 Anna Kukleva, Hilde Kuehne, Bernt Schiele

Both generalized and incremental few-shot learning have to deal with three major challenges: learning novel classes from only few samples per class, preventing catastrophic forgetting of base classes, and classifier calibration across novel and base classes.

Few-Shot Learning

Learning Spatially-Variant MAP Models for Non-Blind Image Deblurring

no code implementations CVPR 2021 Jiangxin Dong, Stefan Roth, Bernt Schiele

The classical maximum a-posteriori (MAP) framework for non-blind image deblurring requires defining suitable data and regularization terms, whose interplay yields the desired clear image through optimization.

Blind Image Deblurring Image Deblurring

Random and Adversarial Bit Error Robustness: Energy-Efficient and Secure DNN Accelerators

no code implementations16 Apr 2021 David Stutz, Nandhini Chandramoorthy, Matthias Hein, Bernt Schiele

Low-voltage operation of DNN accelerators allows to further reduce energy consumption significantly, however, causes bit-level failures in the memory storing the quantized DNN weights.

Quantization

Relating Adversarially Robust Generalization to Flat Minima

no code implementations ICCV 2021 David Stutz, Matthias Hein, Bernt Schiele

To this end, we propose average- and worst-case metrics to measure flatness in the robust loss landscape and show a correlation between good robust generalization and flatness.

Adversarial Robustness

Convolutional Dynamic Alignment Networks for Interpretable Classifications

1 code implementation CVPR 2021 Moritz Böhle, Mario Fritz, Bernt Schiele

Given the alignment of the DAUs, the resulting contribution maps align with discriminative input patterns.

Deep Wiener Deconvolution: Wiener Meets Deep Learning for Image Deblurring

1 code implementation NeurIPS 2020 Jiangxin Dong, Stefan Roth, Bernt Schiele

We present a simple and effective approach for non-blind image deblurring, combining classical techniques and deep learning.

Blind Image Deblurring Image Deblurring

Adjoint Rigid Transform Network: Task-conditioned Alignment of 3D Shapes

no code implementations1 Feb 2021 Keyang Zhou, Bharat Lal Bhatnagar, Bernt Schiele, Gerard Pons-Moll

The remarkable result is that with only self-supervision, ART facilitates learning a unique canonical orientation for both rigid and nonrigid shapes, which leads to a notable boost in performance of aforementioned tasks.

You Only Need Adversarial Supervision for Semantic Image Synthesis

1 code implementation ICLR 2021 Vadim Sushko, Edgar Schönfeld, Dan Zhang, Juergen Gall, Bernt Schiele, Anna Khoreva

By providing stronger supervision to the discriminator as well as to the generator through spatially- and semantically-aware discriminator feedback, we are able to synthesize images of higher fidelity with better alignment to their input label maps, making the use of the perceptual loss superfluous.

Image-to-Image Translation Semantic Segmentation

PoseTrackReID: Dataset Description

no code implementations12 Nov 2020 Andreas Doering, Di Chen, Shanshan Zhang, Bernt Schiele, Juergen Gall

For that reason, we present PoseTrackReID, a large-scale dataset for multi-person pose tracking and video-based person re-ID.

Pose Tracking Video-Based Person Re-Identification

Diverse and Relevant Visual Storytelling with Scene Graph Embeddings

no code implementations CONLL 2020 Xudong Hong, Rakshith Shetty, Asad Sayeed, Khushboo Mehra, Vera Demberg, Bernt Schiele

A problem in automatically generated stories for image sequences is that they use overly generic vocabulary and phrase structure and fail to match the distributional characteristics of human-generated text.

Story Generation Visual Storytelling

Adaptive Aggregation Networks for Class-Incremental Learning

2 code implementations CVPR 2021 Yaoyao Liu, Bernt Schiele, Qianru Sun

Class-Incremental Learning (CIL) aims to learn a classification model with the number of classes increasing phase-by-phase.

class-incremental learning Incremental Learning

Haar Wavelet based Block Autoregressive Flows for Trajectories

no code implementations21 Sep 2020 Apratim Bhattacharyya, Christoph-Nikolas Straehle, Mario Fritz, Bernt Schiele

This yields an exact inference method that models trajectories at different spatio-temporal resolutions in a hierarchical manner.

Synthetic Convolutional Features for Improved Semantic Segmentation

no code implementations18 Sep 2020 Yang He, Bernt Schiele, Mario Fritz

Recently, learning-based image synthesis has enabled to generate high-resolution images, either applying popular adversarial training or a powerful perceptual loss.

Image Generation Semantic Segmentation

Long-Term Anticipation of Activities with Cycle Consistency

no code implementations2 Sep 2020 Yazan Abu Farha, Qiuhong Ke, Bernt Schiele, Juergen Gall

With the success of deep learning methods in analyzing activities in videos, more attention has recently been focused towards anticipating future activities.

Attribute Prototype Network for Zero-Shot Learning

no code implementations NeurIPS 2020 Wenjia Xu, Yongqin Xian, Jiuniu Wang, Bernt Schiele, Zeynep Akata

As an additional benefit, our model points to the visual evidence of the attributes in an image, e. g. for the CUB dataset, confirming the improved attribute localization ability of our image representation.

Representation Learning Zero-Shot Learning

Kinematic 3D Object Detection in Monocular Video

2 code implementations ECCV 2020 Garrick Brazil, Gerard Pons-Moll, Xiaoming Liu, Bernt Schiele

In this work, we propose a novel method for monocular video-based 3D object detection which carefully leverages kinematic motion to improve precision of 3D localization.

Ranked #8 on Monocular 3D Object Detection on KITTI Cars Moderate (using extra training data)

Monocular 3D Object Detection Vehicle Pose Estimation

Bit Error Robustness for Energy-Efficient DNN Accelerators

1 code implementation24 Jun 2020 David Stutz, Nandhini Chandramoorthy, Matthias Hein, Bernt Schiele

Low-voltage operation of DNN accelerators allows to further reduce energy consumption significantly, however, causes bit-level failures in the memory storing the quantized DNN weights.

Quantization

Adversarial Training against Location-Optimized Adversarial Patches

1 code implementation5 May 2020 Sukrut Rao, David Stutz, Bernt Schiele

Then, we apply adversarial training on these location-optimized adversarial patches and demonstrate significantly improved robustness on CIFAR10 and GTSRB.

Normalizing Flows with Multi-Scale Autoregressive Priors

1 code implementation CVPR 2020 Shweta Mahajan, Apratim Bhattacharyya, Mario Fritz, Bernt Schiele, Stefan Roth

Flow-based generative models are an important class of exact inference models that admit efficient inference and sampling for image synthesis.

Density Estimation Image Generation

Long-Tailed Recognition Using Class-Balanced Experts

1 code implementation7 Apr 2020 Saurabh Sharma, Ning Yu, Mario Fritz, Bernt Schiele

Deep learning enables impressive performance in image recognition using large-scale artificially-balanced datasets.

A U-Net Based Discriminator for Generative Adversarial Networks

3 code implementations28 Feb 2020 Edgar Schönfeld, Bernt Schiele, Anna Khoreva

The novel discriminator improves over the state of the art in terms of the standard distribution and image quality metrics, enabling the generator to synthesize images with varying structure, appearance and levels of detail, maintaining global and local realism.

Conditional Image Generation Data Augmentation

Analyzing the Dependency of ConvNets on Spatial Information

no code implementations5 Feb 2020 Yue Fan, Yongqin Xian, Max Maria Losch, Bernt Schiele

In this paper, we are pushing the envelope and aim to further investigate the reliance on spatial information.

Image Classification Object Recognition

Confidence-Calibrated Adversarial Training: Generalizing to Unseen Attacks

2 code implementations ICML 2020 David Stutz, Matthias Hein, Bernt Schiele

Our confidence-calibrated adversarial training (CCAT) tackles this problem by biasing the model towards low confidence predictions on adversarial examples.

Meta-Transfer Learning through Hard Tasks

1 code implementation7 Oct 2019 Qianru Sun, Yaoyao Liu, Zhaozheng Chen, Tat-Seng Chua, Bernt Schiele

In this paper, we propose a novel approach called meta-transfer learning (MTL) which learns to transfer the weights of a deep NN for few-shot learning tasks.

Few-Shot Learning Transfer Learning

"Best-of-Many-Samples" Distribution Matching

1 code implementation27 Sep 2019 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.

``"Best-of-Many-Samples" Distribution Matching

no code implementations25 Sep 2019 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.

Conditional Flow Variational Autoencoders for Structured Sequence Prediction

no code implementations24 Aug 2019 Apratim Bhattacharyya, Michael Hanselmann, Mario Fritz, Bernt Schiele, Christoph-Nikolas Straehle

Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world.

Trajectory Prediction

Interpretability Beyond Classification Output: Semantic Bottleneck Networks

no code implementations25 Jul 2019 Max Losch, Mario Fritz, Bernt Schiele

Additionally we show how the activations of the SB-Layer can be used for both the interpretation of failure cases of the network as well as for confidence prediction of the resulting output.

Dimensionality Reduction General Classification +1

Learning to Self-Train for Semi-Supervised Few-Shot Classification

1 code implementation NeurIPS 2019 Xinzhe Li, Qianru Sun, Yaoyao Liu, Shibao Zheng, Qin Zhou, Tat-Seng Chua, Bernt Schiele

On each task, we train a few-shot model to predict pseudo labels for unlabeled data, and then iterate the self-training steps on labeled and pseudo-labeled data with each step followed by fine-tuning.

General Classification Meta-Learning

A Novel BiLevel Paradigm for Image-to-Image Translation

no code implementations18 Apr 2019 Liqian Ma, Qianru Sun, Bernt Schiele, Luc van Gool

Image-to-image (I2I) translation is a pixel-level mapping that requires a large number of paired training data and often suffers from the problems of high diversity and strong category bias in image scenes.

Image-to-Image Translation Translation

An Ensemble of Epoch-wise Empirical Bayes for Few-shot Learning

1 code implementation ECCV 2020 Yaoyao Liu, Bernt Schiele, Qianru Sun

"Empirical" means that the hyperparameters, e. g., used for learning and ensembling the epoch-wise models, are generated by hyperprior learners conditional on task-specific data.

Few-Shot Learning

f-VAEGAN-D2: A Feature Generating Framework for Any-Shot Learning

no code implementations CVPR 2019 Yongqin Xian, Saurabh Sharma, Bernt Schiele, Zeynep Akata

When labeled training data is scarce, a promising data augmentation approach is to generate visual features of unknown classes using their attributes.

Data Augmentation Few-Shot Learning +1

Learning Decision Trees Recurrently Through Communication

no code implementations CVPR 2021 Stephan Alaniz, Diego Marcos, Bernt Schiele, Zeynep Akata

Integrated interpretability without sacrificing the prediction accuracy of decision making algorithms has the potential of greatly improving their value to the user.

Decision Making Image Classification

Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation

no code implementations17 Dec 2018 Rakshith Shetty, Bernt Schiele, Mario Fritz

We propose a method to quantify the sensitivity of black-box vision models to visual context by editing images to remove selected objects and measuring the response of the target models.

Data Augmentation General Classification +3

Knockoff Nets: Stealing Functionality of Black-Box Models

1 code implementation CVPR 2019 Tribhuvanesh Orekondy, Bernt Schiele, Mario Fritz

We formulate model functionality stealing as a two-step approach: (i) querying a set of input images to the blackbox model to obtain predictions; and (ii) training a "knockoff" with queried image-prediction pairs.

Meta-Transfer Learning for Few-Shot Learning

2 code implementations CVPR 2019 Qianru Sun, Yaoyao Liu, Tat-Seng Chua, Bernt Schiele

In this paper we propose a novel few-shot learning method called meta-transfer learning (MTL) which learns to adapt a deep NN for few shot learning tasks.

Few-Shot Image Classification Transfer Learning

Disentangling Adversarial Robustness and Generalization

2 code implementations CVPR 2019 David Stutz, Matthias Hein, Bernt Schiele

A recent hypothesis even states that both robust and accurate models are impossible, i. e., adversarial robustness and generalization are conflicting goals.

Adversarial Robustness

Parameter-Free Spatial Attention Network for Person Re-Identification

3 code implementations29 Nov 2018 Haoran Wang, Yue Fan, Zexin Wang, Licheng Jiao, Bernt Schiele

We propose a novel architecture for Person Re-Identification, based on a novel parameter-free spatial attention layer introducing spatial relations among the feature map activations back to the model.

Person Re-Identification

Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods

1 code implementation ICLR 2019 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

For autonomous agents to successfully operate in the real world, the ability to anticipate future scene states is a key competence.

Bayesian Inference

Diverse Conditional Image Generation by Stochastic Regression with Latent Drop-Out Codes

1 code implementation ECCV 2018 Yang He, Bernt Schiele, Mario Fritz

Recent advances in Deep Learning and probabilistic modeling have led to strong improvements in generative models for images.

Conditional Image Generation

Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective

1 code implementation20 Jun 2018 Apratim Bhattacharyya, Bernt Schiele, Mario Fritz

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.

Bayesian Prediction of Future Street Scenes through Importance Sampling based Optimization

no code implementations18 Jun 2018 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.

Future prediction Semantic Segmentation

Adversarial Scene Editing: Automatic Object Removal from Weak Supervision

no code implementations NeurIPS 2018 Rakshith Shetty, Mario Fritz, Bernt Schiele

While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets.

Image Manipulation

Accurate and Diverse Sampling of Sequences Based on a “Best of Many” Sample Objective

no code implementations CVPR 2018 Apratim Bhattacharyya, Bernt Schiele, Mario Fritz

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.

Occluded Pedestrian Detection Through Guided Attention in CNNs

no code implementations CVPR 2018 Shanshan Zhang, Jian Yang, Bernt Schiele

In this paper, we aim to propose a simple and compact method based on the FasterRCNN architecture for occluded pedestrian detection.

Pedestrian Detection

Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning

no code implementations15 May 2018 Tribhuvanesh Orekondy, Seong Joon Oh, Yang Zhang, Bernt Schiele, Mario Fritz

At the core of FL is a network of anonymous user devices sharing training information (model parameter updates) computed locally on personal data.

Data Augmentation Federated Learning +1

Learning to Refine Human Pose Estimation

no code implementations21 Apr 2018 Mihai Fieraru, Anna Khoreva, Leonid Pishchulin, Bernt Schiele

Multi-person pose estimation in images and videos is an important yet challenging task with many applications.

Data Augmentation Keypoint Detection +3

A Hybrid Model for Identity Obfuscation by Face Replacement

no code implementations ECCV 2018 Qianru Sun, Ayush Tewari, Weipeng Xu, Mario Fritz, Christian Theobalt, Bernt Schiele

As more and more personal photos are shared and tagged in social media, avoiding privacy risks such as unintended recognition becomes increasingly challenging.

Face Generation

Video Object Segmentation with Language Referring Expressions

1 code implementation21 Mar 2018 Anna Khoreva, Anna Rohrbach, Bernt Schiele

We show that our language-supervised approach performs on par with the methods which have access to a pixel-level mask of the target object on DAVIS'16 and is competitive to methods using scribbles on the challenging DAVIS'17 dataset.

Referring Expression Segmentation Semantic Segmentation +2

Disentangled Person Image Generation

1 code implementation CVPR 2018 Liqian Ma, Qianru Sun, Stamatios Georgoulis, Luc van Gool, Bernt Schiele, Mario Fritz

Generating novel, yet realistic, images of persons is a challenging task due to the complex interplay between the different image factors, such as the foreground, background and pose information.

Gesture-to-Gesture Translation Person Re-Identification +1

Feature Generating Networks for Zero-Shot Learning

4 code implementations CVPR 2018 Yongqin Xian, Tobias Lorenz, Bernt Schiele, Zeynep Akata

Suffering from the extreme training data imbalance between seen and unseen classes, most of existing state-of-the-art approaches fail to achieve satisfactory results for the challenging generalized zero-shot learning task.

Generalized Zero-Shot Learning

Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty

no code implementations CVPR 2018 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

Our experimental results show that it is indeed possible to predict people trajectories at the desired time horizons and that our uncertainty estimates are informative of the prediction error.

Autonomous Driving Trajectory Prediction

Natural and Effective Obfuscation by Head Inpainting

no code implementations CVPR 2018 Qianru Sun, Liqian Ma, Seong Joon Oh, Luc van Gool, Bernt Schiele, Mario Fritz

As more and more personal photos are shared online, being able to obfuscate identities in such photos is becoming a necessity for privacy protection.

$A^{4}NT$: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation

no code implementations6 Nov 2017 Rakshith Shetty, Bernt Schiele, Mario Fritz

In this paper, we propose an automatic method, called Adversarial Author Attribute Anonymity Neural Translation ($A^4NT$), to combat such text-based adversaries.

Machine Translation Translation

Towards Reverse-Engineering Black-Box Neural Networks

2 code implementations ICLR 2018 Seong Joon Oh, Max Augustin, Bernt Schiele, Mario Fritz

On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -- we show that the revealed internal information helps generate more effective adversarial examples against the black box model.

PoseTrack: A Benchmark for Human Pose Estimation and Tracking

2 code implementations CVPR 2018 Mykhaylo Andriluka, Umar Iqbal, Eldar Insafutdinov, Leonid Pishchulin, Anton Milan, Juergen Gall, Bernt Schiele

In this work, we aim to further advance the state of the art by establishing "PoseTrack", a new large-scale benchmark for video-based human pose estimation and articulated tracking, and bringing together the community of researchers working on visual human analysis.

Activity Recognition Multi-Person Pose Estimation +1

Person Recognition in Personal Photo Collections

no code implementations9 Oct 2017 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

Person recognition in social media photos sets new challenges for computer vision, including non-cooperative subjects (e. g. backward viewpoints, unusual poses) and great changes in appearance.

Face Recognition Person Recognition

Learning Dilation Factors for Semantic Segmentation of Street Scenes

1 code implementation6 Sep 2017 Yang He, Margret Keuper, Bernt Schiele, Mario Fritz

In this paper, we present an approach for learning dilation parameters adaptively per channel, consistently improving semantic segmentation results on street-scene datasets like Cityscapes and Camvid.

Semantic Segmentation

Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly

8 code implementations3 Jul 2017 Yongqin Xian, Christoph H. Lampert, Bernt Schiele, Zeynep Akata

Due to the importance of zero-shot learning, i. e. classifying images where there is a lack of labeled training data, the number of proposed approaches has recently increased steadily.

Zero-Shot Learning

Multiple People Tracking by Lifted Multicut and Person Re-Identification

no code implementations CVPR 2017 Siyu Tang, Mykhaylo Andriluka, Bjoern Andres, Bernt Schiele

This allows us to reward tracks that assign detections of similar appearance to the same person in a way that does not introduce implausible solutions.

Multiple People Tracking Person Re-Identification +1

Pose Guided Person Image Generation

2 code implementations NeurIPS 2017 Liqian Ma, Xu Jia, Qianru Sun, Bernt Schiele, Tinne Tuytelaars, Luc van Gool

This paper proposes the novel Pose Guided Person Generation Network (PG$^2$) that allows to synthesize person images in arbitrary poses, based on an image of that person and a novel pose.

Gesture-to-Gesture Translation Pose Transfer

Learning non-maximum suppression

no code implementations CVPR 2017 Jan Hosang, Rodrigo Benenson, Bernt Schiele

Object detectors have hugely profited from moving towards an end-to-end learning paradigm: proposals, features, and the classifier becoming one neural network improved results two-fold on general object detection.

Human Detection Object Detection +1

Generating Descriptions with Grounded and Co-Referenced People

no code implementations CVPR 2017 Anna Rohrbach, Marcus Rohrbach, Siyu Tang, Seong Joon Oh, Bernt Schiele

At training time, we first learn how to localize characters by relating their visual appearance to mentions in the descriptions via a semi-supervised approach.

Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images

no code implementations ICCV 2017 Tribhuvanesh Orekondy, Bernt Schiele, Mario Fritz

Third, we propose models that predict user specific privacy score from images in order to enforce the users' privacy preferences.

Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective

no code implementations ICCV 2017 Seong Joon Oh, Mario Fritz, Bernt Schiele

We derive the optimal strategy for the user that assures an upper bound on the recognition rate independent of the recogniser's counter measure.

Person Recognition

Zero-Shot Learning -- The Good, the Bad and the Ugly

no code implementations CVPR 2017 Yongqin Xian, Bernt Schiele, Zeynep Akata

Due to the importance of zero-shot learning, the number of proposed approaches has increased steadily recently.

Zero-Shot Learning

CityPersons: A Diverse Dataset for Pedestrian Detection

1 code implementation CVPR 2017 Shanshan Zhang, Rodrigo Benenson, Bernt Schiele

Convnets have enabled significant progress in pedestrian detection recently, but there are still open questions regarding suitable architectures and training data.

Pedestrian Detection

Exploiting saliency for object segmentation from image level labels

no code implementations CVPR 2017 Seong Joon Oh, Rodrigo Benenson, Anna Khoreva, Zeynep Akata, Mario Fritz, Bernt Schiele

We show how to combine both information sources in order to recover 80% of the fully supervised performance - which is the new state of the art in weakly supervised training for pixel-wise semantic labelling.

Semantic Segmentation

EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras (Extended Abstract)

no code implementations31 Dec 2016 Helge Rhodin, Christian Richardt, Dan Casas, Eldar Insafutdinov, Mohammad Shafiei, Hans-Peter Seidel, Bernt Schiele, Christian Theobalt

Marker-based and marker-less optical skeletal motion-capture methods use an outside-in arrangement of cameras placed around a scene, with viewpoints converging on the center.

Pose Estimation

Attentive Explanations: Justifying Decisions and Pointing to the Evidence

no code implementations14 Dec 2016 Dong Huk Park, Lisa Anne Hendricks, Zeynep Akata, Bernt Schiele, Trevor Darrell, Marcus Rohrbach

In contrast, humans can justify their decisions with natural language and point to the evidence in the visual world which led to their decisions.

Decision Making Question Answering +1

Analysis and Optimization of Loss Functions for Multiclass, Top-k, and Multilabel Classification

1 code implementation12 Dec 2016 Maksim Lapin, Matthias Hein, Bernt Schiele

In particular, we find that it is possible to obtain effective multilabel classifiers on Pascal VOC using a single label per image for training, while the gap between multiclass and multilabel methods on MS COCO is more significant.

General Classification Image Classification

Learning Video Object Segmentation from Static Images

2 code implementations CVPR 2017 Anna Khoreva, Federico Perazzi, Rodrigo Benenson, Bernt Schiele, Alexander Sorkine-Hornung

Inspired by recent advances of deep learning in instance segmentation and object tracking, we introduce video object segmentation problem as a concept of guided instance segmentation.

Instance Segmentation Semantic Segmentation +3

Gaze Embeddings for Zero-Shot Image Classification

no code implementations CVPR 2017 Nour Karessli, Zeynep Akata, Bernt Schiele, Andreas Bulling

Zero-shot image classification using auxiliary information, such as attributes describing discriminative object properties, requires time-consuming annotation by domain experts.

Fine-Grained Image Classification General Classification

Long-Term Image Boundary Prediction

no code implementations27 Nov 2016 Apratim Bhattacharyya, Mateusz Malinowski, Bernt Schiele, Mario Fritz

Boundary estimation in images and videos has been a very active topic of research, and organizing visual information into boundaries and segments is believed to be a corner stone of visual perception.

Joint Graph Decomposition and Node Labeling: Problem, Algorithms, Applications

1 code implementation14 Nov 2016 Evgeny Levinkov, Jonas Uhrig, Siyu Tang, Mohamed Omran, Eldar Insafutdinov, Alexander Kirillov, Carsten Rother, Thomas Brox, Bernt Schiele, Bjoern Andres

In order to find feasible solutions efficiently, we define two local search algorithms that converge monotonously to a local optimum, offering a feasible solution at any time.

Combinatorial Optimization Multiple Object Tracking +2

Learning What and Where to Draw

no code implementations NeurIPS 2016 Scott Reed, Zeynep Akata, Santosh Mohan, Samuel Tenka, Bernt Schiele, Honglak Lee

Generative Adversarial Networks (GANs) have recently demonstrated the capability to synthesize compelling real-world images, such as room interiors, album covers, manga, faces, birds, and flowers.

Text-to-Image Generation

EgoCap: Egocentric Marker-less Motion Capture with Two Fisheye Cameras

no code implementations23 Sep 2016 Helge Rhodin, Christian Richardt, Dan Casas, Eldar Insafutdinov, Mohammad Shafiei, Hans-Peter Seidel, Bernt Schiele, Christian Theobalt

We therefore propose a new method for real-time, marker-less and egocentric motion capture which estimates the full-body skeleton pose from a lightweight stereo pair of fisheye cameras that are attached to a helmet or virtual reality headset.

Pose Estimation

Multi-Person Tracking by Multicut and Deep Matching

no code implementations17 Aug 2016 Siyu Tang, Bjoern Andres, Mykhaylo Andriluka, Bernt Schiele

In [1], we proposed a graph-based formulation that links and clusters person hypotheses over time by solving a minimum cost subgraph multicut problem.

Faceless Person Recognition; Privacy Implications in Social Media

no code implementations28 Jul 2016 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

As we shift more of our lives into the virtual domain, the volume of data shared on the web keeps increasing and presents a threat to our privacy.

Person Recognition

A Multi-cut Formulation for Joint Segmentation and Tracking of Multiple Objects

no code implementations21 Jul 2016 Margret Keuper, Siyu Tang, Yu Zhongjie, Bjoern Andres, Thomas Brox, Bernt Schiele

Recently, Minimum Cost Multicut Formulations have been proposed and proven to be successful in both motion trajectory segmentation and multi-target tracking scenarios.

Motion Segmentation Object Detection

Generative Adversarial Text to Image Synthesis

38 code implementations17 May 2016 Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, Honglak Lee

Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal.

Adversarial Text Text-to-Image Generation

Movie Description

no code implementations12 May 2016 Anna Rohrbach, Atousa Torabi, Marcus Rohrbach, Niket Tandon, Christopher Pal, Hugo Larochelle, Aaron Courville, Bernt Schiele

In addition we also collected and aligned movie scripts used in prior work and compare the two sources of descriptions.

Latent Embeddings for Zero-shot Classification

no code implementations CVPR 2016 Yongqin Xian, Zeynep Akata, Gaurav Sharma, Quynh Nguyen, Matthias Hein, Bernt Schiele

We train the model with a ranking based objective function which penalizes incorrect rankings of the true class for a given image.

General Classification Zero-Shot Learning

Multi-Cue Zero-Shot Learning with Strong Supervision

no code implementations CVPR 2016 Zeynep Akata, Mateusz Malinowski, Mario Fritz, Bernt Schiele

A promising research direction is zero-shot learning, which does not require any training data to recognize new classes, but rather relies on some form of auxiliary information describing the new classes.

Zero-Shot Learning

Generating Visual Explanations

no code implementations28 Mar 2016 Lisa Anne Hendricks, Zeynep Akata, Marcus Rohrbach, Jeff Donahue, Bernt Schiele, Trevor Darrell

Clearly explaining a rationale for a classification decision to an end-user can be as important as the decision itself.

General Classification

How Far are We from Solving Pedestrian Detection?

no code implementations CVPR 2016 Shanshan Zhang, Rodrigo Benenson, Mohamed Omran, Jan Hosang, Bernt Schiele

We enable our analysis by creating a human baseline for pedestrian detection (over the Caltech dataset), and by manually clustering the recurrent errors of a top detector.

Pedestrian Detection

Loss Functions for Top-k Error: Analysis and Insights

1 code implementation CVPR 2016 Maksim Lapin, Matthias Hein, Bernt Schiele

In the experiments, we compare on various datasets all of the proposed and established methods for top-k error optimization.

Top-k Multiclass SVM

1 code implementation NeurIPS 2015 Maksim Lapin, Matthias Hein, Bernt Schiele

Class ambiguity is typical in image classification problems with a large number of classes.

General Classification Image Classification

A convnet for non-maximum suppression

no code implementations19 Nov 2015 Jan Hosang, Rodrigo Benenson, Bernt Schiele

Non-maximum suppression (NMS) is used in virtually all state-of-the-art object detection pipelines.

Object Detection Pedestrian Detection

Efficient Output Kernel Learning for Multiple Tasks

no code implementations NeurIPS 2015 Pratik Jawanpuria, Maksim Lapin, Matthias Hein, Bernt Schiele

The paradigm of multi-task learning is that one can achieve better generalization by learning tasks jointly and thus exploiting the similarity between the tasks rather than learning them independently of each other.

Multi-Task Learning

Grounding of Textual Phrases in Images by Reconstruction

3 code implementations12 Nov 2015 Anna Rohrbach, Marcus Rohrbach, Ronghang Hu, Trevor Darrell, Bernt Schiele

We propose a novel approach which learns grounding by reconstructing a given phrase using an attention mechanism, which can be either latent or optimized directly.

Language Modelling Natural Language Visual Grounding +2

Scalable Nonlinear Embeddings for Semantic Category-based Image Retrieval

no code implementations ICCV 2015 Gaurav Sharma, Bernt Schiele

We propose a novel algorithm for the task of supervised discriminative distance learning by nonlinearly embedding vectors into a low dimensional Euclidean space.

Image Retrieval Metric Learning

Person Recognition in Personal Photo Collections

no code implementations ICCV 2015 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

Recognising persons in everyday photos presents major challenges (occluded faces, different clothing, locations, etc.)

Person Recognition

The Long-Short Story of Movie Description

no code implementations4 Jun 2015 Anna Rohrbach, Marcus Rohrbach, Bernt Schiele

Generating descriptions for videos has many applications including assisting blind people and human-robot interaction.

Human robot interaction Image Captioning

Subgraph Decomposition for Multi-Target Tracking

no code implementations CVPR 2015 Siyu Tang, Bjoern Andres, Miykhaylo Andriluka, Bernt Schiele

Tracking multiple targets in a video, based on a finite set of detection hypotheses, is a persistent problem in computer vision.

Filtered Feature Channels for Pedestrian Detection

no code implementations CVPR 2015 Shanshan Zhang, Rodrigo Benenson, Bernt Schiele

This paper starts from the observation that multiple top performing pedestrian detectors can be modelled by using an intermediate layer filtering low-level features in combination with a boosted decision forest.

Optical Flow Estimation Pedestrian Detection

Efficient ConvNet-Based Marker-Less Motion Capture in General Scenes With a Low Number of Cameras

no code implementations CVPR 2015 Ahmed Elhayek, Edilson de Aguiar, Arjun Jain, Jonathan Tompson, Leonid Pishchulin, Micha Andriluka, Chris Bregler, Bernt Schiele, Christian Theobalt

Our approach unites a discriminative image-based joint detection method with a model-based generative motion tracking algorithm through a combined pose optimization energy.

Pose Estimation

Building Statistical Shape Spaces for 3D Human Modeling

no code implementations19 Mar 2015 Leonid Pishchulin, Stefanie Wuhrer, Thomas Helten, Christian Theobalt, Bernt Schiele

Statistical models of 3D human shape and pose learned from scan databases have developed into valuable tools to solve a variety of vision and graphics problems.

3D Object Class Detection in the Wild

no code implementations17 Mar 2015 Bojan Pepik, Michael Stark, Peter Gehler, Tobias Ritschel, Bernt Schiele

Object class detection has been a synonym for 2D bounding box localization for the longest time, fueled by the success of powerful statistical learning techniques, combined with robust image representations.

Object Detection Viewpoint Estimation

Recognizing Fine-Grained and Composite Activities using Hand-Centric Features and Script Data

no code implementations23 Feb 2015 Marcus Rohrbach, Anna Rohrbach, Michaela Regneri, Sikandar Amin, Mykhaylo Andriluka, Manfred Pinkal, Bernt Schiele

To attack the second challenge, recognizing composite activities, we leverage the fact that these activities are compositional and that the essential components of the activities can be obtained from textual descriptions or scripts.

Activity Recognition

What makes for effective detection proposals?

no code implementations17 Feb 2015 Jan Hosang, Rodrigo Benenson, Piotr Dollár, Bernt Schiele

Current top performing object detectors employ detection proposals to guide the search for objects, thereby avoiding exhaustive sliding window search across images.

Object Detection

Filtered Channel Features for Pedestrian Detection

no code implementations23 Jan 2015 Shanshan Zhang, Rodrigo Benenson, Bernt Schiele

This paper starts from the observation that multiple top performing pedestrian detectors can be modelled by using an intermediate layer filtering low-level features in combination with a boosted decision forest.

Optical Flow Estimation Pedestrian Detection

A Dataset for Movie Description

no code implementations CVPR 2015 Anna Rohrbach, Marcus Rohrbach, Niket Tandon, Bernt Schiele

In this work we propose a novel dataset which contains transcribed DVS, which is temporally aligned to full length HD movies.

Ten Years of Pedestrian Detection, What Have We Learned?

no code implementations16 Nov 2014 Rodrigo Benenson, Mohamed Omran, Jan Hosang, Bernt Schiele

Paper-by-paper results make it easy to miss the forest for the trees. We analyse the remarkable progress of the last decade by discussing the main ideas explored in the 40+ detectors currently present in the Caltech pedestrian detection benchmark.

Pedestrian Detection

Multiple human pose estimation with temporally consistent 3d pictorial structures

no code implementations6 Sep 2014 Vasileios Belagiannis, Xinchao Wang, Bernt Schiele, Pascal Fua, Slobodan Ilic, Nassir Navab

To address these challenges, we propose a temporally consistent 3D Pictorial Structures model (3DPS) for multiple human pose estimation from multiple camera views.

3D Multi-Person Pose Estimation 3D Pose Estimation

How good are detection proposals, really?

no code implementations26 Jun 2014 Jan Hosang, Rodrigo Benenson, Bernt Schiele

Current top performing Pascal VOC object detectors employ detection proposals to guide the search for objects thereby avoiding exhaustive sliding window search across images.

Object Detection

Fine-grained Activity Recognition with Holistic and Pose based Features

no code implementations7 Jun 2014 Leonid Pishchulin, Mykhaylo Andriluka, Bernt Schiele

Holistic methods based on dense trajectories are currently the de facto standard for recognition of human activities in video.

Activity Recognition

Spectral Graph Reduction for Efficient Image and Streaming Video Segmentation

no code implementations CVPR 2014 Fabio Galasso, Margret Keuper, Thomas Brox, Bernt Schiele

In contrast to previous work, the reduced graph is reweighted such that the resulting segmentation is equivalent, under certain assumptions, to that of the full graph.

Superpixels Video Segmentation +1

Multi-View Priors for Learning Detectors from Sparse Viewpoint Data

no code implementations20 Dec 2013 Bojan Pepik, Michael Stark, Peter Gehler, Bernt Schiele

While the majority of today's object class models provide only 2D bounding boxes, far richer output hypotheses are desirable including viewpoint, fine-grained category, and 3D geometry estimate.

Object Localization Transfer Learning +1

Transfer Learning in a Transductive Setting

no code implementations NeurIPS 2013 Marcus Rohrbach, Sandra Ebert, Bernt Schiele

Our approach consistently outperforms state-of-the-art transfer and semi-supervised approaches on all datasets.

Activity Recognition Few-Shot Learning +2

Learning Using Privileged Information: SVM+ and Weighted SVM

no code implementations13 Jun 2013 Maksim Lapin, Matthias Hein, Bernt Schiele

Prior knowledge can be used to improve predictive performance of learning algorithms or reduce the amount of data required for training.

Poselet Conditioned Pictorial Structures

no code implementations CVPR 2013 Leonid Pishchulin, Mykhaylo Andriluka, Peter Gehler, Bernt Schiele

In this paper we consider the challenging problem of articulated human pose estimation in still images.

Pose Estimation

Occlusion Patterns for Object Class Detection

no code implementations CVPR 2013 Bojan Pepikj, Michael Stark, Peter Gehler, Bernt Schiele

Despite the success of recent object class recognition systems, the long-standing problem of partial occlusion remains a major challenge, and a principled solution is yet to be found.

Grounding Action Descriptions in Videos

no code implementations TACL 2013 Michaela Regneri, Marcus Rohrbach, Dominikus Wetzel, Stefan Thater, Bernt Schiele, Manfred Pinkal

Recent work has shown that the integration of visual information into text-based models can substantially improve model predictions, but so far only visual information extracted from static images has been used.

Semantic Textual Similarity Video Understanding

Cannot find the paper you are looking for? You can Submit a new open access paper.