Search Results for author: Mario Fritz

Found 135 papers, 51 papers with code

Towards Automated Testing and Robustification by Semantic Adversarial Data Generation

no code implementations ECCV 2020 Rakshith Shetty, Mario Fritz, Bernt Schiele

Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector.

Data Augmentation Object

Can LLMs Separate Instructions From Data? And What Do We Even Mean By That?

1 code implementation11 Mar 2024 Egor Zverev, Sahar Abdelnabi, Mario Fritz, Christoph H. Lampert

Instruction-tuned Large Language Models (LLMs) have achieved breakthrough results, opening countless new possibilities for many practical applications.

LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History

1 code implementation28 Feb 2024 Akash Gupta, Ivaxi Sheth, Vyas Raina, Mark Gales, Mario Fritz

With the recent emergence of powerful instruction-tuned large language models (LLMs), various helpful conversational Artificial Intelligence (AI) systems have been deployed across many applications.

Exploring Value Biases: How LLMs Deviate Towards the Ideal

no code implementations16 Feb 2024 Sarath Sivaprasad, Pramod Kaushik, Sahar Abdelnabi, Mario Fritz

We study this sampling of LLMs in light of value bias and show that the sampling of LLMs tends to favour high-value options.

Adaptive Hierarchical Certification for Segmentation using Randomized Smoothing

no code implementations13 Feb 2024 Alaa Anani, Tobias Lorenz, Bernt Schiele, Mario Fritz

In this paper, however, we propose a novel, more general, and practical setting, namely adaptive hierarchical certification for image semantic segmentation.

Semantic Segmentation

Towards Biologically Plausible and Private Gene Expression Data Generation

1 code implementation7 Feb 2024 Dingfan Chen, Marie Oestreich, Tejumade Afonja, Raouf Kerkouche, Matthias Becker, Mario Fritz

In this paper, we initiate a systematic analysis of how DP generative models perform in their natural application scenarios, specifically focusing on real-world gene expression data.

Benchmarking

Privacy-Aware Document Visual Question Answering

no code implementations15 Dec 2023 Rubèn Tito, Khanh Nguyen, Marlon Tobaben, Raouf Kerkouche, Mohamed Ali Souibgui, Kangsoo Jung, Lei Kang, Ernest Valveny, Antti Honkela, Mario Fritz, Dimosthenis Karatzas

We employ a federated learning scheme, that reflects the real-life distribution of documents in different businesses, and we explore the use case where the ID of the invoice issuer is the sensitive information to be protected.

document understanding Federated Learning +3

Don't Miss Out on Novelty: Importance of Novel Features for Deep Anomaly Detection

no code implementations1 Oct 2023 Sarath Sivaprasad, Mario Fritz

We propose a novel approach to AD using explainability to capture such novel features as unexplained observations in the input space.

Anomaly Detection

LLM-Deliberation: Evaluating LLMs with Interactive Multi-Agent Negotiation Games

2 code implementations29 Sep 2023 Sahar Abdelnabi, Amr Gomaa, Sarath Sivaprasad, Lea Schönherr, Mario Fritz

There is a growing interest in using Large Language Models (LLMs) as agents to tackle real-world tasks that may require assessing complex situations.

Decision Making

A Unified View of Differentially Private Deep Generative Modeling

no code implementations27 Sep 2023 Dingfan Chen, Raouf Kerkouche, Mario Fritz

The availability of rich and vast data sources has greatly advanced machine learning applications in various domains.

Privacy Preserving

Certified Robust Models with Slack Control and Large Lipschitz Constants

1 code implementation12 Sep 2023 Max Losch, David Stutz, Bernt Schiele, Mario Fritz

In this paper, we propose a Calibrated Lipschitz-Margin Loss (CLL) that addresses this issue and improves certified robustness by tackling two problems: Firstly, commonly used margin losses do not adjust the penalties to the shrinking output distribution; caused by minimizing the Lipschitz constant $K$.

MargCTGAN: A "Marginally'' Better CTGAN for the Low Sample Regime

no code implementations16 Jul 2023 Tejumade Afonja, Dingfan Chen, Mario Fritz

The potential of realistic and useful synthetic data is significant.

B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers

1 code implementation19 Jun 2023 Moritz Böhle, Navdeeppal Singh, Mario Fritz, Bernt Schiele

We present a new direction for increasing the interpretability of deep neural networks (DNNs) by promoting weight-input alignment during training.

Client-specific Property Inference against Secure Aggregation in Federated Learning

1 code implementation7 Mar 2023 Raouf Kerkouche, Gergely Ács, Mario Fritz

We formulate an optimization problem across different rounds in order to infer a tested property of every client from the output of the linear models, for example, whether they have a specific sample in their training data (membership inference) or whether they misbehave and attempt to degrade the performance of the common model by poisoning attacks.

Federated Learning

Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy

no code implementations15 Feb 2023 Derui Zhu, Dingfan Chen, Jens Grossklags, Mario Fritz

In recent years, diffusion models have achieved tremendous success in the field of image generation, becoming the stateof-the-art technology for AI-based image processing applications.

Image Generation

Holistically Explainable Vision Transformers

no code implementations20 Jan 2023 Moritz Böhle, Mario Fritz, Bernt Schiele

Transformers increasingly dominate the machine learning landscape across many tasks and domains, which increases the importance for understanding their outputs.

Private Set Generation with Discriminative Information

2 code implementations7 Nov 2022 Dingfan Chen, Raouf Kerkouche, Mario Fritz

Differentially private data generation techniques have become a promising solution to the data privacy challenge -- it enables sharing of data while complying with rigorous privacy guarantees, which is essential for scientific progress in sensitive domains.

SimSCOOD: Systematic Analysis of Out-of-Distribution Generalization in Fine-tuned Source Code Models

1 code implementation10 Oct 2022 Hossein Hajipour, Ning Yu, Cristian-Alexandru Staicu, Mario Fritz

In this paper, we contribute the first systematic approach that simulates various OOD scenarios along different dimensions of source code data properties and study the fine-tuned model behaviors in such scenarios.

Code Generation Out-of-Distribution Generalization

UnGANable: Defending Against GAN-based Face Manipulation

1 code implementation3 Oct 2022 Zheng Li, Ning Yu, Ahmed Salem, Michael Backes, Mario Fritz, Yang Zhang

Extensive experiments on four popular GAN models trained on two benchmark face datasets show that UnGANable achieves remarkable effectiveness and utility performance, and outperforms multiple baseline methods.

Face Swapping Misinformation

Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systems

1 code implementation7 Sep 2022 Sahar Abdelnabi, Mario Fritz

In this work, we assume an adversary that automatically tampers with the online evidence in order to disrupt the fact-checking model via camouflaging the relevant evidence or planting a misleading one.

Fact Checking Fact Verification +1

RelaxLoss: Defending Membership Inference Attacks without Losing Utility

1 code implementation ICLR 2022 Dingfan Chen, Ning Yu, Mario Fritz

As a long-term threat to the privacy of training data, membership inference attacks (MIAs) emerge ubiquitously in machine learning models.

B-cos Networks: Alignment is All We Need for Interpretability

1 code implementation CVPR 2022 Moritz Böhle, Mario Fritz, Bernt Schiele

We present a new direction for increasing the interpretability of deep neural networks (DNNs) by promoting weight-input alignment during training.

Practical Challenges in Differentially-Private Federated Survival Analysis of Medical Data

no code implementations8 Feb 2022 Shadi Rahimian, Raouf Kerkouche, Ina Kurth, Mario Fritz

Survival analysis or time-to-event analysis aims to model and predict the time it takes for an event of interest to happen in a population or an individual.

Federated Learning Survival Analysis

ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training

1 code implementation11 Oct 2021 Hui-Po Wang, Sebastian U. Stich, Yang He, Mario Fritz

Federated learning is a powerful distributed learning scheme that allows numerous edge devices to collaboratively train a model without sharing their data.

Federated Learning Image Segmentation +2

Optimising for Interpretability: Convolutional Dynamic Alignment Networks

1 code implementation27 Sep 2021 Moritz Böhle, Mario Fritz, Bernt Schiele

As a result, CoDA Nets model the classification prediction through a series of input-dependent linear transformations, allowing for linear decomposition of the output into individual input contributions.

Certifiers Make Neural Networks Vulnerable to Availability Attacks

no code implementations25 Aug 2021 Tobias Lorenz, Marta Kwiatkowska, Mario Fritz

While this is a key concept towards safe and secure AI, we show for the first time that this approach comes with its own security risks, as such fallback strategies can be deliberately triggered by an adversary.

Data Poisoning

Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis

1 code implementation29 May 2021 Yang He, Ning Yu, Margret Keuper, Mario Fritz

The rapid advances in deep generative models over the past years have led to highly {realistic media, known as deepfakes,} that are commonly indistinguishable from real to human eyes.

Colorization Denoising +2

Convolutional Dynamic Alignment Networks for Interpretable Classifications

1 code implementation CVPR 2021 Moritz Böhle, Mario Fritz, Bernt Schiele

Given the alignment of the DAUs, the resulting contribution maps align with discriminative input patterns.

ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models

1 code implementation4 Feb 2021 Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, Yang Zhang

As a result, we lack a comprehensive picture of the risks caused by the attacks, e. g., the different scenarios they can be applied to, the common factors that influence their performance, the relationship among them, or the effectiveness of possible defenses.

Attribute BIG-bench Machine Learning +3

CosSGD: Communication-Efficient Federated Learning with a Simple Cosine-Based Quantization

no code implementations15 Dec 2020 Yang He, Hui-Po Wang, Maximilian Zenk, Mario Fritz

Despite notable progress in gradient compression, the existing quantization methods require further improvement when low-bits compression is applied, especially the overall systems often degenerate a lot when quantization are applied in double directions to compress model weights and gradients.

Federated Learning Image Classification +2

Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs

1 code implementation CVPR 2021 Hui-Po Wang, Ning Yu, Mario Fritz

While Generative Adversarial Networks (GANs) show increasing performance and the level of realism is becoming indistinguishable from natural images, this also comes with high demands on data and computation.

Image Generation Unconditional Image Generation

SampleFix: Learning to Correct Programs by Efficient Sampling of Diverse Fixes

no code implementations NeurIPS Workshop CAP 2020 Hossein Hajipour, Apratim Bhattacharyya, Mario Fritz

Therefore, we propose a deep generative model to automatically correct programming errors by learning a distribution over potential fixes.

Haar Wavelet based Block Autoregressive Flows for Trajectories

no code implementations21 Sep 2020 Apratim Bhattacharyya, Christoph-Nikolas Straehle, Mario Fritz, Bernt Schiele

This yields an exact inference method that models trajectories at different spatio-temporal resolutions in a hierarchical manner.

Synthetic Convolutional Features for Improved Semantic Segmentation

no code implementations18 Sep 2020 Yang He, Bernt Schiele, Mario Fritz

Recently, learning-based image synthesis has enabled to generate high-resolution images, either applying popular adversarial training or a powerful perceptual loss.

Image Generation Segmentation +1

Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding

1 code implementation7 Sep 2020 Sahar Abdelnabi, Mario Fritz

In this paper, we study natural language watermarking as a defense to help better mark and trace the provenance of text.

Denoising Text Generation

Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries

no code implementations1 Sep 2020 Shadi Rahimian, Tribhuvanesh Orekondy, Mario Fritz

Our work consists of two sides: We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model.

BIG-bench Machine Learning Inference Attack +1

Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data

1 code implementation ICCV 2021 Ning Yu, Vladislav Skripniuk, Sahar Abdelnabi, Mario Fritz

Thus, we seek a proactive and sustainable solution on deepfake detection, that is agnostic to the evolution of generative models, by introducing artificial fingerprints into the models.

DeepFake Detection Face Swapping +2

GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators

1 code implementation NeurIPS 2020 Dingfan Chen, Tribhuvanesh Orekondy, Mario Fritz

The wide-spread availability of rich data has fueled the growth of machine learning applications in numerous domains.

InfoScrub: Towards Attribute Privacy by Targeted Obfuscation

no code implementations20 May 2020 Hui-Po Wang, Tribhuvanesh Orekondy, Mario Fritz

Personal photos of individuals when shared online, apart from exhibiting a myriad of memorable details, also reveals a wide range of private information and potentially entails privacy risks (e. g., online harassment, tracking).

Attribute Translation

Normalizing Flows with Multi-Scale Autoregressive Priors

1 code implementation CVPR 2020 Shweta Mahajan, Apratim Bhattacharyya, Mario Fritz, Bernt Schiele, Stefan Roth

Flow-based generative models are an important class of exact inference models that admit efficient inference and sampling for image synthesis.

Density Estimation Image Generation

Long-Tailed Recognition Using Class-Balanced Experts

1 code implementation7 Apr 2020 Saurabh Sharma, Ning Yu, Mario Fritz, Bernt Schiele

Deep learning enables impressive performance in image recognition using large-scale artificially-balanced datasets.

Long-tail Learning

Inclusive GAN: Improving Data and Minority Coverage in Generative Models

1 code implementation ECCV 2020 Ning Yu, Ke Li, Peng Zhou, Jitendra Malik, Larry Davis, Mario Fritz

Generative Adversarial Networks (GANs) have brought about rapid progress towards generating photorealistic images.

Everything About You: A Multimodal Approach towards Friendship Inference in Online Social Networks

1 code implementation2 Mar 2020 Tahleen Rahman, Mario Fritz, Michael Backes, Yang Zhang

Most previous works in privacy of Online Social Networks (OSN) focus on a restricted scenario of using one type of information to infer another type of information or using only static profile data such as username, profile picture or home location.

Social and Information Networks

Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing

no code implementations CVPR 2020 Vedika Agarwal, Rakshith Shetty, Mario Fritz

Despite significant success in Visual Question Answering (VQA), VQA models have been shown to be notoriously brittle to linguistic variations in the questions.

Question Answering Visual Question Answering

"Best-of-Many-Samples" Distribution Matching

1 code implementation27 Sep 2019 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.

``"Best-of-Many-Samples" Distribution Matching

no code implementations25 Sep 2019 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.

GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models

1 code implementation9 Sep 2019 Dingfan Chen, Ning Yu, Yang Zhang, Mario Fritz

In addition, we propose the first generic attack model that can be instantiated in a large range of settings and is applicable to various kinds of deep generative models.

Inference Attack Membership Inference Attack

Conditional Flow Variational Autoencoders for Structured Sequence Prediction

no code implementations24 Aug 2019 Apratim Bhattacharyya, Michael Hanselmann, Mario Fritz, Bernt Schiele, Christoph-Nikolas Straehle

Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world.

Trajectory Prediction

Interpretability Beyond Classification Output: Semantic Bottleneck Networks

no code implementations25 Jul 2019 Max Losch, Mario Fritz, Bernt Schiele

Additionally we show how the activations of the SB-Layer can be used for both the interpretation of failure cases of the network as well as for confidence prediction of the resulting output.

Classification Dimensionality Reduction +2

Body Shape Privacy in Images: Understanding Privacy and Preventing Automatic Shape Extraction

no code implementations27 May 2019 Hosnieh Sattar, Katharina Krombholz, Gerard Pons-Moll, Mario Fritz

Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions.

Recommendation Systems Virtual Try-on

Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning

no code implementations1 Apr 2019 Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, Yang Zhang

As data generation is a continuous process, this leads to ML model owners updating their models frequently with newly-collected data in an online learning scenario.

Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation

no code implementations17 Dec 2018 Rakshith Shetty, Bernt Schiele, Mario Fritz

We propose a method to quantify the sensitivity of black-box vision models to visual context by editing images to remove selected objects and measuring the response of the target models.

Classification Data Augmentation +5

Knockoff Nets: Stealing Functionality of Black-Box Models

2 code implementations CVPR 2019 Tribhuvanesh Orekondy, Bernt Schiele, Mario Fritz

We formulate model functionality stealing as a two-step approach: (i) querying a set of input images to the blackbox model to obtain predictions; and (ii) training a "knockoff" with queried image-prediction pairs.

Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints

2 code implementations ICCV 2019 Ning Yu, Larry Davis, Mario Fritz

Our experiments show that (1) GANs carry distinct model fingerprints and leave stable fingerprints in their generated images, which support image attribution; (2) even minor differences in GAN training can result in different fingerprints, which enables fine-grained model authentication; (3) fingerprints persist across different image frequencies and patches and are not biased by GAN artifacts; (4) fingerprint finetuning is effective in immunizing against five types of adversarial image perturbations; and (5) comparisons also show our learned fingerprints consistently outperform several baselines in a variety of setups.

Image Generation

Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods

1 code implementation ICLR 2019 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

For autonomous agents to successfully operate in the real world, the ability to anticipate future scene states is a key competence.

Bayesian Inference Precipitation Forecasting

Fashion is Taking Shape: Understanding Clothing Preference Based on Body Shape From Online Sources

no code implementations9 Jul 2018 Hosnieh Sattar, Gerard Pons-Moll, Mario Fritz

To study the correlation between clothing garments and body shape, we collected a new dataset (Fashion Takes Shape), which includes images of users with clothing category annotations.

Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective

1 code implementation20 Jun 2018 Apratim Bhattacharyya, Bernt Schiele, Mario Fritz

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.

Human Pose Forecasting

Bayesian Prediction of Future Street Scenes through Importance Sampling based Optimization

no code implementations18 Jun 2018 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.

Future prediction Segmentation +1

Adversarial Scene Editing: Automatic Object Removal from Weak Supervision

no code implementations NeurIPS 2018 Rakshith Shetty, Mario Fritz, Bernt Schiele

While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets.

Generative Adversarial Network Image Manipulation +1

ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models

7 code implementations4 Jun 2018 Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, Michael Backes

In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.

BIG-bench Machine Learning Inference Attack +1

Accurate and Diverse Sampling of Sequences Based on a “Best of Many” Sample Objective

no code implementations CVPR 2018 Apratim Bhattacharyya, Bernt Schiele, Mario Fritz

For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.

Sequential Attacks on Agents for Long-Term Adversarial Goals

no code implementations31 May 2018 Edgar Tretschk, Seong Joon Oh, Mario Fritz

As a result of our attack, the victim agent is misguided to optimise for the adversarial reward over time.

Adversarial Attack Reinforcement Learning (RL) +1

Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning

no code implementations15 May 2018 Tribhuvanesh Orekondy, Seong Joon Oh, Yang Zhang, Bernt Schiele, Mario Fritz

At the core of FL is a network of anonymous user devices sharing training information (model parameter updates) computed locally on personal data.

Data Augmentation Federated Learning +1

A Hybrid Model for Identity Obfuscation by Face Replacement

no code implementations ECCV 2018 Qianru Sun, Ayush Tewari, Weipeng Xu, Mario Fritz, Christian Theobalt, Bernt Schiele

As more and more personal photos are shared and tagged in social media, avoiding privacy risks such as unintended recognition becomes increasingly challenging.

Face Generation

Deep Appearance Maps

no code implementations ICCV 2019 Maxim Maximov, Laura Leal-Taixé, Mario Fritz, Tobias Ritschel

Second, we demonstrate how another network can be used to map from an image or video frames to a DAM network to reproduce this appearance, without using a lengthy optimization such as stochastic gradient descent (learning-to-learn).

Disentangled Person Image Generation

1 code implementation CVPR 2018 Liqian Ma, Qianru Sun, Stamatios Georgoulis, Luc van Gool, Bernt Schiele, Mario Fritz

Generating novel, yet realistic, images of persons is a challenging task due to the complex interplay between the different image factors, such as the foreground, background and pose information.

Gesture-to-Gesture Translation Person Re-Identification +1

Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty

no code implementations CVPR 2018 Apratim Bhattacharyya, Mario Fritz, Bernt Schiele

Our experimental results show that it is indeed possible to predict people trajectories at the desired time horizons and that our uncertainty estimates are informative of the prediction error.

Autonomous Driving Trajectory Prediction

Natural and Effective Obfuscation by Head Inpainting

no code implementations CVPR 2018 Qianru Sun, Liqian Ma, Seong Joon Oh, Luc van Gool, Bernt Schiele, Mario Fritz

As more and more personal photos are shared online, being able to obfuscate identities in such photos is becoming a necessity for privacy protection.

MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation

6 code implementations24 Nov 2017 Xucong Zhang, Yusuke Sugano, Mario Fritz, Andreas Bulling

Second, we present an extensive evaluation of state-of-the-art gaze estimation methods on three current datasets, including MPIIGaze.

Gaze Estimation

Towards Reverse-Engineering Black-Box Neural Networks

3 code implementations ICLR 2018 Seong Joon Oh, Max Augustin, Bernt Schiele, Mario Fritz

On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -- we show that the revealed internal information helps generate more effective adversarial examples against the black box model.

$A^{4}NT$: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation

no code implementations6 Nov 2017 Rakshith Shetty, Bernt Schiele, Mario Fritz

In this paper, we propose an automatic method, called Adversarial Author Attribute Anonymity Neural Translation ($A^4NT$), to combat such text-based adversaries.

Attribute Machine Translation +1

Acquiring Target Stacking Skills by Goal-Parameterized Deep Reinforcement Learning

no code implementations ICLR 2018 Wenbin Li, Jeannette Bohg, Mario Fritz

We created a synthetic block stacking environment with physics simulation in which the agent can learn a policy end-to-end through trial and error.

reinforcement-learning Reinforcement Learning (RL)

Person Recognition in Personal Photo Collections

no code implementations9 Oct 2017 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

Person recognition in social media photos sets new challenges for computer vision, including non-cooperative subjects (e. g. backward viewpoints, unusual poses) and great changes in appearance.

Face Recognition Person Recognition

Learning Dilation Factors for Semantic Segmentation of Street Scenes

1 code implementation6 Sep 2017 Yang He, Margret Keuper, Bernt Schiele, Mario Fritz

In this paper, we present an approach for learning dilation parameters adaptively per channel, consistently improving semantic segmentation results on street-scene datasets like Cityscapes and Camvid.

Segmentation Semantic Segmentation

Advanced Steel Microstructural Classification by Deep Learning Methods

no code implementations20 Jun 2017 Seyed Majid Azimi, Dominik Britz, Michael Engstler, Mario Fritz, Frank Mücklich

In this work, we propose a Deep Learning method for microstructural classification in the examples of certain microstructural constituents of low carbon steel.

Classification General Classification

Visual Decoding of Targets During Visual Search From Human Eye Fixations

no code implementations19 Jun 2017 Hosnieh Sattar, Mario Fritz, Andreas Bulling

Such visual decoding is challenging for two reasons: 1) the search target only resides in the user's mind as a subjective visual pattern, and can most often not even be described verbally by the person, and 2) it is, as of yet, unclear if gaze fixations contain sufficient information for this task at all.

Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images

no code implementations ICCV 2017 Tribhuvanesh Orekondy, Bernt Schiele, Mario Fritz

Third, we propose models that predict user specific privacy score from images in order to enforce the users' privacy preferences.

Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective

no code implementations ICCV 2017 Seong Joon Oh, Mario Fritz, Bernt Schiele

We derive the optimal strategy for the user that assures an upper bound on the recognition rate independent of the recogniser's counter measure.

Person Recognition

Exploiting saliency for object segmentation from image level labels

no code implementations CVPR 2017 Seong Joon Oh, Rodrigo Benenson, Anna Khoreva, Zeynep Akata, Mario Fritz, Bernt Schiele

We show how to combine both information sources in order to recover 80% of the fully supervised performance - which is the new state of the art in weakly supervised training for pixel-wise semantic labelling.

Object Semantic Segmentation

Predicting the Category and Attributes of Visual Search Targets Using Deep Gaze Pooling

no code implementations27 Nov 2016 Hosnieh Sattar, Andreas Bulling, Mario Fritz

Predicting the target of visual search from eye fixation (gaze) data is a challenging problem with many applications in human-computer interaction.

Long-Term Image Boundary Prediction

no code implementations27 Nov 2016 Apratim Bhattacharyya, Mateusz Malinowski, Bernt Schiele, Mario Fritz

Boundary estimation in images and videos has been a very active topic of research, and organizing visual information into boundaries and segments is believed to be a corner stone of visual perception.

Tutorial on Answering Questions about Images with Deep Learning

1 code implementation4 Oct 2016 Mateusz Malinowski, Mario Fritz

Together with the development of more accurate methods in Computer Vision and Natural Language Understanding, holistic architectures that answer on questions about the content of real-world images have emerged.

Natural Language Understanding Visual Question Answering (VQA)

Visual Stability Prediction and Its Application to Manipulation

no code implementations15 Sep 2016 Wenbin Li, Aleš Leonardis, Mario Fritz

We present a learning-based approach based on simulated data that predicts stability of towers comprised of wooden blocks under different conditions and quantities related to the potential fall of the towers.

Mean Box Pooling: A Rich Image Representation and Output Embedding for the Visual Madlibs Task

no code implementations9 Aug 2016 Ashkan Mokarian, Mateusz Malinowski, Mario Fritz

We present Mean Box Pooling, a novel visual representation that pools over CNN representations of a large number, highly overlapping object proposals.

Faceless Person Recognition; Privacy Implications in Social Media

no code implementations28 Jul 2016 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

As we shift more of our lives into the virtual domain, the volume of data shared on the web keeps increasing and presents a threat to our privacy.

Person Recognition

Spatio-Temporal Image Boundary Extrapolation

no code implementations24 May 2016 Apratim Bhattacharyya, Mateusz Malinowski, Mario Fritz

Furthermore, we show long-term prediction of boundaries in situations where the motion is governed by the laws of physics.

Video Segmentation Video Semantic Segmentation

Ask Your Neurons: A Deep Learning Approach to Visual Question Answering

1 code implementation9 May 2016 Mateusz Malinowski, Marcus Rohrbach, Mario Fritz

By combining latest advances in image representation and natural language processing, we propose Ask Your Neurons, a scalable, jointly trained, end-to-end formulation to this problem.

Question Answering Visual Question Answering

VConv-DAE: Deep Volumetric Shape Learning Without Object Labels

1 code implementation13 Apr 2016 Abhishek Sharma, Oliver Grau, Mario Fritz

Prior work has shown encouraging results on problems ranging from shape completion to recognition.

Denoising Object

To Fall Or Not To Fall: A Visual Approach to Physical Stability Prediction

no code implementations31 Mar 2016 Wenbin Li, Seyedmajid Azimi, Aleš Leonardis, Mario Fritz

In this paper, we contrast a more traditional approach of taking a model-based route with explicit 3D representations and physical simulation by an end-to-end approach that directly predicts stability and related quantities from appearance.

Multi-Cue Zero-Shot Learning with Strong Supervision

no code implementations CVPR 2016 Zeynep Akata, Mateusz Malinowski, Mario Fritz, Bernt Schiele

A promising research direction is zero-shot learning, which does not require any training data to recognize new classes, but rather relies on some form of auxiliary information describing the new classes.

Attribute Retrieval +1

DeLight-Net: Decomposing Reflectance Maps into Specular Materials and Natural Illumination

no code implementations27 Mar 2016 Stamatios Georgoulis, Konstantinos Rematas, Tobias Ritschel, Mario Fritz, Luc van Gool, Tinne Tuytelaars

In this paper we are extracting surface reflectance and natural environmental illumination from a reflectance map, i. e. from a single 2D image of a sphere of one material under one illumination.

Contextual Media Retrieval Using Natural Language Queries

no code implementations16 Feb 2016 Sreyasi Nag Chowdhury, Mateusz Malinowski, Andreas Bulling, Mario Fritz

We show that our retrieval system can cope with this variability using personalisation through an online learning-based retrieval formulation.

Natural Language Queries Retrieval

Novel Views of Objects from a Single Image

no code implementations31 Jan 2016 Konstantinos Rematas, Chuong Nguyen, Tobias Ritschel, Mario Fritz, Tinne Tuytelaars

We propose a technique to use the structural information extracted from a 3D model that matches the image object in terms of viewpoint and shape.

Novel View Synthesis Object

Deep Reflectance Maps

no code implementations CVPR 2016 Konstantinos Rematas, Tobias Ritschel, Mario Fritz, Efstratios Gavves, Tinne Tuytelaars

Undoing the image formation process and therefore decomposing appearance into its intrinsic properties is a challenging task due to the under-constraint nature of this inverse problem.

Person Recognition in Personal Photo Collections

no code implementations ICCV 2015 Seong Joon Oh, Rodrigo Benenson, Mario Fritz, Bernt Schiele

Recognising persons in everyday photos presents major challenges (occluded faces, different clothing, locations, etc.)

Informativeness Person Recognition

GazeDPM: Early Integration of Gaze Information in Deformable Part Models

no code implementations21 May 2015 Iaroslav Shcherbatyi, Andreas Bulling, Mario Fritz

An increasing number of works explore collaborative human-computer systems in which human gaze is used to enhance computer vision systems.

Gaze Estimation object-detection +1

Ask Your Neurons: A Neural-based Approach to Answering Questions about Images

no code implementations ICCV 2015 Mateusz Malinowski, Marcus Rohrbach, Mario Fritz

In contrast to previous efforts, we are facing a multi-modal problem where the language output (answer) is conditioned on visual and natural language input (image and question).

Question Answering

See the Difference: Direct Pre-Image Reconstruction and Pose Estimation by Differentiating HOG

no code implementations ICCV 2015 Wei-Chen Chiu, Mario Fritz

The Histogram of Oriented Gradient (HOG) descriptor has led to many advances in computer vision over the last decade and is still part of many state of the art approaches.

Image Reconstruction Pose Estimation

Appearance-Based Gaze Estimation in the Wild

6 code implementations CVPR 2015 Xucong Zhang, Yusuke Sugano, Mario Fritz, Andreas Bulling

Appearance-based gaze estimation is believed to work well in real-world settings, but existing datasets have been collected under controlled laboratory conditions and methods have been not evaluated across multiple datasets.

Gaze Estimation

Prediction of Search Targets From Fixations in Open-World Settings

no code implementations CVPR 2015 Hosnieh Sattar, Sabine Müller, Mario Fritz, Andreas Bulling

Previous work on predicting the target of visual search from human fixations only considered closed-world settings in which training labels are available and predictions are performed for a known set of potential targets.

Hard to Cheat: A Turing Test based on Answering Questions about Images

no code implementations14 Jan 2015 Mateusz Malinowski, Mario Fritz

Progress in language and image understanding by machines has sparkled the interest of the research community in more open-ended, holistic tasks, and refueled an old AI dream of building intelligent machines.

Question Answering

A Pooling Approach to Modelling Spatial Relations for Image Retrieval and Annotation

no code implementations19 Nov 2014 Mateusz Malinowski, Mario Fritz

Over the last two decades we have witnessed strong progress on modeling visual object classes, scenes and attributes that have significantly contributed to automated image understanding.

Image Retrieval Retrieval

Towards a Visual Turing Challenge

no code implementations29 Oct 2014 Mateusz Malinowski, Mario Fritz

As language and visual understanding by machines progresses rapidly, we are observing an increasing interest in holistic architectures that tightly interlink both modalities in a joint learning and inference process.

Question Answering

A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input

no code implementations NeurIPS 2014 Mateusz Malinowski, Mario Fritz

We propose a method for automatically answering questions about images by bringing together recent advances from natural language processing and computer vision.

Question Answering

Learning Multi-Scale Representations for Material Classification

no code implementations13 Aug 2014 Wenbin Li, Mario Fritz

The recent progress in sparse coding and deep learning has made unsupervised feature learning methods a strong competitor to hand-crafted descriptors.

Classification General Classification +3

Image-based Synthesis and Re-Synthesis of Viewpoints Guided by 3D Models

no code implementations CVPR 2014 Konstantinos Rematas, Tobias Ritschel, Mario Fritz, Tinne Tuytelaars

We propose a technique to use the structural information extracted from a set of 3D models of an object class to improve novel-view synthesis for images showing unknown instances of this class.

Novel View Synthesis Position +1

Anytime Recognition of Objects and Scenes

no code implementations CVPR 2014 Sergey Karayev, Mario Fritz, Trevor Darrell

On suitable datasets, we can incorporate a semantic back-off strategy that gives maximally specific predictions for a desired level of accuracy; this provides a new view on the time course of human visual perception.

General Classification Object Recognition

Ubic: Bridging the gap between digital cryptography and the physical world

no code implementations6 Mar 2014 Mark Simkin, Dominique Schroeder, Andreas Bulling, Mario Fritz

We describe Ubic, a framework that allows users to bridge the gap between digital cryptography and the physical world.

Multi-class Video Co-segmentation with a Generative Multi-video Model

no code implementations CVPR 2013 Wei-Chen Chiu, Mario Fritz

This is a clear mismatch to the challenges that we are facing with videos from online resources or consumer videos.

Segmentation Video Segmentation +1

Learnable Pooling Regions for Image Classification

no code implementations15 Jan 2013 Mateusz Malinowski, Mario Fritz

Biologically inspired, from the early HMAX model to Spatial Pyramid Matching, pooling has played an important role in visual recognition pipelines.

Classification General Classification +2

Size Matters: Metric Visual Search Constraints from Monocular Metadata

no code implementations NeurIPS 2010 Mario Fritz, Kate Saenko, Trevor Darrell

Metric constraints are known to be highly discriminative for many objects, but if training is limited to data captured from a particular 3-D sensor the quantity of training data may be severly limited.

An Additive Latent Feature Model for Transparent Object Recognition

no code implementations NeurIPS 2009 Mario Fritz, Gary Bradski, Sergey Karayev, Trevor Darrell, Michael J. Black

The appearance of a transparent patch is determined in part by the refraction of a background pattern through a transparent medium: the energy from the background usually dominates the patch appearance.

Object Object Recognition +2

Cannot find the paper you are looking for? You can Submit a new open access paper.