Search Results for author:
Found 123 papers, 42 papers with code
Towards Automated Testing and Robustification by Semantic Adversarial Data Generation
Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector.
Client-specific Property Inference against Secure Aggregation in Federated Learning
We formulate an optimization problem across different rounds in order to infer a tested property of every client from the output of the linear models, for example, whether they have a specific sample in their training data (membership inference) or whether they misbehave and attempt to degrade the performance of the common model by poisoning attacks.
More than you've asked for: A Comprehensive Analysis of Novel Prompt Injection Threats to Application-Integrated Large Language Models
In such attacks, an adversary can prompt the LLM to produce malicious content or override the original instructions and the employed filtering schemes.
Data Forensics in Diffusion Models: A Systematic Analysis of Membership Privacy
In recent years, diffusion models have achieved tremendous success in the field of image generation, becoming the stateof-the-art technology for AI-based image processing applications.
Systematically Finding Security Vulnerabilities in Black-Box Code Generation Models
Given the wide use of these models in the daily workflow of developers, it is crucial to study the security aspects of these models systematically.
Fed-GLOSS-DP: Federated, Global Learning using Synthetic Sets with Record Level Differential Privacy
In our approach, the server recovers an approximation of the global loss landscape in a local neighborhood based on synthetic samples received from the clients.
Holistically Explainable Vision Transformers
Transformers increasingly dominate the machine learning landscape across many tasks and domains, which increases the importance for understanding their outputs.
Private Set Generation with Discriminative Information
Differentially private data generation techniques have become a promising solution to the data privacy challenge -- it enables sharing of data while complying with rigorous privacy guarantees, which is essential for scientific progress in sensitive domains.
SimSCOOD: Systematic Analysis of Out-of-Distribution Behavior of Source Code Models
While large code datasets have become available in recent years, acquiring representative training data with full coverage of general code distribution remains challenging due to the compositional nature of code and the complexity of software.
UnGANable: Defending Against GAN-based Face Manipulation
Extensive experiments on four popular GAN models trained on two benchmark face datasets show that UnGANable achieves remarkable effectiveness and utility performance, and outperforms multiple baseline methods.
Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systems
In this work, we assume an adversary that automatically tampers with the online evidence in order to disrupt the fact-checking model via camouflaging the relevant evidence or planting a misleading one.
RelaxLoss: Defending Membership Inference Attacks without Losing Utility
As a long-term threat to the privacy of training data, membership inference attacks (MIAs) emerge ubiquitously in machine learning models.
B-cos Networks: Alignment is All We Need for Interpretability
We present a new direction for increasing the interpretability of deep neural networks (DNNs) by promoting weight-input alignment during training.
Practical Challenges in Differentially-Private Federated Survival Analysis of Medical Data
Survival analysis or time-to-event analysis aims to model and predict the time it takes for an event of interest to happen in a population or an individual.
Open-Domain, Content-based, Multi-modal Fact-checking of Out-of-Context Images via Online Resources
Misinformation is now a major problem due to its potential high risks to our core democratic and societal values and orders.
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training
Federated learning is a powerful distributed learning scheme that allows numerous edge devices to collaboratively train a model without sharing their data.
Optimising for Interpretability: Convolutional Dynamic Alignment Networks
As a result, CoDA Nets model the classification prediction through a series of input-dependent linear transformations, allowing for linear decomposition of the output into individual input contributions.
Availability Attacks Against Neural Network Certifiers Based on Backdoors
To achieve reliable, robust, and safe AI systems it is important to implement fallback strategies when AI predictions cannot be trusted.
Euro-PVI: Pedestrian Vehicle Interactions in Dense Urban Centers
In this work, we propose Euro-PVI, a dataset of pedestrian and bicyclist trajectories.
Ranked #1 on
Pedestrian Trajectory Prediction
on Euro-PVI
Beyond the Spectrum: Detecting Deepfakes via Re-Synthesis
The rapid advances in deep generative models over the past years have led to highly {realistic media, known as deepfakes,} that are commonly indistinguishable from real to human eyes.
Dual Contrastive Loss and Attention for GANs
Lastly, we study different attention architectures in the discriminator, and propose a reference attention mechanism.
Convolutional Dynamic Alignment Networks for Interpretable Classifications
Given the alignment of the DAUs, the resulting contribution maps align with discriminative input patterns.
"What's in the box?!": Deflecting Adversarial Attacks by Randomly Deploying Adversarially-Disjoint Models
Machine learning models are now widely deployed in real-world applications.
ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Models
As a result, we lack a comprehensive picture of the risks caused by the attacks, e. g., the different scenarios they can be applied to, the common factors that influence their performance, the relationship among them, or the effectiveness of possible defenses.
Responsible Disclosure of Generative Models Using Scalable Fingerprinting
Over the past years, deep generative models have achieved a new level of performance.
CosSGD: Communication-Efficient Federated Learning with a Simple Cosine-Based Quantization
Despite notable progress in gradient compression, the existing quantization methods require further improvement when low-bits compression is applied, especially the overall systems often degenerate a lot when quantization are applied in double directions to compress model weights and gradients.
Hijack-GAN: Unintended-Use of Pretrained, Black-Box GANs
While Generative Adversarial Networks (GANs) show increasing performance and the level of realism is becoming indistinguishable from natural images, this also comes with high demands on data and computation.
SampleFix: Learning to Correct Programs by Efficient Sampling of Diverse Fixes
Therefore, we propose a deep generative model to automatically correct programming errors by learning a distribution over potential fixes.
Haar Wavelet based Block Autoregressive Flows for Trajectories
This yields an exact inference method that models trajectories at different spatio-temporal resolutions in a hierarchical manner.
Synthetic Convolutional Features for Improved Semantic Segmentation
Recently, learning-based image synthesis has enabled to generate high-resolution images, either applying popular adversarial training or a powerful perceptual loss.
Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding
In this paper, we study natural language watermarking as a defense to help better mark and trace the provenance of text.
Sampling Attacks: Amplification of Membership Inference Attacks by Repeated Queries
Our work consists of two sides: We introduce sampling attack, a novel membership inference technique that unlike other standard membership adversaries is able to work under severe restriction of no access to scores of the victim model.
Artificial Fingerprinting for Generative Models: Rooting Deepfake Attribution in Training Data
Thus, we seek a proactive and sustainable solution on deepfake detection, that is agnostic to the evolution of generative models, by introducing artificial fingerprints into the models.
IReEn: Reverse-Engineering of Black-Box Functions via Iterative Neural Program Synthesis
In this work, we investigate the problem of revealing the functionality of a black-box agent.
GS-WGAN: A Gradient-Sanitized Approach for Learning Differentially Private Generators
The wide-spread availability of rich data has fueled the growth of machine learning applications in numerous domains.
InfoScrub: Towards Attribute Privacy by Targeted Obfuscation
Personal photos of individuals when shared online, apart from exhibiting a myriad of memorable details, also reveals a wide range of private information and potentially entails privacy risks (e. g., online harassment, tracking).
Normalizing Flows with Multi-Scale Autoregressive Priors
Flow-based generative models are an important class of exact inference models that admit efficient inference and sampling for image synthesis.
Inclusive GAN: Improving Data and Minority Coverage in Generative Models
Generative Adversarial Networks (GANs) have brought about rapid progress towards generating photorealistic images.
Long-Tailed Recognition Using Class-Balanced Experts
Deep learning enables impressive performance in image recognition using large-scale artificially-balanced datasets.
Ranked #17 on
Long-tail Learning
on Places-LT
Everything About You: A Multimodal Approach towards Friendship Inference in Online Social Networks
Most previous works in privacy of Online Social Networks (OSN) focus on a restricted scenario of using one type of information to infer another type of information or using only static profile data such as username, profile picture or home location.
Social and Information Networks
Segmentations-Leak: Membership Inference Attacks and Defenses in Semantic Image Segmentation
Today's success of state of the art methods for semantic segmentation is driven by large datasets.
Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing
Despite significant success in Visual Question Answering (VQA), VQA models have been shown to be notoriously brittle to linguistic variations in the questions.
"Best-of-Many-Samples" Distribution Matching
Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.
``"Best-of-Many-Samples" Distribution Matching
Recent works have proposed hybrid VAE-GAN frameworks which integrate a GAN-based synthetic likelihood to the VAE objective to address both the mode collapse and sample quality issues, with limited success.
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
In addition, we propose the first generic attack model that can be instantiated in a large range of settings and is applicable to various kinds of deep generative models.
VisualPhishNet: Zero-Day Phishing Website Detection by Visual Similarity
Phishing websites are still a major threat in today's Internet ecosystem.
Conditional Flow Variational Autoencoders for Structured Sequence Prediction
Prediction of future states of the environment and interacting agents is a key competence required for autonomous agents to operate successfully in the real world.
Ranked #9 on
Trajectory Prediction
on Stanford Drone
Interpretability Beyond Classification Output: Semantic Bottleneck Networks
Additionally we show how the activations of the SB-Layer can be used for both the interpretation of failure cases of the network as well as for confidence prediction of the resulting output.
Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attacks
We find such passive defenses ineffective against DNN stealing attacks.
SampleFix: Learning to Generate Functionally Diverse Fixes
Therefore, we propose a generative model that learns a distribution over potential fixes.
Ranked #3 on
Program Repair
on DeepFix
Body Shape Privacy in Images: Understanding Privacy and Preventing Automatic Shape Extraction
Modern approaches to pose and body shape estimation have recently achieved strong performance even under challenging real-world conditions.
Updates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning
As data generation is a continuous process, this leads to ML model owners updating their models frequently with newly-collected data in an online learning scenario.
Not Using the Car to See the Sidewalk: Quantifying and Controlling the Effects of Context in Classification and Segmentation
We propose a method to quantify the sensitivity of black-box vision models to visual context by editing images to remove selected objects and measuring the response of the target models.
Knockoff Nets: Stealing Functionality of Black-Box Models
We formulate model functionality stealing as a two-step approach: (i) querying a set of input images to the blackbox model to obtain predictions; and (ii) training a "knockoff" with queried image-prediction pairs.
Attributing Fake Images to GANs: Learning and Analyzing GAN Fingerprints
Our experiments show that (1) GANs carry distinct model fingerprints and leave stable fingerprints in their generated images, which support image attribution; (2) even minor differences in GAN training can result in different fingerprints, which enables fine-grained model authentication; (3) fingerprints persist across different image frequencies and patches and are not biased by GAN artifacts; (4) fingerprint finetuning is effective in immunizing against five types of adversarial image perturbations; and (5) comparisons also show our learned fingerprints consistently outperform several baselines in a variety of setups.
Bayesian Prediction of Future Street Scenes using Synthetic Likelihoods
For autonomous agents to successfully operate in the real world, the ability to anticipate future scene states is a key competence.
Diverse Conditional Image Generation by Stochastic Regression with Latent Drop-Out Codes
Recent advances in Deep Learning and probabilistic modeling have led to strong improvements in generative models for images.
MLCapsule: Guarded Offline Deployment of Machine Learning as a Service
In this paper, we propose MLCapsule, a guarded offline deployment of machine learning as a service.
Fashion is Taking Shape: Understanding Clothing Preference Based on Body Shape From Online Sources
To study the correlation between clothing garments and body shape, we collected a new dataset (Fashion Takes Shape), which includes images of users with clothing category annotations.
Accurate and Diverse Sampling of Sequences based on a "Best of Many" Sample Objective
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.
Ranked #6 on
Human Pose Forecasting
on Human3.6M
(ADE metric)
Bayesian Prediction of Future Street Scenes through Importance Sampling based Optimization
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.
Adversarial Scene Editing: Automatic Object Removal from Weak Supervision
While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets.
ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models
In addition, we propose the first effective defense mechanisms against such broader class of membership inference attacks that maintain a high level of utility of the ML model.
Accurate and Diverse Sampling of Sequences Based on a âBest of Manyâ Sample Objective
For autonomous agents to successfully operate in the real world, anticipation of future events and states of their environment is a key competence.
Sequential Attacks on Agents for Long-Term Adversarial Goals
As a result of our attack, the victim agent is misguided to optimise for the adversarial reward over time.
Gradient-Leaks: Understanding and Controlling Deanonymization in Federated Learning
At the core of FL is a network of anonymous user devices sharing training information (model parameter updates) computed locally on personal data.
A Hybrid Model for Identity Obfuscation by Face Replacement
As more and more personal photos are shared and tagged in social media, avoiding privacy risks such as unintended recognition becomes increasingly challenging.
Deep Appearance Maps
Second, we demonstrate how another network can be used to map from an image or video frames to a DAM network to reproduce this appearance, without using a lengthy optimization such as stochastic gradient descent (learning-to-learn).
Disentangled Person Image Generation
Generating novel, yet realistic, images of persons is a challenging task due to the complex interplay between the different image factors, such as the foreground, background and pose information.
Ranked #2 on
Gesture-to-Gesture Translation
on Senz3D
Connecting Pixels to Privacy and Utility: Automatic Redaction of Private Information in Images
Images convey a broad spectrum of personal information.
Natural and Effective Obfuscation by Head Inpainting
As more and more personal photos are shared online, being able to obfuscate identities in such photos is becoming a necessity for privacy protection.
MPIIGaze: Real-World Dataset and Deep Appearance-Based Gaze Estimation
Second, we present an extensive evaluation of state-of-the-art gaze estimation methods on three current datasets, including MPIIGaze.
Long-Term On-Board Prediction of People in Traffic Scenes under Uncertainty
Our experimental results show that it is indeed possible to predict people trajectories at the desired time horizons and that our uncertainty estimates are informative of the prediction error.
Ranked #5 on
Trajectory Prediction
on JAAD
Towards Reverse-Engineering Black-Box Neural Networks
On the one hand, our work exposes the vulnerability of black-box neural networks to different types of attacks -- we show that the revealed internal information helps generate more effective adversarial examples against the black box model.
$A^{4}NT$: Author Attribute Anonymity by Adversarial Training of Neural Machine Translation
In this paper, we propose an automatic method, called Adversarial Author Attribute Anonymity Neural Translation ($A^4NT$), to combat such text-based adversaries.
Acquiring Target Stacking Skills by Goal-Parameterized Deep Reinforcement Learning
We created a synthetic block stacking environment with physics simulation in which the agent can learn a policy end-to-end through trial and error.
Person Recognition in Personal Photo Collections
Person recognition in social media photos sets new challenges for computer vision, including non-cooperative subjects (e. g. backward viewpoints, unusual poses) and great changes in appearance.
Learning Dilation Factors for Semantic Segmentation of Street Scenes
In this paper, we present an approach for learning dilation parameters adaptively per channel, consistently improving semantic segmentation results on street-scene datasets like Cityscapes and Camvid.
Advanced Steel Microstructural Classification by Deep Learning Methods
In this work, we propose a Deep Learning method for microstructural classification in the examples of certain microstructural constituents of low carbon steel.
Visual Decoding of Targets During Visual Search From Human Eye Fixations
Such visual decoding is challenging for two reasons: 1) the search target only resides in the user's mind as a subjective visual pattern, and can most often not even be described verbally by the person, and 2) it is, as of yet, unclear if gaze fixations contain sufficient information for this task at all.
A Domain Based Approach to Social Relation Recognition
Social relations are the foundation of human daily life.
Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images
Third, we propose models that predict user specific privacy score from images in order to enforce the users' privacy preferences.
Speaking the Same Language: Matching Machine to Human Captions by Adversarial Training
While strong progress has been made in image captioning over the last years, machine and human captions are still quite distinct.
Adversarial Image Perturbation for Privacy Protection -- A Game Theory Perspective
We derive the optimal strategy for the user that assures an upper bound on the recognition rate independent of the recogniser's counter measure.
Exploiting saliency for object segmentation from image level labels
We show how to combine both information sources in order to recover 80% of the fully supervised performance - which is the new state of the art in weakly supervised training for pixel-wise semantic labelling.
Ranked #25 on
Semantic Segmentation
on PASCAL VOC 2012 val
What Is Around The Camera?
How much does a single image reveal about the environment it was taken in?
It's Written All Over Your Face: Full-Face Appearance-Based Gaze Estimation
Eye gaze is an important non-verbal cue for human affect analysis.
Long-Term Image Boundary Prediction
Boundary estimation in images and videos has been a very active topic of research, and organizing visual information into boundaries and segments is believed to be a corner stone of visual perception.
Predicting the Category and Attributes of Visual Search Targets Using Deep Gaze Pooling
Predicting the target of visual search from eye fixation (gaze) data is a challenging problem with many applications in human-computer interaction.
Tutorial on Answering Questions about Images with Deep Learning
Together with the development of more accurate methods in Computer Vision and Natural Language Understanding, holistic architectures that answer on questions about the content of real-world images have emerged.
Visual Stability Prediction and Its Application to Manipulation
We present a learning-based approach based on simulated data that predicts stability of towers comprised of wooden blocks under different conditions and quantities related to the potential fall of the towers.
Towards Segmenting Consumer Stereo Videos: Benchmark, Baselines and Ensembles
Are we ready to segment consumer stereo videos?
Mean Box Pooling: A Rich Image Representation and Output Embedding for the Visual Madlibs Task
We present Mean Box Pooling, a novel visual representation that pools over CNN representations of a large number, highly overlapping object proposals.
Faceless Person Recognition; Privacy Implications in Social Media
As we shift more of our lives into the virtual domain, the volume of data shared on the web keeps increasing and presents a threat to our privacy.
Spatio-Temporal Image Boundary Extrapolation
Furthermore, we show long-term prediction of boundaries in situations where the motion is governed by the laws of physics.
Ask Your Neurons: A Deep Learning Approach to Visual Question Answering
By combining latest advances in image representation and natural language processing, we propose Ask Your Neurons, a scalable, jointly trained, end-to-end formulation to this problem.
VConv-DAE: Deep Volumetric Shape Learning Without Object Labels
Prior work has shown encouraging results on problems ranging from shape completion to recognition.
STD2P: RGBD Semantic Segmentation Using Spatio-Temporal Data-Driven Pooling
The proposed network produces a high quality segmentation of a single image by leveraging information from additional views of the same scene.
Ranked #59 on
Semantic Segmentation
on NYU Depth v2
To Fall Or Not To Fall: A Visual Approach to Physical Stability Prediction
In this paper, we contrast a more traditional approach of taking a model-based route with explicit 3D representations and physical simulation by an end-to-end approach that directly predicts stability and related quantities from appearance.
Multi-Cue Zero-Shot Learning with Strong Supervision
A promising research direction is zero-shot learning, which does not require any training data to recognize new classes, but rather relies on some form of auxiliary information describing the new classes.
DeLight-Net: Decomposing Reflectance Maps into Specular Materials and Natural Illumination
In this paper we are extracting surface reflectance and natural environmental illumination from a reflectance map, i. e. from a single 2D image of a sphere of one material under one illumination.
Contextual Media Retrieval Using Natural Language Queries
We show that our retrieval system can cope with this variability using personalisation through an online learning-based retrieval formulation.
Novel Views of Objects from a Single Image
We propose a technique to use the structural information extracted from a 3D model that matches the image object in terms of viewpoint and shape.
Deep Reflectance Maps
Undoing the image formation process and therefore decomposing appearance into its intrinsic properties is a challenging task due to the under-constraint nature of this inverse problem.
Person Recognition in Personal Photo Collections
Recognising persons in everyday photos presents major challenges (occluded faces, different clothing, locations, etc.)
GazeDPM: Early Integration of Gaze Information in Deformable Part Models
An increasing number of works explore collaborative human-computer systems in which human gaze is used to enhance computer vision systems.
Ask Your Neurons: A Neural-based Approach to Answering Questions about Images
In contrast to previous efforts, we are facing a multi-modal problem where the language output (answer) is conditioned on visual and natural language input (image and question).
See the Difference: Direct Pre-Image Reconstruction and Pose Estimation by Differentiating HOG
The Histogram of Oriented Gradient (HOG) descriptor has led to many advances in computer vision over the last decade and is still part of many state of the art approaches.
Appearance-Based Gaze Estimation in the Wild
Appearance-based gaze estimation is believed to work well in real-world settings, but existing datasets have been collected under controlled laboratory conditions and methods have been not evaluated across multiple datasets.
Prediction of Search Targets From Fixations in Open-World Settings
Previous work on predicting the target of visual search from human fixations only considered closed-world settings in which training labels are available and predictions are performed for a known set of potential targets.
Hard to Cheat: A Turing Test based on Answering Questions about Images
Progress in language and image understanding by machines has sparkled the interest of the research community in more open-ended, holistic tasks, and refueled an old AI dream of building intelligent machines.
A Pooling Approach to Modelling Spatial Relations for Image Retrieval and Annotation
Over the last two decades we have witnessed strong progress on modeling visual object classes, scenes and attributes that have significantly contributed to automated image understanding.
Towards a Visual Turing Challenge
As language and visual understanding by machines progresses rapidly, we are observing an increasing interest in holistic architectures that tightly interlink both modalities in a joint learning and inference process.
A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input
We propose a method for automatically answering questions about images by bringing together recent advances from natural language processing and computer vision.
Learning Multi-Scale Representations for Material Classification
The recent progress in sparse coding and deep learning has made unsupervised feature learning methods a strong competitor to hand-crafted descriptors.
Anytime Recognition of Objects and Scenes
On suitable datasets, we can incorporate a semantic back-off strategy that gives maximally specific predictions for a desired level of accuracy; this provides a new view on the time course of human visual perception.
Image-based Synthesis and Re-Synthesis of Viewpoints Guided by 3D Models
We propose a technique to use the structural information extracted from a set of 3D models of an object class to improve novel-view synthesis for images showing unknown instances of this class.
Ubic: Bridging the gap between digital cryptography and the physical world
We describe Ubic, a framework that allows users to bridge the gap between digital cryptography and the physical world.
Multi-class Video Co-segmentation with a Generative Multi-video Model
This is a clear mismatch to the challenges that we are facing with videos from online resources or consumer videos.
Learnable Pooling Regions for Image Classification
Biologically inspired, from the early HMAX model to Spatial Pyramid Matching, pooling has played an important role in visual recognition pipelines.
Timely Object Recognition
On the timeliness measure, our method obtains at least $11\%$ better performance.
Size Matters: Metric Visual Search Constraints from Monocular Metadata
Metric constraints are known to be highly discriminative for many objects, but if training is limited to data captured from a particular 3-D sensor the quantity of training data may be severly limited.
An Additive Latent Feature Model for Transparent Object Recognition
The appearance of a transparent patch is determined in part by the refraction of a background pattern through a transparent medium: the energy from the background usually dominates the patch appearance.
