Search Results for author:
Found 94 papers, 54 papers with code
A Neural Algorithm of Artistic Style
In fine art, especially painting, humans have mastered the skill to create unique visual experiences through composing a complex interplay between the content and style of an image.
Markerless tracking of user-defined features with deep learning
Quantifying behavior is crucial for many applications in neuroscience.
Pretraining boosts out-of-domain robustness for pose estimation
Neural networks are highly effective tools for pose estimation.
Ranked #1 on
Animal Pose Estimation
on Horse-10
ImageNet performance correlates with pose estimation robustness and generalization on out-of-domain data
Neural networks are highly effective tools for pose estimation.
Preserving Color in Neural Artistic Style Transfer
This note presents an extension to the neural artistic style transfer algorithm (Gatys et al.).
Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNet
Deep Neural Networks (DNNs) excel on many complex perceptual tasks but it has proven notoriously difficult to understand how they reach their decisions.
Foolbox: A Python toolbox to benchmark the robustness of machine learning models
Foolbox is a new Python package to generate such adversarial perturbations and to quantify and compare the robustness of machine learning models.
Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models
Such decision-based attacks are (1) applicable to real-world black-box models such as autonomous cars, (2) need less knowledge and are easier to apply than transfer-based attacks and (3) are more robust to simple defences than gradient- or score-based attacks.
Texture Synthesis Using Convolutional Neural Networks
Here we introduce a new model of natural textures based on the feature spaces of convolutional neural networks optimised for object recognition.
A simple way to make neural networks robust against diverse image corruptions
The human visual system is remarkably robust against a wide range of naturally occurring variations and corruptions like rain or snow.
Controlling Perceptual Factors in Neural Style Transfer
Neural Style Transfer has shown very exciting results enabling new forms of image manipulation.
Generalisation in humans and deep neural networks
We compare the robustness of humans and current convolutional deep neural networks (DNNs) on object recognition under twelve different types of image degradations.
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes.
Ranked #1 on
Out-of-Distribution Generalization
on ImageNet-W
EagerPy: Writing Code That Works Natively with PyTorch, TensorFlow, JAX, and NumPy
EagerPy is a Python framework that lets you write code that automatically works natively with PyTorch, TensorFlow, JAX, and NumPy.
Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming
The ability to detect objects regardless of image distortions or weather conditions is crucial for real-world applications of deep learning like autonomous driving.
Ranked #1 on
Robust Object Detection
on MS COCO
One-Shot Instance Segmentation
We demonstrate empirical results on MS Coco highlighting challenges of the one-shot setting: while transferring knowledge about instance segmentation to novel object categories works very well, targeting the detection network towards the reference category appears to be more difficult.
Ranked #1 on
One-Shot Instance Segmentation
on MS COCO
Partial success in closing the gap between human and machine vision
The longstanding distortion robustness gap between humans and CNNs is closing, with the best models now exceeding human feedforward performance on most of the investigated OOD datasets.
Image Style Transfer Using Convolutional Neural Networks
Rendering the semantic content of an image in different styles is a difficult image processing task.
Improving robustness against common corruptions by covariate shift adaptation
With the more robust DeepAugment+AugMix model, we improve the state of the art achieved by a ResNet50 model up to date from 53. 6% mCE to 45. 4% mCE.
Ranked #4 on
Unsupervised Domain Adaptation
on ImageNet-R
If your data distribution shifts, use self-learning
We demonstrate that self-learning techniques like entropy minimization and pseudo-labeling are simple and effective at improving performance of a deployed computer vision model under systematic domain shifts.
Ranked #1 on
Unsupervised Domain Adaptation
on ImageNet-A
(using extra training data)
Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet
Recent results suggest that state-of-the-art saliency models perform far from optimal in predicting fixations.
DeepGaze IIE: Calibrated prediction in and out-of-domain for state-of-the-art saliency modeling
Since 2014 transfer learning has become the key driver for the improvement of spatial saliency prediction; however, with stagnant progress in the last 3-5 years.
Shortcut Learning in Deep Neural Networks
Deep learning has triggered the current rise of artificial intelligence and is the workhorse of today's machine intelligence.
Contrastive Learning Inverts the Data Generating Process
Contrastive learning has recently seen tremendous success in self-supervised learning.
Ranked #1 on
Disentanglement
on KITTI-Masks
Towards Nonlinear Disentanglement in Natural Data with Temporal Sparse Coding
We construct an unsupervised learning model that achieves nonlinear disentanglement of underlying factors of variation in naturalistic videos.
Ranked #1 on
Disentanglement
on Natural Sprites
Towards the first adversarially robust neural network model on MNIST
Despite much effort, deep neural networks remain highly susceptible to tiny input perturbations and even for MNIST, one of the most common toy datasets in computer vision, no neural network model exists for which adversarial perturbations are large and make semantic sense to humans.
One-Shot Segmentation in Clutter
We tackle the problem of one-shot segmentation: finding and segmenting a previously unseen object in a cluttered scene based on a single instruction example.
Ranked #1 on
One-Shot Segmentation
on Cluttered Omniglot
Trace your sources in large-scale data: one ring to find them all
An important preprocessing step in most data analysis pipelines aims to extract a small set of sources that explain most of the data.
One-shot Texture Segmentation
We introduce one-shot texture segmentation: the task of segmenting an input image containing multiple textures given a patch of a reference texture.
Adversarial Vision Challenge
The NIPS 2018 Adversarial Vision Challenge is a competition to facilitate measurable progress towards robust machine vision models and more generally applicable adversarial attacks.
Benchmarking Unsupervised Object Representations for Video Sequences
Perceiving the world in terms of objects and tracking them through time is a crucial prerequisite for reasoning and scene understanding.
Comparing deep neural networks against humans: object recognition when the signal gets weaker
In addition, we find progressively diverging classification error-patterns between humans and DNNs when the signal gets weaker, indicating that there may still be marked differences in the way humans and current DNNs perform visual object recognition.
Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
An important component for generalization in machine learning is to uncover underlying latent factors of variation as well as the mechanism through which each factor acts in the world.
Neural system identification for large populations separating “what” and “where”
Traditional methods for neural system identification do not capitalize on this separation of “what” and “where”.
No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance
Web-crawled pretraining datasets underlie the impressive "zero-shot" evaluation performance of multimodal models, such as CLIP for classification/retrieval and Stable-Diffusion for image generation.
RDumb: A simple approach that questions our progress in continual test-time adaptation
Test-Time Adaptation (TTA) allows to update pre-trained models to changing data distributions at deployment time.
Signatures of criticality arise in simple neural population models with correlations
Support for this notion has come from a series of studies which identified statistical signatures of criticality in the ensemble activity of retinal ganglion cells.
Neurons and Cognition
A rotation-equivariant convolutional neural network model of primary visual cortex
We present a framework to identify common features independent of individual neurons' orientation selectivity by using a rotation-equivariant convolutional neural network, which automatically extracts every feature at multiple different orientations.
Exemplary Natural Images Explain CNN Activations Better than State-of-the-Art Feature Visualization
Even if only a single reference image is given, synthetic images provide less information than natural images ($65\pm5\%$ vs. $73\pm4\%$).
Visual Data-Type Understanding does not emerge from Scaling Vision-Language Models
This finding points to a blind spot in current frontier VLMs: they excel in recognizing semantic content but fail to acquire an understanding of visual data-types through scaling.
Five Points to Check when Comparing Visual Perception in Humans and Machines
In the second case study, we highlight the difference between necessary and sufficient mechanisms in visual reasoning tasks.
How Well do Feature Visualizations Support Causal Understanding of CNN Activations?
A precise understanding of why units in an artificial network respond to certain stimuli would constitute a big step towards explainable artificial intelligence.
A note on the evaluation of generative models
In particular, we show that three of the currently most commonly used criteria---average log-likelihood, Parzen window estimates, and visual fidelity of samples---are largely independent of each other when the data is high-dimensional.
Accurate, reliable and fast robustness evaluation
We here develop a new set of gradient-based adversarial attacks which (a) are more reliable in the face of gradient-masking than other gradient-based attacks, (b) perform better and are more query efficient than current state-of-the-art gradient-based attacks, (c) can be flexibly adapted to a wide range of adversarial criteria and (d) require virtually no hyperparameter tuning.
Diverse feature visualizations reveal invariances in early layers of deep neural networks
Here we propose a method to discover invariances in the responses of hidden layer units of deep neural networks.
Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity?
Foundation models like CLIP are trained on hundreds of millions of samples and effortlessly generalize to new tasks and inputs.
Modulated Neural ODEs
Neural ordinary differential equations (NODEs) have been proven useful for learning non-linear dynamics of arbitrary trajectories.
Lifelong Benchmarks: Efficient Model Evaluation in an Era of Rapid Progress
However, with repeated testing, the risk of overfitting grows as algorithms over-exploit benchmark idiosyncrasies.
Fast Differentiable Clipping-Aware Normalization and Rescaling
When the rescaled perturbation $\eta \vec{\delta}$ is added to a starting point $\vec{x} \in D$ (where $D$ is the data domain, e. g. $D = [0, 1]^n$), the resulting vector $\vec{v} = \vec{x} + \eta \vec{\delta}$ will in general not be in $D$.
Unsupervised Object Learning via Common Fate
Learning generative object models from unlabelled videos is a long standing problem and required for causal scene modeling.
Visual cognition in multimodal large language models
A chief goal of artificial intelligence is to build machines that think like people.
Infinite dSprites for Disentangled Continual Learning: Separating Memory Edits from Generalization
In a simple setting with direct supervision on the generative factors, we show how learning class-agnostic transformations offers a way to circumvent catastrophic forgetting and improve classification accuracy over time.
System Identification with Biophysical Constraints: A Circuit Model of the Inner Retina
Here, we present a computational model of temporal processing in the inner retina, including inhibitory feedback circuits and realistic synaptic release mechanisms.
Neural system identification for large populations separating "what" and "where"
Our network scales well to thousands of neurons and short recordings and can be trained end-to-end.
Guiding human gaze with convolutional neural networks
Thus, manipulating fixation patterns to guide human attention is an exciting challenge in digital image processing.
Saliency Benchmarking Made Easy: Separating Models, Maps and Metrics
Here we show that no single saliency map can perform well under all metrics.
Comment on "Biologically inspired protection of deep networks from adversarial attacks"
A recent paper suggests that Deep Neural Networks can be protected from gradient-based adversarial perturbations by driving the network activations into a highly saturated regime.
Synthesising Dynamic Textures using Convolutional Neural Networks
Here we present a parametric model for dynamic textures.
DeepGaze II: Reading fixations from deep features trained on object recognition
Here we present DeepGaze II, a model that predicts where people look in images.
Texture Synthesis Using Shallow Convolutional Networks with Random Filters
The current state of the art in parametric texture synthesis relies on the multi-layer feature space of deep CNNs that were trained on natural images.
Inference and Mixture Modeling with the Elliptical Gamma Distribution
We study modeling and inference with the Elliptical Gamma Distribution (EGD).
Generative Image Modeling Using Spatial LSTMs
Modeling the distribution of natural images is challenging, partly because of strong statistical dependencies which can extend over hundreds of pixels.
Ranked #59 on
Image Generation
on CIFAR-10
(bits/dimension metric)
A Generative Model of Natural Texture Surrogates
In order to model this variability, we first applied the parametric texture algorithm of Portilla and Simoncelli to image patches of 64X64 pixels in a large database of natural images such that each image patch is then described by 655 texture parameters which specify certain statistics, such as variances and covariances of wavelet coefficients or coefficient magnitudes within that patch.
Supervised learning sets benchmark for robust spike detection from calcium imaging signals
A fundamental challenge in calcium imaging has been to infer the timing of action potentials from the measured noisy calcium fluorescence traces.
How close are we to understanding image-based saliency?
Within the set of the many complex factors driving gaze placement, the properities of an image that are associated with fixations under free viewing conditions have been studied extensively.
Excessive Invariance Causes Adversarial Vulnerability
Despite their impressive performance, deep neural networks exhibit striking failures on out-of-distribution inputs.
Training sparse natural image models with a fast Gibbs sampler of an extended state space
We present a new learning strategy based on an efficient blocked Gibbs sampler for sparse overcomplete linear models.
Evaluating neuronal codes for inference using Fisher information
We characterize the response distribution for the binocular energy model in response to random dot stereograms and find it to be very different from the Poisson-like noise usually assumed.
Neurometric function analysis of population codes
In this way, we provide a new rigorous framework for assessing the functional consequences of noise correlation structures for the representational accuracy of neural population codes that is in particular applicable to short-time population coding.
A joint maximum-entropy model for binary neural population patterns and continuous signals
Second-order maximum-entropy models have recently gained much interest for describing the statistics of binary spike trains.
Bayesian estimation of orientation preference maps
Imaging techniques such as optical imaging of intrinsic signals, 2-photon calcium imaging and voltage sensitive dye imaging can be used to measure the functional organization of visual cortex across different spatial scales.
Hierarchical Modeling of Local Image Features through L_p-Nested Symmetric Distributions
We introduce a new family of distributions, called $L_p${\em -nested symmetric distributions}, whose densities access the data exclusively through a hierarchical cascade of $L_p$-norms.
The Conjoint Effect of Divisive Normalization and Orientation Selectivity on Redundancy Reduction
Bandpass filtering, orientation selectivity, and contrast gain control are prominent features of sensory coding at the level of V1 simple cells.
Receptive Fields without Spike-Triggering
Can we find a concise description for the processing of a whole population of neurons analogous to the receptive field for single neurons?
Ranked #107 on
Image Classification
on STL-10
Understanding Low- and High-Level Contributions to Fixation Prediction
This model achieves better performance than all models not using features pre-trained on object recognition, making it a strong baseline to assess the utility of high-level features.
Learning From Brains How to Regularize Machines
We propose to regularize CNNs using large-scale neuroscience data to learn more robust neural features in terms of representational similarity.
Towards causal generative scene models via competition of experts
Learning how to model complex scenes in a modular way with recombinable components is a pre-requisite for higher-order reasoning and acting in the physical world.
Rotation-invariant clustering of neuronal responses in primary visual cortex
Similar to a convolutional neural network (CNN), the mammalian retina encodes visual information into several dozen nonlinear feature maps, each formed by one ganglion cell type that tiles the visual space in an approximately shift-equivariant manner.
Measuring the Importance of Temporal Features in Video Saliency
Where people look when watching videos is believed to be heavily influenced by temporal patterns.
Exemplary natural images explain CNN activations better than synthetic feature visualizations
Using a well-controlled psychophysical paradigm, we compare the informativeness of synthetic images \citep{olah2017feature} with a simple baseline visualization, namely exemplary natural images that also strongly activate a specific feature map.
On the surprising similarities between supervised and self-supervised models
In the light of this recent breakthrough, we here compare self-supervised networks to supervised models and human behaviour.
A Broad Dataset is All You Need for One-Shot Object Detection
We here show that this generalization gap can be nearly closed by increasing the number of object categories used during training.
State-of-the-Art in Human Scanpath Prediction
The last years have seen a surge in models predicting the scanpaths of fixations made by humans when viewing images.
The Geometry of Adversarial Subspaces
We undertake analysis to characterize the geometry of the boundary, which is more curved within the adversarial subspace than within a random subspace of equal dimensionality.
Out-of-distribution generalization of internal models is correlated with reward
We investigate the behavior of reinforcement learning (RL) agents under morphological distribution shifts.
Out-of-Distribution Generalization
reinforcement-learning
+1
How well do deep neural networks trained on object recognition characterize the mouse visual system?
Recent work on modeling neural responses in the primate visual system has benefited from deep neural networks trained on large-scale object recognition, and found a hierarchical correspondence between layers of the artificial neural network and brain areas along the ventral visual stream.
Natural Images are More Informative for Interpreting CNN Activations than State-of-the-Art Synthetic Feature Visualizations
Using a well-controlled psychophysical paradigm, we compare the informativeness of synthetic images by Olah et al. [45] with a simple baseline visualization, namely natural images that also strongly activate a specific feature map.
Disentanglement and Generalization Under Correlation Shifts
Exploiting such correlations may increase predictive performance on noisy data; however, often correlations are not robust (e. g., they may change between domains, datasets, or applications) and models that exploit them do not generalize when correlations shift.
Playing repeated games with Large Language Models
In a large set of two players-two strategies games, we find that LLMs are particularly good at games where valuing their own self-interest pays off, like the iterated Prisoner's Dilemma family.
Provable Compositional Generalization for Object-Centric Learning
Learning representations that generalize to novel compositions of known concepts is crucial for bridging the gap between human and machine perception.
Continual Learning: Applications and the Road Forward
Continual learning is a subfield of machine learning, which aims to allow machine learning models to continuously learn on new data, by accumulating knowledge without forgetting what was learned in the past.
Most discriminative stimuli for functional cell type clustering
Thus, for unbiased identification of the functional cell types in retina and visual cortex, new approaches are needed.
Investigating Continual Pretraining in Large Language Models: Insights and Implications
This paper studies the evolving domain of Continual Learning (CL) in large language models (LLMs), with a focus on developing strategies for efficient and sustainable training.
Wu's Method can Boost Symbolic AI to Rival Silver Medalists and AlphaGeometry to Outperform Gold Medalists at IMO Geometry
In this note, we revisit the IMO-AG-30 Challenge introduced with AlphaGeometry, and find that Wu's method is surprisingly strong.
