Search Results for author:
Found 342 papers, 142 papers with code
Learning explanations that are hard to vary
In this paper, we investigate the principle that `good explanations are hard to vary' in the context of deep learning.
Towards Total Recall in Industrial Anomaly Detection
Being able to spot defective parts is a critical component in large-scale industrial manufacturing.
Ranked #3 on
Anomaly Detection
on AeBAD-V
Parameter-Efficient Orthogonal Finetuning via Butterfly Factorization
We apply this parameterization to OFT, creating a novel parameter-efficient finetuning method, called Orthogonal Butterfly (BOFT).
Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion
Recent years have seen the rapid development of large generative models for text; however, much less research has explored the connection between text and another "language" of communication -- music.
From Variational to Deterministic Autoencoders
Variational Autoencoders (VAEs) provide a theoretically-backed and popular framework for deep generative models.
Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations
The key idea behind the unsupervised learning of disentangled representations is that real-world data is generated by a few explanatory factors of variation which can be recovered by unsupervised learning algorithms.
Weakly-Supervised Disentanglement Without Compromises
Third, we perform a large-scale empirical study and show that such pairs of observations are sufficient to reliably learn disentangled representations on several benchmark data sets.
Probable Domain Generalization via Quantile Risk Minimization
By minimizing the $\alpha$-quantile of predictor's risk distribution over domains, QRM seeks predictors that perform well with probability $\alpha$.
Flow Annealed Importance Sampling Bootstrap
Normalizing flows are tractable density models that can approximate complicated target distributions, e. g. Boltzmann distributions of physical systems.
normflows: A PyTorch Package for Normalizing Flows
It allows to build normalizing flow models from a suite of base distributions, flow layers, and neural networks.
Pairwise Similarity Learning is SimPLE
In this paper, we focus on a general yet important learning problem, pairwise similarity learning (PSL).
CausalWorld: A Robotic Manipulation Benchmark for Causal Structure and Transfer Learning
To facilitate research addressing this problem, we propose CausalWorld, a benchmark for causal structure and transfer learning in a robotic manipulation environment.
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis
Single image super-resolution is the task of inferring a high-resolution image from a single low-resolution input.
Recurrent Independent Mechanisms
Learning modular structures which reflect the dynamics of the environment can lead to better generalization and robustness to changes which only affect a few of the underlying causes.
Learning Blind Motion Deblurring
As handheld video cameras are now commonplace and available in every smartphone, images and videos can be recorded almost everywhere at anytime.
Learning Neural Causal Models from Unknown Interventions
Promising results have driven a recent surge of interest in continuous optimization methods for Bayesian network structure learning from observational data.
Learning Neural Causal Models with Active Interventions
Discovering causal structures from data is a challenging inference problem of fundamental importance in all areas of science.
Algorithmic Recourse: from Counterfactual Explanations to Interventions
As machine learning is increasingly used to inform consequential decision-making (e. g., pre-trial bail and loan approval), it becomes important to explain how the system arrived at its decision, and also suggest actions to achieve a favorable decision.
CLadder: Assessing Causal Reasoning in Language Models
Much of the existing work in natural language processing (NLP) focuses on evaluating commonsense causal reasoning in LLMs, thus failing to assess whether a model can perform causal inference in accordance with a set of well-defined formal rules.
AdaGAN: Boosting Generative Models
Generative Adversarial Networks (GAN) (Goodfellow et al., 2014) are an effective method for training generative models of complex data such as natural images.
Quantifying the Effects of Contact Tracing, Testing, and Containment Measures in the Presence of Infection Hotspots
Multiple lines of evidence strongly suggest that infection hotspots, where a single individual infects many others, play a key role in the transmission dynamics of COVID-19.
Can Large Language Models Infer Causation from Correlation?
In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs).
On the Transfer of Inductive Bias from Simulation to the Real World: a New Disentanglement Dataset
Learning meaningful and compact representations with disentangled semantic aspects is considered to be of key importance in representation learning.
Generalization and Robustness Implications in Object-Centric Learning
The idea behind object-centric representation learning is that natural scenes can better be modeled as compositions of objects and their relations as opposed to distributed representations.
Logical Fallacy Detection
In this paper, we propose the task of logical fallacy detection, and provide a new dataset (Logic) of logical fallacies generally found in text, together with an additional challenge set for detecting logical fallacies in climate change claims (LogicClimate).
Bridging the Gap to Real-World Object-Centric Learning
Humans naturally decompose their environment into entities at the appropriate level of abstraction to act in the world.
On Disentangled Representations Learned From Correlated Data
The focus of disentanglement approaches has been on identifying independent factors of variation in data.
Towards Principled Disentanglement for Domain Generalization
To tackle this challenge, we first formalize the OOD generalization problem as constrained optimization, called Disentanglement-constrained Domain Generalization (DDG).
Unifying distillation and privileged information
Distillation (Hinton et al., 2015) and privileged information (Vapnik & Izmailov, 2015) are two techniques that enable machines to learn from other machines.
Deconfounding Reinforcement Learning in Observational Settings
Using this benchmark, we demonstrate that the proposed algorithms are superior to traditional RL methods in confounded environments with observational data.
Multidimensional Contrast Limited Adaptive Histogram Equalization
Contrast enhancement is an important preprocessing technique for improving the performance of downstream tasks in image processing and computer vision.
Amortized Inference for Causal Structure Learning
Rather than searching over structures, we train a variational inference model to directly predict the causal structure from observational or interventional data.
Deep Energy Estimator Networks
We present the utility of DEEN in learning the energy, the score function, and in single-step denoising experiments for synthetic and high-dimensional data.
DiBS: Differentiable Bayesian Structure Learning
In this work, we propose a general, fully differentiable framework for Bayesian structure learning (DiBS) that operates in the continuous space of a latent probabilistic graph representation.
Self-Supervised Learning with Data Augmentations Provably Isolates Content from Style
A common practice is to perform data augmentation via hand-crafted transformations intended to leave the semantics of the data invariant.
Ranked #1 on
Image Classification
on Causal3DIdent
Real-time gravitational-wave science with neural posterior estimation
We demonstrate unprecedented accuracy for rapid gravitational-wave parameter estimation with deep learning.
Group equivariant neural posterior estimation
We here describe an alternative method to incorporate equivariances under joint transformations of parameters and data.
Neural Importance Sampling for Rapid and Reliable Gravitational-Wave Inference
This shows a median sample efficiency of $\approx 10\%$ (two orders-of-magnitude better than standard samplers) as well as a ten-fold reduction in the statistical uncertainty in the log evidence.
CausalAdv: Adversarial Robustness through the Lens of Causality
The adversarial vulnerability of deep neural networks has attracted significant attention in machine learning.
TriFinger: An Open-Source Robot for Learning Dexterity
Dexterous object manipulation remains an open problem in robotics, despite the rapid progress in machine learning during the past decade.
Resampling Base Distributions of Normalizing Flows
Normalizing flows are a popular class of models for approximating probability distributions.
Ranked #47 on
Image Generation
on CIFAR-10
(bits/dimension metric)
Causal Influence Detection for Improving Efficiency in Reinforcement Learning
Many reinforcement learning (RL) environments consist of independent entities that interact sparsely.
When to Make Exceptions: Exploring Language Models as Accounts of Human Moral Judgment
Using a state-of-the-art large language model (LLM) as a basis, we propose a novel moral chain of thought (MORALCOT) prompting strategy that combines the strengths of LLMs with theories of moral reasoning developed in cognitive science to predict human moral judgments.
Causal Curiosity: RL Agents Discovering Self-supervised Experiments for Causal Representation Learning
Inspired by this, we attempt to equip reinforcement learning agents with the ability to perform experiments that facilitate a categorization of the rolled-out trajectories, and to subsequently infer the causal factors of the environment in a hierarchical manner.
The Unreasonable Effectiveness of Texture Transfer for Single Image Super-resolution
Moreover, we show that a texture representation of those deep features better capture the perceptual quality of an image than the original deep features.
Algorithmic recourse under imperfect causal knowledge: a probabilistic approach
Recent work has discussed the limitations of counterfactual explanations to recommend actions for algorithmic recourse, and argued for the need of taking causal relationships between features into consideration.
On the Fairness of Causal Algorithmic Recourse
Algorithmic fairness is typically studied from the perspective of predictions.
Visual Representation Learning Does Not Generalize Strongly Within the Same Domain
An important component for generalization in machine learning is to uncover underlying latent factors of variation as well as the mechanism through which each factor acts in the world.
Convolutional neural networks: a magic bullet for gravitational-wave detection?
In the last few years, machine learning techniques, in particular convolutional neural networks, have been investigated as a method to replace or complement traditional matched filtering techniques that are used to detect the gravitational-wave signature of merging black holes.
Invariant Models for Causal Transfer Learning
We focus on the problem of Domain Generalization, in which no examples from the test task are observed.
Towards a Learning Theory of Cause-Effect Inference
We pose causal inference as the problem of learning to classify probability distributions.
Causal Triplet: An Open Challenge for Intervention-centric Causal Representation Learning
Recent years have seen a surge of interest in learning high-level causal representations from low-level image pairs under interventions.
Optimal Decision Making Under Strategic Behavior
Using this characterization, we first show that, in general, we cannot expect to find optimal decision policies in polynomial time and there are cases in which deterministic policies are suboptimal.
Independent mechanism analysis, a new concept?
Specifically, our approach is motivated by thinking of each source as independently influencing the mixing process.
Adaptive Skip Intervals: Temporal Abstraction for Recurrent Dynamical Models
We introduce a method which enables a recurrent dynamics model to be temporally abstract.
Bayesian Online Prediction of Change Points
Online detection of instantaneous changes in the generative process of a data sequence generally focuses on retrospective inference of such change points without considering their future occurrences.
Kernel Mean Matching for Content Addressability of GANs
We propose a novel procedure which adds "content-addressability" to any given unconditional implicit model e. g., a generative adversarial network (GAN).
Embrace the Gap: VAEs Perform Independent Mechanism Analysis
Leveraging self-consistency, we show that the ELBO converges to a regularized log-likelihood.
Relative gradient optimization of the Jacobian term in unsupervised deep learning
Learning expressive probabilistic models correctly describing the data is a ubiquitous problem in machine learning.
Interventions, Where and How? Experimental Design for Causal Models at Scale
Existing methods in experimental design for causal discovery from limited data either rely on linear assumptions for the SCM or select only the intervention target.
Membership Inference Attacks against Language Models via Neighbourhood Comparison
To investigate whether this fragility provides a layer of safety, we propose and evaluate neighbourhood attacks, which compare model scores for a given sample to scores of synthetically generated neighbour texts and therefore eliminate the need for access to the training data distribution.
SE(3) Equivariant Augmented Coupling Flows
Coupling normalizing flows allow for fast sampling and density evaluation, making them the tool of choice for probabilistic modeling of physical systems.
AutoML Two-Sample Test
Two-sample tests are important in statistics and machine learning, both as tools for scientific discovery as well as to detect distribution shifts.
Informative Features for Model Comparison
Given two candidate models, and a set of target observations, we address the problem of measuring the relative goodness of fit of the two models.
Causality for Machine Learning
Graphical causal inference as pioneered by Judea Pearl arose from research on artificial intelligence (AI), and for a long time had little connection to the field of machine learning.
First-order Adversarial Vulnerability of Neural Networks and Input Dimension
Over the past few years, neural networks were proven vulnerable to adversarial images: targeted but imperceptible image perturbations lead to drastically different predictions.
Source-Free Adaptation to Measurement Shift via Bottom-Up Feature Restoration
Existing methods for SFDA leverage entropy-minimization techniques which: (i) apply only to classification; (ii) destroy model calibration; and (iii) rely on the source model achieving a good level of feature-space class-separation in the target domain.
Causal Component Analysis
As a corollary, this interventional perspective also leads to new identifiability results for nonlinear ICA -- a special case of CauCA with an empty graph -- requiring strictly fewer datasets than previous results.
Benchmarking Offline Reinforcement Learning on Real-Robot Hardware
To coordinate the efforts of the research community toward tackling this problem, we propose a benchmark including: i) a large collection of data for offline learning from a dexterous manipulation platform on two tasks, obtained with capable RL agents trained in simulation; ii) the option to execute learned policies on a real-world robotic system and a simulation for efficient debugging.
Flow Matching for Scalable Simulation-Based Inference
Neural posterior estimation methods based on discrete normalizing flows have become established tools for simulation-based inference (SBI), but scaling them to high-dimensional problems can be challenging.
Automatic Estimation of Modulation Transfer Functions
The modulation transfer function (MTF) is widely used to characterise the performance of optical systems.
Discovering Causal Signals in Images
Our experiments demonstrate the existence of a relation between the direction of causality and the difference between objects and their contexts, and by the same token, the existence of observable signals that reveal the causal dispositions of objects.
Predicting Infectiousness for Proactive Contact Tracing
Predictions are used to provide personalized recommendations to the individual via an app, as well as to send anonymized messages to the individual's contacts, who use this information to better predict their own infectiousness, an approach we call proactive contact tracing (PCT).
A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models
By grounding the behavioral analysis in a causal graph describing an intuitive reasoning process, we study the behavior of language models in terms of robustness and sensitivity to direct interventions in the input space.
AReS and MaRS - Adversarial and MMD-Minimizing Regression for SDEs
Stochastic differential equations are an important modeling class in many disciplines.
Kernel Distributionally Robust Optimization
We prove a theorem that generalizes the classical duality in the mathematical problem of moments.
Simpson's paradox in Covid-19 case fatality rates: a mediation analysis of age-related causal effects
We point out limitations and extensions for future work, and, finally, discuss the role of causal reasoning in the broader context of using AI to combat the Covid-19 pandemic.
Applications Methodology
A machine learning route between band mapping and band structure
Electronic band structure (BS) and crystal structure are the two complementary identifiers of solid state materials.
Data Analysis, Statistics and Probability Materials Science Computational Physics
Robustifying Independent Component Analysis by Adjusting for Group-Wise Stationary Noise
We introduce coroICA, confounding-robust independent component analysis, a novel ICA algorithm which decomposes linearly mixed multivariate observations into independent components that are corrupted (and rendered dependent) by hidden group-wise stationary confounding.
Differentially Private Database Release via Kernel Mean Embeddings
First, releasing (an estimate of) the kernel mean embedding of the data generating random variable instead of the database itself still allows third-parties to construct consistent estimators of a wide class of population statistics.
Learning Independent Causal Mechanisms
The approach is unsupervised and based on a set of experts that compete for data generated by the mechanisms, driving specialization.
Testing Goodness of Fit of Conditional Density Models with Kernels
We propose two nonparametric statistical tests of goodness of fit for conditional distributions: given a conditional probability density function $p(y|x)$ and a joint sample, decide whether the sample is drawn from $p(y|x)r_x(x)$ for some density $r_x$.
Assaying Out-Of-Distribution Generalization in Transfer Learning
Since out-of-distribution generalization is a generally ill-posed problem, various proxy targets (e. g., calibration, adversarial robustness, algorithmic corruptions, invariance across shifts) were studied across different research programs resulting in different recommendations.
Adversarial Robustness
Out-of-Distribution Generalization
+1
Homomorphism Autoencoder -- Learning Group Structured Representations from Observed Transitions
How can agents learn internal models that veridically represent interactions with the real world is a largely open question.
Adversarial Extreme Multi-label Classification
The goal in extreme multi-label classification is to learn a classifier which can assign a small subset of relevant labels to an instance from an extremely large set of target labels.
Recovery of non-linear cause-effect relationships from linearly mixed neuroimaging data
Causal inference concerns the identification of cause-effect relationships between variables.
ODIN: ODE-Informed Regression for Parameter and State Inference in Time-Continuous Dynamical Systems
Parameter inference in ordinary differential equations is an important problem in many applied sciences and in engineering, especially in a data-scarce setting.
MYND: Unsupervised Evaluation of Novel BCI Control Strategies on Consumer Hardware
We introduce MYND: A framework that couples consumer-grade recording hardware with an easy-to-use application for the unsupervised evaluation of BCI control strategies.
Human-Computer Interaction Neurons and Cognition 68U35 H.5.2
Bayesian Quadrature on Riemannian Data Manifolds
Riemannian manifolds provide a principled way to model nonlinear geometric structure inherent in data.
Learning with Hyperspherical Uniformity
Due to the over-parameterization nature, neural networks are a powerful tool for nonlinear function approximation.
Pyfectious: An individual-level simulator to discover optimal containment polices for epidemic diseases
Simulating the spread of infectious diseases in human communities is critical for predicting the trajectory of an epidemic and verifying various policies to control the devastating impacts of the outbreak.
BaCaDI: Bayesian Causal Discovery with Unknown Interventions
Inferring causal structures from experimentation is a central task in many domains.
The Expressive Leaky Memory Neuron: an Efficient and Expressive Phenomenological Neuron Model Can Solve Long-Horizon Tasks
Biological cortical neurons are remarkably sophisticated computational devices, temporally integrating their vast synaptic input over an intricate dendritic tree, subject to complex, nonlinearly interacting internal biological processes.
Ranked #1 on
Time Series
on neuronIO
Fair Decisions Despite Imperfect Predictions
In this paper, we show that in this selective labels setting, learning a predictor directly only from available labeled data is suboptimal in terms of both fairness and utility.
Semi-Supervised Learning, Causality and the Conditional Cluster Assumption
While the success of semi-supervised learning (SSL) is still not fully understood, Sch\"olkopf et al. (2012) have established a link to the principle of independent causal mechanisms.
Kernel Stein Tests for Multiple Model Comparison
The first test, building on the post selection inference framework, provably controls the number of best models that are wrongly declared worse (false positive rate).
Instrumental Variable Regression via Kernel Maximum Moment Loss
We investigate a simple objective for nonlinear instrumental variable (IV) regression based on a kernelized conditional moment restriction (CMR) known as a maximum moment restriction (MMR).
Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP
The principle of independent causal mechanisms (ICM) states that generative processes of real world data consist of independent modules which do not influence or inform each other.
Estimation Beyond Data Reweighting: Kernel Method of Moments
We provide a variant of our estimator for conditional moment restrictions and show that it is asymptotically first-order optimal for such problems.
CausalCite: A Causal Formulation of Paper Citations
Evaluating the significance of a paper is pivotal yet challenging for the scientific community.
GeNet: Deep Representations for Metagenomics
We introduce GeNet, a method for shotgun metagenomic classification from raw DNA sequences that exploits the known hierarchical structure between labels for training.
Neural Lyapunov Redesign
We provide theoretical results on the class of systems that can be treated with the proposed algorithm and empirically evaluate the effectiveness of our method using an exemplary dynamical system.
Adversarially Robust Kernel Smoothing
We propose a scalable robust learning algorithm combining kernel smoothing and robust optimization.
On the Adversarial Robustness of Causal Algorithmic Recourse
Algorithmic recourse seeks to provide actionable recommendations for individuals to overcome unfavorable classification outcomes from automated decision-making systems.
Adaptation and Robust Learning of Probabilistic Movement Primitives
However, to be able to capture variability and correlations between different joints, a probabilistic movement primitive requires the estimation of a larger number of parameters compared to their deterministic counterparts, that focus on modeling only the mean behavior.
Learning Kernel Tests Without Data Splitting
Modern large-scale kernel-based tests such as maximum mean discrepancy (MMD) and kernelized Stein discrepancy (KSD) optimize kernel hyperparameters on a held-out sample via data splitting to obtain the most powerful test statistics.
Exploring the Latent Space of Autoencoders with Interventional Assays
Autoencoders exhibit impressive abilities to embed the data manifold into a low-dimensional latent space, making them a staple of representation learning methods.
Direct Advantage Estimation
The predominant approach in reinforcement learning is to assign credit to actions based on the expected return.
DCI-ES: An Extended Disentanglement Framework with Connections to Identifiability
In representation learning, a common approach is to seek representations which disentangle the underlying factors of variation.
Iterative Teaching by Data Hallucination
We consider the problem of iterative machine teaching, where a teacher sequentially provides examples based on the status of a learner under a discrete input space (i. e., a pool of finite samples), which greatly limits the teacher's capability.
Removing systematic errors for exoplanet search via latent causes
We describe a method for removing the effect of confounders in order to reconstruct a latent quantity of interest.
Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance
We show that these two factors have a large causal effect on the MT performance, in addition to the test-model direction mismatch highlighted by existing work on the impact of translationese.
Domain generalization via invariant feature representation
This paper investigates domain generalization: How to take knowledge acquired from an arbitrary number of related domains and apply it to previously unseen domains?
On the Identifiability and Estimation of Causal Location-Scale Noise Models
We study the class of location-scale or heteroscedastic noise models (LSNMs), in which the effect $Y$ can be written as a function of the cause $X$ and a noise source $N$ independent of $X$, which may be scaled by a positive function $g$ over the cause, i. e., $Y = f(X) + g(X)N$.
Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap
We then use hyperspherical uniformity (which characterizes the degree of uniformity on the unit hypersphere) as a unified framework to quantify these two objectives.
Beyond Good Intentions: Reporting the Research Landscape of NLP for Social Good
Using state-of-the-art NLP models, we address each of these tasks and use them on the entire ACL Anthology, resulting in a visualization workspace that gives researchers a comprehensive overview of the field of NLP4SG.
Stochastic Marginal Likelihood Gradients using Neural Tangent Kernels
Recent works show that Bayesian model selection with Laplace approximations can allow to optimize such hyperparameters just like standard neural network parameters using gradients and on the training data.
A New Distribution-Free Concept for Representing, Comparing, and Propagating Uncertainty in Dynamical Systems with Kernel Probabilistic Programming
This work presents the concept of kernel mean embedding and kernel probabilistic programming in the context of stochastic systems.
A Kernel Mean Embedding Approach to Reducing Conservativeness in Stochastic Programming and Control
We apply kernel mean embedding methods to sample-based stochastic optimization and control.
The Inductive Bias of Quantum Kernels
Quantum computers offer the possibility to efficiently compute inner products of exponentially large density operators that are classically hard to compute.
Half-sibling regression meets exoplanet imaging: PSF modeling and subtraction using a flexible, domain knowledge-driven, causal framework
Our HSR-based method provides an alternative, flexible and promising approach to the challenge of modeling and subtracting the stellar PSF and systematic noise in exoplanet imaging data.
Federated Causal Discovery From Interventions
We propose FedCDI, a federated framework for inferring causal structures from distributed data containing interventional samples.
All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Transformer models bring propelling advances in various NLP tasks, thus inducing lots of interpretability research on the learned representations of the models.
A diverse Multilingual News Headlines Dataset from around the World
Babel Briefings is a novel dataset featuring 4. 7 million news headlines from August 2020 to November 2021, across 30 languages and 54 locations worldwide with English translations of all articles included.
SLEIPNIR: Deterministic and Provably Accurate Feature Expansion for Gaussian Process Regression with Derivatives
Gaussian processes are an important regression tool with excellent analytic properties which allow for direct integration of derivative observations.
Unsupervised Object Learning via Common Fate
Learning generative object models from unlabelled videos is a long standing problem and required for causal scene modeling.
Causal Inference Through the Structural Causal Marginal Problem
We introduce an approach to counterfactual inference based on merging information from multiple datasets.
Functional Generalized Empirical Likelihood Estimation for Conditional Moment Restrictions
Important problems in causal inference, economics, and, more generally, robust machine learning can be expressed as conditional moment restrictions, but estimation becomes challenging as it requires solving a continuum of unconditional moment restrictions.
Evaluating vaccine allocation strategies using simulation-assisted causal modelling
Early on during a pandemic, vaccine availability is limited, requiring prioritisation of different population groups.
Parameterizing pressure-temperature profiles of exoplanet atmospheres with neural networks
When training and evaluating our method on two publicly available datasets of self-consistent PT profiles, we find that our method achieves, on average, better fit quality than existing baseline methods, despite using fewer parameters.
Discrete Key-Value Bottleneck
Deep neural networks perform well on classification tasks where data streams are i. i. d.
Robustness Implies Fairness in Causal Algorithmic Recourse
Algorithmic recourse aims to disclose the inner workings of the black-box decision process in situations where decisions have significant consequences, by providing recommendations to empower beneficiaries to achieve a more favorable outcome.
Competition of Mechanisms: Tracing How Language Models Handle Facts and Counterfactuals
Interpretability research aims to bridge the gap between the empirical success and our scientific understanding of the inner workings of large language models (LLMs).
Tempered Adversarial Networks
A possible explanation for training instabilities is the inherent imbalance between the networks: While the discriminator is trained directly on both real and fake samples, the generator only has control over the fake samples it produces since the real data distribution is fixed by the choice of a given dataset.
On Matching Pursuit and Coordinate Descent
Exploiting the connection between the two algorithms, we present a unified analysis of both, providing affine invariant sublinear $\mathcal{O}(1/t)$ rates on smooth objectives and linear convergence on strongly convex objectives.
Competitive Training of Mixtures of Independent Deep Generative Models
A common assumption in causal modeling posits that the data is generated by a set of independent mechanisms, and algorithms should aim to recover this structure.
A Local Information Criterion for Dynamical Systems
Apart from its application for encoding a sequence of observations, we propose to use the compression achieved by this encoding as a criterion for model selection.
Fidelity-Weighted Learning
To this end, we propose "fidelity-weighted learning" (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data.
Coordinating users of shared facilities via data-driven predictive assistants and game theory
We study data-driven assistants that provide congestion forecasts to users of shared facilities (roads, cafeterias, etc.
Experimental and causal view on information integration in autonomous agents
The amount of digitally available but heterogeneous information about the world is remarkable, and new technologies such as self-driving cars, smart homes, or the internet of things may further increase it.
Analysis of cause-effect inference by comparing regression errors
We address the problem of inferring the causal direction between two variables by comparing the least-squares errors of the predictions in both possible directions.
Avoiding Discrimination through Causal Reasoning
Going beyond observational criteria, we frame the problem of discrimination based on protected attributes in the language of causal reasoning.
Discriminative k-shot learning using probabilistic models
The goal is to generalise from an initial large-scale classification task to a separate task comprising new classes and small numbers of examples.
Causal Consistency of Structural Equation Models
Complex systems can be modelled at various levels of detail.
Interpolated Policy Gradient: Merging On-Policy and Off-Policy Gradient Estimation for Deep Reinforcement Learning
Off-policy model-free deep reinforcement learning methods using previously collected data can improve sample efficiency over on-policy policy gradient techniques.
Local Group Invariant Representations via Orbit Embeddings
We consider transformations that form a \emph{group} and propose an approach based on kernel methods to derive local group invariant representations.
Annealed Generative Adversarial Networks
We introduce a novel framework for adversarial training where the target distribution is annealed between the uniform distribution and the data distribution.
Group invariance principles for causal generative models
The postulate of independence of cause and mechanism (ICM) has recently led to several new causal discovery algorithms.
Online Video Deblurring via Dynamic Temporal Blending Network
We show the superiority of the proposed method in an extensive experimental evaluation.
A note on the expected minimum error probability in equientropic channels
While the channel capacity reflects a theoretical upper bound on the achievable information transmission rate in the limit of infinitely many bits, it does not characterise the information transfer of a given encoding routine with finitely many bits.
Distilling Information Reliability and Source Trustworthiness from Digital Traces
Then, we propose a temporal point process modeling framework that links these temporal traces to robust, unbiased and interpretable notions of information reliability and source trustworthiness.
Discriminative Transfer Learning for General Image Restoration
Recently, several discriminative learning approaches have been proposed for effective image restoration, achieving convincing trade-off between image quality and computational efficiency.
Kernel Mean Embedding of Distributions: A Review and Beyond
Next, we discuss the Hilbert space embedding for conditional distributions, give theoretical insights, and review some applications.
Kernel-based Tests for Joint Independence
Based on an empirical estimate of dHSIC, we define three different non-parametric hypothesis tests: a permutation test, a bootstrap test and a test based on a Gamma approximation.
Consistent Kernel Mean Estimation for Functions of Random Variables
We provide a theoretical foundation for non-parametric estimation of functions of random variables using kernel mean embeddings.
Screening Rules for Convex Problems
We propose a new framework for deriving screening rules for convex optimization problems.
End-to-End Learning for Image Burst Deblurring
We present a neural network model approach for multi-frame blind deconvolution.
Depth Estimation Through a Generative Model of Light Field Synthesis
Light field photography captures rich structural information that may facilitate a number of traditional image processing and computer vision tasks.
Discovery and Visualization of Nonstationary Causal Models
Third, we develop a method for visualizing the nonstationarity of causal modules.
Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions
The RKHS distance of two mapped measures is a semi-metric $d_k$ over $M$.
Kernel Mean Shrinkage Estimators
A mean function in a reproducing kernel Hilbert space (RKHS), or a kernel mean, is central to kernel methods in that it is used by many classical algorithms such as kernel principal component analysis, and it also forms the core inference step of modern kernel methods that rely on embedding probability distributions in RKHSs.
Distinguishing cause from effect using observational data: methods and benchmarks
We evaluate the performance of several bivariate causal discovery methods on these real-world benchmark data and in addition on artificially simulated data.
Causal Inference by Identification of Vector Autoregressive Processes with Hidden Components
A widely applied approach to causal inference from a non-experimental time series $X$, often referred to as "(linear) Granger causal analysis", is to regress present on past and interpret the regression matrix $\hat{B}$ causally.
Causal and anti-causal learning in pattern recognition for neuroimaging
Pattern recognition in neuroimaging distinguishes between two types of models: encoding- and decoding models.
Decoding index finger position from EEG using random forests
While invasively recorded brain activity is known to provide detailed information on motor commands, it is an open question at what level of detail information about positions of body parts can be decoded from non-invasively acquired signals.
Causal interpretation rules for encoding and decoding models in neuroimaging
Causal terminology is often introduced in the interpretation of encoding and decoding models trained on neuroimaging data.
Distinguishing Cause from Effect Based on Exogeneity
Recent developments in structural equation modeling have produced several methods that can usually distinguish cause from effect in the two-variable case.
Computing Functions of Random Variables via Reproducing Kernel Hilbert Space Representations
We describe a method to perform functional operations on probability distributions of random variables.
Kernel Mean Estimation via Spectral Filtering
The problem of estimating the kernel mean in a reproducing kernel Hilbert space (RKHS) is central to kernel methods in that it is used by classical approaches (e. g., when centering a kernel PCA matrix), and it also forms the core inference step of modern kernel methods (e. g., kernel-based non-parametric tests) that rely on embedding probability distributions in RKHSs.
Randomized Nonlinear Component Analysis
Although nonlinear variants of PCA and CCA have been proposed, these are computationally prohibitive in the large scale.
Causal Discovery with Continuous Additive Noise Models
We consider the problem of learning causal directed acyclic graphs from an observational joint distribution.
Justifying Information-Geometric Causal Inference
Information Geometric Causal Inference (IGCI) is a new approach to distinguish between cause and effect for two variables.
Consistency of Causal Inference under the Additive Noise Model
We analyze a family of methods for statistical causal inference from sample under the so-called Additive Noise Model.
Spatial statistics, image analysis and percolation theory
We specifically address the problem of detection of multiple objects of unknown shapes in the case of nonparametric noise.
Kernel Mean Estimation and Stein's Effect
A mean function in reproducing kernel Hilbert space, or a kernel mean, is an important part of many applications ranging from kernel principal component analysis to Hilbert-space embedding of distributions.
The Randomized Dependence Coefficient
We introduce the Randomized Dependence Coefficient (RDC), a measure of non-linear dependence between random variables of arbitrary dimension based on the Hirschfeld-Gebelein-R\'enyi Maximum Correlation Coefficient.
One-Class Support Measure Machines for Group Anomaly Detection
We propose one-class support measure machines (OCSMMs) for group anomaly detection which aims at recognizing anomalous aggregate behaviors of data points.
From Ordinary Differential Equations to Structural Causal Models: the deterministic case
We show how, and under which conditions, the equilibrium states of a first-order Ordinary Differential Equation (ODE) system can be described with a deterministic Structural Causal Model (SCM).
Hilbert space embeddings and metrics on probability measures
First, we consider the question of determining the conditions on the kernel $k$ for which $\gamma_k$ is a metric: such $k$ are denoted {\em characteristic kernels}.
Uncovering the Temporal Dynamics of Diffusion Networks
Time plays an essential role in the diffusion of information, influence and disease over networks.
Perceptual Video Super Resolution with Enhanced Temporal Consistency
Together with a video discriminator, we also propose additional loss functions to further reinforce temporal consistency in the generated sequences.
On integral probability metrics, φ-divergences and binary classification
First, to understand the relation between IPMs and $\phi$-divergences, the necessary and sufficient conditions under which these classes intersect are derived: the total variation distance is shown to be the only non-trivial $\phi$-divergence that is also an IPM.
Information Theory Information Theory
Robustly Disentangled Causal Mechanisms: Validating Deep Representations for Interventional Robustness
The ability to learn disentangled representations that split underlying sources of variation in high dimensional, unstructured data is important for data efficient and robust use of neural networks.
Deep Nonlinear Non-Gaussian Filtering for Dynamical Systems
Filtering is a general name for inferring the states of a dynamical system given observations.
Generalization in anti-causal learning
Anti-causal models are used to drive this search, but a causal model is required for validation.
Counterfactuals uncover the modular structure of deep generative models
Deep generative models can emulate the perceptual properties of complex image datasets, providing a latent representation of the data.
Minimax Estimation of Maximum Mean Discrepancy with Radial Kernels
Maximum Mean Discrepancy (MMD) is a distance on the space of probability measures which has found numerous applications in machine learning and nonparametric testing.
Causal Inference on Time Series using Restricted Structural Equation Models
We study a class of restricted Structural Equation Models for time series that we call Time Series Models with Independent Noise (TiMINo).
Statistical analysis of coupled time series with Kernel Cross-Spectral Density operators.
This framework enables us to develop an independence test between time series as well as a similarity measure to compare different types of coupling.
Learning from Distributions via Support Measure Machines
This paper presents a kernel-based discriminative learning framework on probability measures.
The representer theorem for Hilbert spaces: a necessary and sufficient condition
In particular, the main result of this paper implies that, for a sufficiently large family of regularization functionals, radial nondecreasing functions are the only lower semicontinuous regularization terms that guarantee existence of a representer theorem for any choice of the data.
Semi-Supervised Domain Adaptation with Non-Parametric Copulas
A new framework based on the theory of copulas is proposed to address semi-supervised domain adaptation problems.
On Causal Discovery with Cyclic Additive Noise Models
We study a particular class of cyclic causal models, where each variable is a (possibly nonlinear) function of its parents and additive noise.
Recovering Intrinsic Images with a Global Sparsity Prior on Reflectance
We address the challenging task of decoupling material properties from lighting properties given a single image.
Switched Latent Force Models for Movement Segmentation
Latent force models encode the interaction between multiple related dynamical systems in the form of a kernel or covariance function.
Space-Variant Single-Image Blind Deconvolution for Removing Camera Shake
Modelling camera shake as a space-invariant convolution simplifies the problem of removing camera shake, but often insufficiently models actual motion blur such as those due to camera rotation and movements outside the sensor plane or when objects in the scene have different distances to the camera.
Probabilistic latent variable models for distinguishing between cause and effect
To this end, we consider the hypothetical effect variable to be a function of the hypothetical cause variable and an independent noise term (not necessarily additive).
