Search Results for author:
Found 30 papers, 13 papers with code
On the Role of Speech Data in Reducing Toxicity Detection Bias
Text toxicity detection systems exhibit significant biases, producing disproportionate rates of false positives on samples mentioning demographic groups.
The Root Shapes the Fruit: On the Persistence of Gender-Exclusive Harms in Aligned Language Models
Natural-language assistants are designed to provide users with helpful responses while avoiding harmful outputs, largely achieved through alignment to human preferences.
An Effective Theory of Bias Amplification
Machine learning models may capture and amplify biases present in data, leading to disparate test performance across social groups.
Reassessing the Validity of Spurious Correlations Benchmarks
Neural networks can fail when the data contains spurious correlations.
Networked Inequality: Preferential Attachment Bias in Graph Neural Network Link Prediction
We further bridge GCN's preferential attachment bias with unfairness in link prediction and propose a new within-group fairness metric.
Weisfeiler and Leman Go Measurement Modeling: Probing the Validity of the WL Test
The expressive power of graph neural networks is usually measured by comparing how many pairs of graphs or nodes an architecture can possibly distinguish as non-isomorphic to those distinguishable by the $k$-dimensional Weisfeiler-Leman ($k$-WL) test.
Simplicity Bias Leads to Amplified Performance Disparities
Finally, we present two real-world examples of difficulty amplification in action, resulting in worse-than-expected performance disparities between groups even when using a balanced dataset.
Measuring and signing fairness as performance under multiple stakeholder distributions
By highlighting connections to the literature in domain generalization, we propose to measure fairness as the ability of the system to generalize under multiple stress tests -- distributions of examples with social relevance.
Understanding out-of-distribution accuracies through quantifying difficulty of test samples
Existing works show that although modern neural networks achieve remarkable generalization performance on the in-distribution (ID) dataset, the accuracy drops significantly on the out-of-distribution (OOD) datasets \cite{recht2018cifar, recht2019imagenet}.
Vision Models Are More Robust And Fair When Pretrained On Uncurated Images Without Supervision
Discriminative self-supervised learning allows training models on any random group of internet images, and possibly recover salient information that helps differentiate between the images.
Ranked #1 on
Copy Detection
on Copydays strong subset
(using extra training data)
Fairness Indicators for Systematic Assessments of Visual Feature Extractors
Systematic diagnosis of fairness, harms, and biases of computer vision systems is an important step towards building socially responsible systems.
Transformed CNNs: recasting pre-trained convolutional layers with self-attention
Finally, we experiment initializing the T-CNN from a partially trained CNN, and find that it reaches better performance than the corresponding hybrid model trained from scratch, while reducing training time.
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
We initialise the GPSA layers to mimic the locality of convolutional layers, then give each attention head the freedom to escape locality by adjusting a gating parameter regulating the attention paid to position versus content information.
Ranked #523 on
Image Classification
on ImageNet
On the interplay between data structure and loss function in classification problems
One of the central puzzles in modern machine learning is the ability of heavily overparametrized models to generalize well.
Post-Workshop Report on Science meets Engineering in Deep Learning, NeurIPS 2019, Vancouver
Science meets Engineering in Deep Learning took place in Vancouver as part of the Workshop section of NeurIPS 2019.
Triple descent and the two kinds of overfitting: Where & why do they appear?
We show that this peak is implicitly regularized by the nonlinearity, which is why it only becomes salient at high noise and is weakly affected by explicit regularization.
On the Heavy-Tailed Theory of Stochastic Gradient Descent for Deep Neural Networks
This assumption is often made for mathematical convenience, since it enables SGD to be analyzed as a stochastic differential equation (SDE) driven by a Brownian motion.
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
The aim of this work is to understand this fact through the lens of dynamics in the loss landscape.
A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks
This assumption is often made for mathematical convenience, since it enables SGD to be analyzed as a stochastic differential equation (SDE) driven by a Brownian motion.
Scaling description of generalization with number of parameters in deep learning
At this threshold, we argue that $\|f_{N}\|$ diverges.
A jamming transition from under- to over-parametrization affects loss landscape and generalization
We argue that in fully-connected networks a phase transition delimits the over- and under-parametrized regimes where fitting can or cannot be achieved.
The jamming transition as a paradigm to understand the loss landscape of deep neural networks
In the vicinity of this transition, properties of the curvature of the minima of the loss are critical.
Comparing Dynamics: Deep Neural Networks versus Glassy Systems
We analyze numerically the training dynamics of deep neural networks (DNN) by using methods developed in statistical physics of glassy systems.
Empirical Analysis of the Hessian of Over-Parametrized Neural Networks
In particular, we present a case that links the two observations: small and large batch gradient descent appear to converge to different basins of attraction but we show that they are in fact connected through their flat region and so belong to the same basin.
SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine
We publicly release a new large-scale dataset, called SearchQA, for machine comprehension, or question-answering.
Perspective: Energy Landscapes for Machine Learning
Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences.
Eigenvalues of the Hessian in Deep Learning: Singularity and Beyond
We look at the eigenvalues of the Hessian of a loss function before and after training.
Entropy-SGD: Biasing Gradient Descent Into Wide Valleys
This paper proposes a new optimization algorithm called Entropy-SGD for training deep neural networks that is motivated by the local geometry of the energy landscape.
Universal halting times in optimization and machine learning
Given an algorithm, which we take to be both the optimization routine and the form of the random landscape, the fluctuations of the halting time follow a distribution that, after centering and scaling, remains unchanged even when the distribution on the landscape is changed.
Explorations on high dimensional landscapes
Finding minima of a real valued non-convex function over a high dimensional space is a major challenge in science.
