Search Results for author:
Found 26 papers, 14 papers with code
Scaling Laws for Adversarial Attacks on Language Model Activations
We find that the number of bits of control in the input space needed to control a single bit in the output space (what we call attack resistance $\chi$) is remarkably constant between $\approx 16$ and $\approx 25$ over 2 orders of magnitude of model sizes for different language models.
Multi-attacks: Many images $+$ the same adversarial attack $\to$ many target labels
We show that we can easily design a single adversarial perturbation $P$ that changes the class of $n$ images $X_1, X_2,\dots, X_n$ from their original, unperturbed classes $c_1, c_2,\dots, c_n$ to desired (not necessarily all the same) classes $c^*_1, c^*_2,\dots, c^*_n$ for up to hundreds of images and target classes at once.
Constitutional AI: Harmlessness from AI Feedback
In the RL phase, we sample from the finetuned model, use a model to evaluate which of the two samples is better, and then train a preference model from this dataset of AI preferences.
Measuring Progress on Scalable Oversight for Large Language Models
Developing safe and useful general-purpose AI systems will require us to make progress on scalable oversight: the problem of supervising systems that potentially outperform us on most skills relevant to the task at hand.
What does a deep neural network confidently perceive? The effective dimension of high certainty class manifolds and their low confidence boundaries
Deep neural network classifiers partition input space into high confidence regions for each class.
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
We provide our own analysis of the data and find a variety of harmful outputs, which range from offensive language to more subtly harmful non-violent unethical outputs.
Language Models (Mostly) Know What They Know
We study whether language models can evaluate the validity of their own claims and predict which questions they will be able to answer correctly.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
We apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models to act as helpful and harmless assistants.
Adversarial vulnerability of powerful near out-of-distribution detection
There has been a significant progress in detecting out-of-distribution (OOD) inputs in neural networks recently, primarily due to the use of large models pretrained on large datasets, and an emerging use of multi-modality.
How many degrees of freedom do we need to train deep networks: a loss landscape perspective
In particular, we show via Gordon's escape theorem, that the training dimension plus the Gaussian width of the desired loss sub-level set, projected onto a unit sphere surrounding the initialization, must exceed the total number of parameters for the success probability to be large.
A Simple Fix to Mahalanobis Distance for Improving Near-OOD Detection
Mahalanobis distance (MD) is a simple and popular post-processing method for detecting out-of-distribution (OOD) inputs in neural networks.
Exploring the Limits of Out-of-Distribution Detection
Near out-of-distribution detection (OOD) is a major challenge for deep neural networks.
Ranked #2 on
Out-of-Distribution Detection
on CIFAR-10 vs CIFAR-100
(using extra training data)
Out-of-Distribution Detection
Out of Distribution (OOD) Detection
+1
Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error
In this work, we provide a detailed empirical evaluation of how the number of augmentation samples per unique image influences model performance on held out data when training deep ResNets.
Ranked #124 on
Image Classification
on ImageNet
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes
Linear interpolation between initial neural network parameters and converged parameters after training with stochastic gradient descent (SGD) typically leads to a monotonic decrease in the training objective.
Slice, Dice, and Optimize: Measuring the Dimension of Neural Network Class Manifolds
Deep neural network classifiers naturally partition input space into regions belonging to different classes.
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel
We study the relationship between the training dynamics of nonlinear deep networks, the geometry of the loss landscape, and the time evolution of a data-dependent NTK.
Training independent subnetworks for robust prediction
Recent approaches to efficiently ensemble neural networks have shown that strong robustness and uncertainty performance can be achieved with a negligible gain in parameters over the original network.
The Break-Even Point on Optimization Trajectories of Deep Neural Networks
We argue for the existence of the "break-even" point on this trajectory, beyond which the curvature of the loss surface and noise in the gradient are implicitly regularized by SGD.
Deep Ensembles: A Loss Landscape Perspective
One possible explanation for this gap between theory and practice is that popular scalable variational Bayesian methods tend to focus on a single mode, whereas deep ensembles tend to explore diverse modes in function space.
Emergent properties of the local geometry of neural loss landscapes
The local geometry of high dimensional neural network loss landscapes can both challenge our cherished theoretical intuitions as well as dramatically impact the practical success of neural network training.
Large Scale Structure of Neural Network Loss Landscapes
There are many surprising and perhaps counter-intuitive properties of optimization of deep neural networks.
Stiffness: A New Perspective on Generalization in Neural Networks
In particular, we study how stiffness depends on 1) class membership, 2) distance between data points in the input space, 3) training iteration, and 4) learning rate.
Adaptive Quantum State Tomography with Neural Networks
We demonstrate that our algorithm learns to work with basis, symmetric informationally complete (SIC), as well as other types of POVMs.
The Goldilocks zone: Towards better understanding of neural network loss landscapes
We observe this effect for fully-connected neural networks over a range of network widths and depths on MNIST and CIFAR-10 datasets with the $\mathrm{ReLU}$ and $\tanh$ non-linearities, and a similar effect for convolutional networks.
Towards understanding feedback from supermassive black holes using convolutional neural networks
Supermassive black holes at centers of clusters of galaxies strongly interact with their host environment via AGN feedback.
Gaussian Prototypical Networks for Few-Shot Learning on Omniglot
We show that Gaussian prototypical networks are a preferred architecture over vanilla prototypical networks with an equivalent number of parameters.
