Search Results for author:
Found 27 papers, 10 papers with code
Solving math word problems with process- and outcome-based feedback
Recent work has shown that asking language models to generate reasoning steps improves performance on many reasoning tasks.
Ranked #11 on
Arithmetic Reasoning
on GSM8K
(using extra training data)
Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals
However, an AI system may pursue an undesired goal even when the specification is correct, in the case of goal misgeneralization.
Improving alignment of dialogue agents via targeted human judgements
We present Sparrow, an information-seeking dialogue agent trained to be more helpful, correct, and harmless compared to prompted language model baselines.
Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models
Large language models produce human-like text that drive a growing number of applications.
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Language modelling provides a step towards intelligent communication systems by harnessing large repositories of written human knowledge to better predict and understand the world.
Ranked #1 on
College Mathematics
on BIG-bench
(using extra training data)
Ethical and social risks of harm from Language Models
We discuss the points of origin of different risks and point to potential mitigation approaches.
An Empirical Investigation of Learning from Biased Toxicity Labels
Collecting annotations from human raters often results in a trade-off between the quantity of labels one wishes to gather and the quality of these labels.
Challenges in Detoxifying Language Models
Large language models (LM) generate remarkably fluent text and can be efficiently adapted across NLP tasks.
Make Sure You're Unsure: A Framework for Verifying Probabilistic Specifications
In this direction, we first introduce a general formulation of probabilistic specifications for neural networks, which captures both probabilistic networks (e. g., Bayesian neural networks, MC-Dropout networks) and uncertain inputs (distributions over inputs arising from sensor noise or other perturbations).
Avoiding Tampering Incentives in Deep RL via Decoupled Approval
How can we design agents that pursue a given objective when all feedback mechanisms are influenceable by the agent?
REALab: An Embedded Perspective on Tampering
Standard Markov Decision Process (MDP) formulations of RL and simulated environments mirroring the MDP structure assume secure access to feedback (e. g., rewards).
Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming
In this work, we propose a first-order dual SDP algorithm that (1) requires memory only linear in the total number of network activations, (2) only requires a fixed number of forward/backward passes through the network per iteration.
Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
In the setting with additional unlabeled data, we obtain an accuracy under attack of 65. 88% against $\ell_\infty$ perturbations of size $8/255$ on CIFAR-10 (+6. 35% with respect to prior art).
An Alternative Surrogate Loss for PGD-based Adversarial Testing
Adversarial testing methods based on Projected Gradient Descent (PGD) are widely used for searching norm-bounded perturbations that cause the inputs of neural networks to be misclassified.
Scalable Neural Learning for Verifiable Consistency with Temporal Specifications
Formal verification of machine learning models has attracted attention recently, and significant progress has been made on proving simple properties like robustness to small perturbations of the input features.
Are Labels Required for Improving Adversarial Robustness?
Recent work has uncovered the interesting (and somewhat surprising) finding that training models to be invariant to adversarial perturbations requires substantially larger datasets than those required for standard classification.
Uncovering Surprising Behaviors in Reinforcement Learning via Worst-case Analysis
Reinforcement learning agents are typically trained and evaluated according to their performance averaged over some distribution of environment settings.
Verification of Non-Linear Specifications for Neural Networks
We show that a number of important properties of interest can be modeled within this class, including conservation of energy in a learned dynamics model of a physical system; semantic consistency of a classifier's output labels under adversarial perturbations and bounding errors in a system that predicts the summation of handwritten digits.
Rigorous Agent Evaluation: An Adversarial Approach to Uncover Catastrophic Failures
We demonstrate this is an issue for current agents, where even matching the compute used for training is sometimes insufficient for evaluation.
Robustness via curvature regularization, and vice versa
State-of-the-art classifiers have been shown to be largely vulnerable to adversarial perturbations.
Strength in Numbers: Trading-off Robustness and Computation via Adversarially-Trained Ensembles
We show that increasing the number of parameters in adversarially-trained models increases their robustness, and in particular that ensembling smaller models while adversarially training the entire ensemble as a single model is a more efficient way of spending said budget than simply using a larger single model.
On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
Recent work has shown that it is possible to train deep neural networks that are provably robust to norm-bounded adversarial perturbations.
Training verified learners with learned verifiers
This paper proposes a new algorithmic framework, predictor-verifier training, to train neural networks that are verifiable, i. e., networks that provably satisfy some desired input-output properties.
Adversarial Risk and the Dangers of Evaluating Against Weak Attacks
We motivate 'adversarial risk' as an objective for achieving models robust to worst-case inputs.
Semantic Code Repair using Neuro-Symbolic Transformation Networks
We study the problem of semantic code repair, which can be broadly defined as automatically fixing non-syntactic bugs in source code.
RobustFill: Neural Program Learning under Noisy I/O
Recently, two competing approaches for automatic program learning have received significant attention: (1) neural program synthesis, where a neural network is conditioned on input/output (I/O) examples and learns to generate a program, and (2) neural program induction, where a neural network generates new outputs directly using a latent program representation.
Technical Report on the CleverHans v2.1.0 Adversarial Examples Library
An adversarial example library for constructing attacks, building defenses, and benchmarking both
