Search Results for author: Leonard Berrada

Found 12 papers, 9 papers with code

ConvNets Match Vision Transformers at Scale

no code implementations25 Oct 2023 Samuel L. Smith, Andrew Brock, Leonard Berrada, Soham De

Many researchers believe that ConvNets perform well on small or moderately sized datasets, but are not competitive with Vision Transformers when given access to datasets on the web-scale.

Unlocking Accuracy and Fairness in Differentially Private Image Classification

2 code implementations21 Aug 2023 Leonard Berrada, Soham De, Judy Hanwen Shen, Jamie Hayes, Robert Stanforth, David Stutz, Pushmeet Kohli, Samuel L. Smith, Borja Balle

The poor performance of classifiers trained with DP has prevented the widespread adoption of privacy preserving machine learning in industry.

Classification Fairness +2

Differentially Private Diffusion Models Generate Useful Synthetic Images

no code implementations27 Feb 2023 Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L. Smith, Olivia Wiles, Borja Balle

By privately fine-tuning ImageNet pre-trained diffusion models with more than 80M parameters, we obtain SOTA results on CIFAR-10 and Camelyon17 in terms of both FID and the accuracy of downstream classifiers trained on synthetic data.

Image Generation Privacy Preserving

Unlocking High-Accuracy Differentially Private Image Classification through Scale

2 code implementations28 Apr 2022 Soham De, Leonard Berrada, Jamie Hayes, Samuel L. Smith, Borja Balle

Differential Privacy (DP) provides a formal privacy guarantee preventing adversaries with access to a machine learning model from extracting information about individual training points.

Classification Image Classification with Differential Privacy +1

A Stochastic Bundle Method for Interpolating Networks

1 code implementation29 Jan 2022 Alasdair Paren, Leonard Berrada, Rudra P. K. Poudel, M. Pawan Kumar

We propose a novel method for training deep neural networks that are capable of interpolation, that is, driving the empirical loss to zero.

Comment on Stochastic Polyak Step-Size: Performance of ALI-G

no code implementations20 May 2021 Leonard Berrada, Andrew Zisserman, M. Pawan Kumar

This is a short note on the performance of the ALI-G algorithm (Berrada et al., 2020) as reported in (Loizou et al., 2021).

Make Sure You're Unsure: A Framework for Verifying Probabilistic Specifications

1 code implementation NeurIPS 2021 Leonard Berrada, Sumanth Dathathri, Krishnamurthy Dvijotham, Robert Stanforth, Rudy Bunel, Jonathan Uesato, Sven Gowal, M. Pawan Kumar

In this direction, we first introduce a general formulation of probabilistic specifications for neural networks, which captures both probabilistic networks (e. g., Bayesian neural networks, MC-Dropout networks) and uncertain inputs (distributions over inputs arising from sensor noise or other perturbations).

Adversarial Robustness Out of Distribution (OOD) Detection

Training Neural Networks for and by Interpolation

1 code implementation ICML 2020 Leonard Berrada, Andrew Zisserman, M. Pawan Kumar

In modern supervised learning, many deep neural networks are able to interpolate the data: the empirical loss can be driven to near zero on all samples simultaneously.

Deep Frank-Wolfe For Neural Network Optimization

1 code implementation ICLR 2019 Leonard Berrada, Andrew Zisserman, M. Pawan Kumar

Furthermore, we compare our algorithm to SGD with a hand-designed learning rate schedule, and show that it provides similar generalization while converging faster.

Smooth Loss Functions for Deep Top-k Classification

1 code implementation ICLR 2018 Leonard Berrada, Andrew Zisserman, M. Pawan Kumar

We compare the performance of the cross-entropy loss and our margin-based losses in various regimes of noise and data size, for the predominant use case of k=5.

Classification General Classification

Trusting SVM for Piecewise Linear CNNs

2 code implementations7 Nov 2016 Leonard Berrada, Andrew Zisserman, M. Pawan Kumar

We present a novel layerwise optimization algorithm for the learning objective of Piecewise-Linear Convolutional Neural Networks (PL-CNNs), a large class of convolutional neural networks.

Cannot find the paper you are looking for? You can Submit a new open access paper.