Search Results for author: Borja Balle

Found 44 papers, 14 papers with code

$(\varepsilon, δ)$ Considered Harmful: Best Practices for Reporting Differential Privacy Guarantees

1 code implementation13 Mar 2025 Juan Felipe Gomez, Bogdan Kulynych, Georgios Kaissis, Jamie Hayes, Borja Balle, Antti Honkela

Current practices for reporting the level of differential privacy (DP) guarantees for machine learning (ML) algorithms provide an incomplete and potentially misleading picture of the guarantees and make it difficult to compare privacy levels across different settings.

Image Classification

Scaling Laws for Differentially Private Language Models

no code implementations31 Jan 2025 Ryan McKenna, Yangsibo Huang, Amer Sinha, Borja Balle, Zachary Charles, Christopher A. Choquette-Choo, Badih Ghazi, George Kaissis, Ravi Kumar, Ruibo Liu, Da Yu, Chiyuan Zhang

Scaling laws have emerged as important components of large language model (LLM) training as they can predict performance gains through scale, and provide guidance on important hyper-parameter choices that would otherwise be expensive.

Language Modeling Language Modelling +1

Trusted Machine Learning Models Unlock Private Inference for Problems Currently Infeasible with Cryptography

no code implementations15 Jan 2025 Ilia Shumailov, Daniel Ramage, Sarah Meiklejohn, Peter Kairouz, Florian Hartmann, Borja Balle, Eugene Bagdasarian

In this paper, we argue that capable machine learning models can fulfill the role of a trusted third party, thus enabling secure computations for applications that were previously infeasible.

Computational Efficiency

Preserving Expert-Level Privacy in Offline Reinforcement Learning

no code implementations18 Nov 2024 Navodita Sharma, Vishnu Vinod, Abhradeep Thakurta, Alekh Agarwal, Borja Balle, Christoph Dann, Aravindan Raghuveer

The offline reinforcement learning (RL) problem aims to learn an optimal policy from historical data collected by one or more behavioural policies (experts) by interacting with an environment.

Offline RL reinforcement-learning +3

To Shuffle or not to Shuffle: Auditing DP-SGD with Shuffling

no code implementations15 Nov 2024 Meenatchi Sundaram Muthu Selva Annamalai, Borja Balle, Jamie Hayes, Emiliano De Cristofaro

At the same time, we do not know how to compute tight theoretical guarantees for shuffling; thus, DP guarantees of models privately trained with shuffling are often reported as though Poisson sub-sampling was used.

The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD

no code implementations8 Oct 2024 Thomas Steinke, Milad Nasr, Arun Ganesh, Borja Balle, Christopher A. Choquette-Choo, Matthew Jagielski, Jamie Hayes, Abhradeep Guha Thakurta, Adam Smith, Andreas Terzis

The standard composition-based privacy analysis of DP-SGD effectively assumes that the adversary has access to all intermediate iterates, which is often unrealistic.

DiSK: Differentially Private Optimizer with Simplified Kalman Filter for Noise Reduction

no code implementations4 Oct 2024 Xinwei Zhang, Zhiqi Bu, Borja Balle, Mingyi Hong, Meisam Razaviyayn, Vahab Mirrokni

This approach led to the development of DP optimizers that have comparable performance with their non-private counterparts in fine-tuning tasks or in tasks with a small number of training parameters.

Operationalizing Contextual Integrity in Privacy-Conscious Assistants

no code implementations5 Aug 2024 Sahra Ghalebikesabi, Eugene Bagdasaryan, Ren Yi, Itay Yona, Ilia Shumailov, Aneesh Pappu, Chongyang Shi, Laura Weidinger, Robert Stanforth, Leonard Berrada, Pushmeet Kohli, Po-Sen Huang, Borja Balle

To steer information-sharing assistants to behave in accordance with privacy expectations, we propose to operationalize contextual integrity (CI), a framework that equates privacy with the appropriate flow of information in a given context.

Beyond the Calibration Point: Mechanism Comparison in Differential Privacy

no code implementations13 Jun 2024 Georgios Kaissis, Stefan Kolek, Borja Balle, Jamie Hayes, Daniel Rueckert

In differentially private (DP) machine learning, the privacy guarantees of DP mechanisms are often reported and compared on the basis of a single $(\varepsilon, \delta)$-pair.

Decision Making

AirGapAgent: Protecting Privacy-Conscious Conversational Agents

no code implementations8 May 2024 Eugene Bagdasarian, Ren Yi, Sahra Ghalebikesabi, Peter Kairouz, Marco Gruteser, Sewoong Oh, Borja Balle, Daniel Ramage

The growing use of large language model (LLM)-based conversational agents to manage sensitive user data raises significant privacy concerns.

Language Modeling Language Modelling +1

On the Privacy of Selection Mechanisms with Gaussian Noise

1 code implementation9 Feb 2024 Jonathan Lebensold, Doina Precup, Borja Balle

In this work, we revisit the analysis of Report Noisy Max and Above Threshold with Gaussian noise and show that, under the additional assumption that the underlying queries are bounded, it is possible to provide pure ex-ante DP bounds for Report Noisy Max and pure ex-post DP bounds for Above Threshold.

Unlocking Accuracy and Fairness in Differentially Private Image Classification

2 code implementations21 Aug 2023 Leonard Berrada, Soham De, Judy Hanwen Shen, Jamie Hayes, Robert Stanforth, David Stutz, Pushmeet Kohli, Samuel L. Smith, Borja Balle

The poor performance of classifiers trained with DP has prevented the widespread adoption of privacy preserving machine learning in industry.

Classification Fairness +2

Differentially Private Diffusion Models Generate Useful Synthetic Images

no code implementations27 Feb 2023 Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L. Smith, Olivia Wiles, Borja Balle

By privately fine-tuning ImageNet pre-trained diffusion models with more than 80M parameters, we obtain SOTA results on CIFAR-10 and Camelyon17 in terms of both FID and the accuracy of downstream classifiers trained on synthetic data.

Image Generation Privacy Preserving

Tight Auditing of Differentially Private Machine Learning

no code implementations15 Feb 2023 Milad Nasr, Jamie Hayes, Thomas Steinke, Borja Balle, Florian Tramèr, Matthew Jagielski, Nicholas Carlini, Andreas Terzis

Moreover, our auditing scheme requires only two training runs (instead of thousands) to produce tight privacy estimates, by adapting recent advances in tight composition theorems for differential privacy.

Federated Learning

Extracting Training Data from Diffusion Models

1 code implementation30 Jan 2023 Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tramèr, Borja Balle, Daphne Ippolito, Eric Wallace

Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images.

Privacy Preserving

Unlocking High-Accuracy Differentially Private Image Classification through Scale

3 code implementations28 Apr 2022 Soham De, Leonard Berrada, Jamie Hayes, Samuel L. Smith, Borja Balle

Differential Privacy (DP) provides a formal privacy guarantee preventing adversaries with access to a machine learning model from extracting information about individual training points.

Classification Image Classification with Differential Privacy +1

Reconstructing Training Data with Informed Adversaries

2 code implementations13 Jan 2022 Borja Balle, Giovanni Cherubin, Jamie Hayes

Our work provides an effective reconstruction attack that model developers can use to assess memorization of individual points in general settings beyond those considered in previous works (e. g. generative language models or access to training gradients); it shows that standard models have the capacity to store enough information to enable high-fidelity reconstruction of training data points; and it demonstrates that differential privacy can successfully mitigate such attacks in a parameter regime where utility degradation is minimal.

Memorization Reconstruction Attack

Learning to be adversarially robust and differentially private

no code implementations6 Jan 2022 Jamie Hayes, Borja Balle, M. Pawan Kumar

We study the difficulties in learning that arise from robust and differentially private optimization.

Binary Classification

A Law of Robustness for Weight-bounded Neural Networks

no code implementations16 Feb 2021 Hisham Husain, Borja Balle

Our result coincides with that conjectured in (Bubeck et al., 2020) for two-layer networks under the assumption of bounded weights.

Private Reinforcement Learning with PAC and Regret Guarantees

no code implementations18 Sep 2020 Giuseppe Vietri, Borja Balle, Akshay Krishnamurthy, Zhiwei Steven Wu

Motivated by high-stakes decision-making domains like personalized medicine where user information is inherently sensitive, we design privacy preserving exploration policies for episodic reinforcement learning (RL).

Decision Making Privacy Preserving +3

A FRAMEWORK FOR ROBUSTNESS CERTIFICATION OF SMOOTHED CLASSIFIERS USING F-DIVERGENCES

no code implementations ICLR 2020 Krishnamurthy (Dj) Dvijotham, Jamie Hayes, Borja Balle, Zico Kolter, Chongli Qin, Andras Gyorgy, Kai Xiao, Sven Gowal, Pushmeet Kohli

Formal verification techniques that compute provable guarantees on properties of machine learning models, like robustness to norm-bounded adversarial perturbations, have yielded impressive results.

Audio Classification BIG-bench Machine Learning +1

Privacy- and Utility-Preserving Textual Analysis via Calibrated Multivariate Perturbations

1 code implementation20 Oct 2019 Oluwaseyi Feyisetan, Borja Balle, Thomas Drake, Tom Diethe

We conduct privacy audit experiments against 2 baseline models and utility experiments on 3 datasets to demonstrate the tradeoff between privacy and utility for varying values of epsilon on different task types.

Privacy Preserving

Actor Critic with Differentially Private Critic

no code implementations14 Oct 2019 Jonathan Lebensold, William Hamilton, Borja Balle, Doina Precup

Reinforcement learning algorithms are known to be sample inefficient, and often performance on one task can be substantially improved by leveraging information (e. g., via pre-training) on other related tasks.

reinforcement-learning Reinforcement Learning +2

Differentially Private Summation with Multi-Message Shuffling

1 code implementation20 Jun 2019 Borja Balle, James Bell, Adria Gascon, Kobbi Nissim

In recent work, Cheu et al. (Eurocrypt 2019) proposed a protocol for $n$-party real summation in the shuffle model of differential privacy with $O_{\epsilon, \delta}(1)$ error and $\Theta(\epsilon\sqrt{n})$ one-bit messages per party.

Privacy Amplification by Mixing and Diffusion Mechanisms

no code implementations NeurIPS 2019 Borja Balle, Gilles Barthe, Marco Gaboardi, Joseph Geumlek

A fundamental result in differential privacy states that the privacy guarantees of a mechanism are preserved by any post-processing of its output.

Model-Agnostic Counterfactual Explanations for Consequential Decisions

1 code implementation27 May 2019 Amir-Hossein Karimi, Gilles Barthe, Borja Balle, Isabel Valera

Predictive models are being increasingly used to support consequential decision making at the individual level in contexts such as pretrial bail and loan approval.

counterfactual Decision Making +1

Hypothesis Testing Interpretations and Renyi Differential Privacy

no code implementations24 May 2019 Borja Balle, Gilles Barthe, Marco Gaboardi, Justin Hsu, Tetsuya Sato

These conditions are useful to analyze the distinguishability power of divergences and we use them to study the hypothesis testing interpretation of some relaxations of differential privacy based on Renyi divergence.

Two-sample testing

Privacy-preserving Active Learning on Sensitive Data for User Intent Classification

no code implementations26 Mar 2019 Oluwaseyi Feyisetan, Thomas Drake, Borja Balle, Tom Diethe

Active learning holds promise of significantly reducing data annotation costs while maintaining reasonable model performance.

Active Learning Binary Classification +4

Continual Learning in Practice

no code implementations12 Mar 2019 Tom Diethe, Tom Borchert, Eno Thereska, Borja Balle, Neil Lawrence

This paper describes a reference architecture for self-maintaining systems that can learn continually, as data arrives.

AutoML BIG-bench Machine Learning +1

The Privacy Blanket of the Shuffle Model

1 code implementation7 Mar 2019 Borja Balle, James Bell, Adria Gascon, Kobbi Nissim

Additionally, Erlingsson et al. (SODA 2019) provide a privacy amplification bound quantifying the level of curator differential privacy achieved by the shuffle model in terms of the local differential privacy of the randomizer used by each user.

model

Hierarchical Methods of Moments

1 code implementation NeurIPS 2017 Matteo Ruffini, Guillaume Rabusseau, Borja Balle

Spectral methods of moments provide a powerful tool for learning the parameters of latent variable models.

Tensor Decomposition

Subsampled Rényi Differential Privacy and Analytical Moments Accountant

1 code implementation31 Jul 2018 Yu-Xiang Wang, Borja Balle, Shiva Kasiviswanathan

We study the problem of subsampling in differential privacy (DP), a question that is the centerpiece behind many successful differentially private machine learning algorithms.

BIG-bench Machine Learning

Privacy Amplification by Subsampling: Tight Analyses via Couplings and Divergences

no code implementations NeurIPS 2018 Borja Balle, Gilles Barthe, Marco Gaboardi

Differential privacy comes equipped with multiple analytical tools for the design of private data analyses.

Improving the Gaussian Mechanism for Differential Privacy: Analytical Calibration and Optimal Denoising

1 code implementation ICML 2018 Borja Balle, Yu-Xiang Wang

The Gaussian mechanism is an essential building block used in multitude of differentially private data analysis algorithms.

Denoising

Multitask Spectral Learning of Weighted Automata

no code implementations NeurIPS 2017 Guillaume Rabusseau, Borja Balle, Joelle Pineau

We first present a natural notion of relatedness between WFAs by considering to which extent several WFAs can share a common underlying representation.

Spectral Learning from a Single Trajectory under Finite-State Policies

no code implementations ICML 2017 Borja Balle, Odalric-Ambrym Maillard

We present spectral methods of moments for learning sequential models from a single trajectory, in stark contrast with the classical literature that assumes the availability of multiple i. i. d.

Generalization Bounds for Weighted Automata

no code implementations25 Oct 2016 Borja Balle, Mehryar Mohri

We present new data-dependent generalization guarantees for learning weighted automata expressed in terms of the Rademacher complexity of these families.

Generalization Bounds

Differentially Private Policy Evaluation

no code implementations7 Mar 2016 Borja Balle, Maziar Gomrokchi, Doina Precup

We present the first differentially private algorithms for reinforcement learning, which apply to the task of evaluating a fixed policy.

reinforcement-learning Reinforcement Learning +1

Low-Rank Approximation of Weighted Tree Automata

no code implementations4 Nov 2015 Guillaume Rabusseau, Borja Balle, Shay B. Cohen

We describe a technique to minimize weighted tree automata (WTA), a powerful formalisms that subsumes probabilistic context-free grammars (PCFGs) and latent-variable PCFGs.

Cannot find the paper you are looking for? You can Submit a new open access paper.