Search Results for author: Valentin Hartmann

Found 6 papers, 4 papers with code

Neural Redshift: Random Networks are not Random Functions

no code implementations4 Mar 2024 Damien Teney, Armand Nicolicioiu, Valentin Hartmann, Ehsan Abbasnejad

Prevailing explanations are based on implicit biases of gradient descent (GD) but they cannot account for the capabilities of models from gradient-free methods nor the simplicity bias recently observed in untrained networks.

SoK: Memorization in General-Purpose Large Language Models

no code implementations24 Oct 2023 Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, Robert West

A major part of this success is due to their huge training datasets and the unprecedented number of model parameters, which allow them to memorize large amounts of information contained in the training data.

Memorization Question Answering

Language Model Decoding as Likelihood-Utility Alignment

1 code implementation13 Oct 2022 Martin Josifoski, Maxime Peyrard, Frano Rajic, Jiheng Wei, Debjit Paul, Valentin Hartmann, Barun Patra, Vishrav Chaudhary, Emre Kiciman, Boi Faltings, Robert West

Specifically, by analyzing the correlation between the likelihood and the utility of predictions across a diverse set of tasks, we provide empirical evidence supporting the proposed taxonomy and a set of principles to structure reasoning when choosing a decoding algorithm.

Language Modelling Text Generation

Distribution inference risks: Identifying and mitigating sources of leakage

2 code implementations18 Sep 2022 Valentin Hartmann, Léo Meynent, Maxime Peyrard, Dimitrios Dimitriadis, Shruti Tople, Robert West

We identify three sources of leakage: (1) memorizing specific information about the $\mathbb{E}[Y|X]$ (expected label given the feature values) of interest to the adversary, (2) wrong inductive bias of the model, and (3) finiteness of the training data.

Inductive Bias

Privacy-Preserving Classification with Secret Vector Machines

1 code implementation8 Jul 2019 Valentin Hartmann, Konark Modi, Josep M. Pujol, Robert West

Second, we implement SecVM's distributed framework for the Cliqz web browser and deploy it for predicting user gender in a large-scale online evaluation with thousands of clients, outperforming baselines by a large margin and thus showcasing that SecVM is suitable for production environments.

Classification Federated Learning +2

Secure Summation via Subset Sums: A New Primitive for Privacy-Preserving Distributed Machine Learning

1 code implementation27 Jun 2019 Valentin Hartmann, Robert West

For population studies or for the training of complex machine learning models, it is often required to gather data from different actors.

BIG-bench Machine Learning Privacy Preserving

Cannot find the paper you are looking for? You can Submit a new open access paper.