Search Results for author:
Found 6 papers, 4 papers with code
Neural Redshift: Random Networks are not Random Functions
Prevailing explanations are based on implicit biases of gradient descent (GD) but they cannot account for the capabilities of models from gradient-free methods nor the simplicity bias recently observed in untrained networks.
SoK: Memorization in General-Purpose Large Language Models
A major part of this success is due to their huge training datasets and the unprecedented number of model parameters, which allow them to memorize large amounts of information contained in the training data.
Language Model Decoding as Likelihood-Utility Alignment
Specifically, by analyzing the correlation between the likelihood and the utility of predictions across a diverse set of tasks, we provide empirical evidence supporting the proposed taxonomy and a set of principles to structure reasoning when choosing a decoding algorithm.
Distribution inference risks: Identifying and mitigating sources of leakage
We identify three sources of leakage: (1) memorizing specific information about the $\mathbb{E}[Y|X]$ (expected label given the feature values) of interest to the adversary, (2) wrong inductive bias of the model, and (3) finiteness of the training data.
Privacy-Preserving Classification with Secret Vector Machines
Second, we implement SecVM's distributed framework for the Cliqz web browser and deploy it for predicting user gender in a large-scale online evaluation with thousands of clients, outperforming baselines by a large margin and thus showcasing that SecVM is suitable for production environments.
Secure Summation via Subset Sums: A New Primitive for Privacy-Preserving Distributed Machine Learning
For population studies or for the training of complex machine learning models, it is often required to gather data from different actors.
