A common countermeasure against MI attacks is to utilize differential privacy (DP) during model training to mask the presence of individual examples.
Reconstruction attacks allow an adversary to regenerate data samples of the training set using access to only a trained model.
We revisit watermarking techniques based on pre-trained deep networks, in the light of self-supervised approaches.
Modern approaches for fast retrieval of similar vectors on billion-scaled datasets rely on compressed-domain approaches such as binary sketches or product quantization.
1 code implementation • 25 Sep 2021 • Ashkan Yousefpour, Igor Shilov, Alexandre Sablayrolles, Davide Testuggine, Karthik Prasad, Mani Malek, John Nguyen, Sayan Ghosh, Akash Bharadwaj, Jessica Zhao, Graham Cormode, Ilya Mironov
We introduce Opacus, a free, open-source PyTorch library for training deep learning models with differential privacy (hosted at opacus. ai).
In particular, we investigate the interplay of architecture and optimization of such dedicated transformers.
Ranked #3 on Image Classification on CIFAR-10 (using extra training data)
In this work, we produce a competitive convolution-free transformer by training on Imagenet only.
Ranked #3 on Document Layout Analysis on PubLayNet val
By jointly leveraging the coarse labels and the underlying fine-grained latent space, it significantly improves the accuracy of category-level retrieval methods.
Ranked #2 on Image Classification on iNaturalist 2019
Membership inference determines, given a sample and trained parameters of a machine learning model, whether the sample was part of the training set.
In our experiments we consider a dataset with up to 30 billion words, and we plug our memory layer in a state-of-the-art transformer-based architecture.
Convolutional neural networks memorize part of their training data, which is why strategies such as data augmentation and drop-out are employed to mitigate overfitting.
Similarity search approaches based on graph walks have recently attained outstanding speed-accuracy trade-offs, taking aside the memory requirements.
Hashing produces compact representations for documents, to perform tasks like classification or retrieval based on these short codes.