Search Results for author:
Found 37 papers, 22 papers with code
Approximating Language Model Training Data from Weights
When applied to a model trained with SFT on MSMARCO web documents, our method reduces perplexity from 3. 3 to 2. 3, compared to an expert LLAMA model's perplexity of 2. 0.
Harnessing the Universal Geometry of Embeddings
We introduce the first method for translating text embeddings from one vector space to another without any paired data, encoders, or predefined sets of matches.
Universal Zero-shot Embedding Inversion
From the NLP perspective, it helps determine how much semantic information about the input is retained in the embedding.
Multi-Agent Systems Execute Arbitrary Malicious Code
Multi-agent systems coordinate LLM-based agents to perform tasks on users' behalf.
Rerouting LLM Routers
LLM routers aim to balance quality and cost of generation by classifying queries and routing them to a cheaper or more expensive LLM depending on their complexity.
Adversarial Hubness in Multi-Modal Retrieval
For example, in text-caption-to-image retrieval, a single adversarial hub, generated with respect to 100 randomly selected target queries, is retrieved as the top-1 most relevant image for more than 21, 000 out of 25, 000 test queries (by contrast, the most common natural hub is the top-1 response to only 102 queries), demonstrating the strong generalization capabilities of adversarial hubs.
Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy, Research, and Practice
We articulate fundamental mismatches between technical methods for machine unlearning in Generative AI, and documented aspirations for broader impact that these methods could have for law and policy.
Adversarial Decoding: Generating Readable Documents for Adversarial Objectives
We design, implement, and evaluate adversarial decoding, a new, generic text generation technique that produces readable documents for different adversarial objectives.
Self-interpreting Adversarial Images
We introduce a new type of indirect, cross-modal injection attacks against visual language models that enable creation of self-interpreting images.
Extracting Prompts by Inverting LLM Outputs
We consider the problem of language model inversion: given outputs of a language model, we seek to extract the prompt that generated these outputs.
Language Model Inversion
We consider the problem of language model inversion and show that next-token probabilities contain a surprising amount of information about the preceding text.
Text Embeddings Reveal (Almost) As Much As Text
How much private information do text embeddings reveal about the original text?
Adversarial Illusions in Multi-Modal Embeddings
In this paper, we show that multi-modal embeddings can be vulnerable to an attack we call "adversarial illusions."
Abusing Images and Sounds for Indirect Instruction Injection in Multi-Modal LLMs
We demonstrate how images and sounds can be used for indirect prompt and instruction injection in multi-modal LLMs.
Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines
Given the variety of potential backdoor attacks, ML engineers who are not security experts have no way to measure how vulnerable their current training pipelines are, nor do they have a practical way to compare training configurations so as to pick the more resistant ones.
Data Isotopes for Data Provenance in DNNs
With only query access to a trained model and no knowledge of the model training process, or control of the data labels, a user can apply statistical hypothesis testing to detect if a model has learned the spurious features associated with their isotopes by training on the user's data.
Spinning Language Models: Risks of Propaganda-As-A-Service and Countermeasures
Whereas conventional backdoors cause models to produce incorrect outputs on inputs with the trigger, outputs of spinned models preserve context and maintain standard accuracy metrics, yet also satisfy a meta-task chosen by the adversary.
Spinning Sequence-to-Sequence Models with Meta-Backdoors
We introduce the concept of a "meta-backdoor" to explain model-spinning attacks.
Adversarial Semantic Collisions
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models.
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion
We demonstrate that neural code autocompleters are vulnerable to poisoning attacks.
De-Anonymizing Text by Fingerprinting Language Generation
Components of machine learning systems are not (yet) perceived as security hotspots.
Blind Backdoors in Deep Learning Models
We investigate a new method for injecting backdoors into machine learning models, based on compromising the loss-value computation in the model-training code.
Salvaging Federated Learning by Local Adaptation
First, we show that on standard tasks such as next-word prediction, many participants gain no benefit from FL because the federated model is less accurate on their data than the models they can train locally on their own.
Humpty Dumpty: Controlling Word Meanings via Corpus Poisoning
Word embeddings, i. e., low-dimensional vector representations such as GloVe and SGNS, encode word "meaning" in the sense that distances between words' vectors correspond to their semantic proximity.
Overlearning Reveals Sensitive Attributes
For example, a binary gender classifier of facial images also learns to recognize races\textemdash even races that are not represented in the training data\textemdash and identities.
Differential Privacy Has Disparate Impact on Model Accuracy
The cost of differential privacy is a reduction in the model's accuracy.
Auditing Data Provenance in Text-Generation Models
To help enforce data-protection regulations such as GDPR and detect unauthorized uses of personal data, we develop a new \emph{model auditing} technique that helps users check if their data was used to train a machine learning model.
How To Backdoor Federated Learning
An attacker selected in a single round of federated learning can cause the global model to immediately reach 100% accuracy on the backdoor task.
Exploiting Unintended Feature Leakage in Collaborative Learning
First, we show that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data (i. e., membership inference).
Chiron: Privacy-preserving Machine Learning as a Service
Existing ML-as-a-service platforms require users to reveal all training data to the service operator.
Cryptography and Security
Fooling OCR Systems with Adversarial Text Images
We demonstrate that state-of-the-art optical character recognition (OCR) based on deep learning is vulnerable to adversarial images.
Machine Learning Models that Remember Too Much
In this setting, we design and implement practical algorithms, some of them very similar to standard ML techniques such as regularization and data augmentation, that "memorize" information about the training dataset in the model yet the model is as accurate and predictive as a conventionally trained model.
Membership Inference Attacks against Machine Learning Models
We quantitatively investigate how machine learning models leak information about the individual data records on which they were trained.
Defeating Image Obfuscation with Deep Learning
We demonstrate that modern image recognition methods based on artificial neural networks can recover hidden information from images protected by various forms of obfuscation.
Can we still avoid automatic face detection?
In this setting, is it still possible for privacy-conscientious users to avoid automatic face detection and recognition?
How To Break Anonymity of the Netflix Prize Dataset
We present a new class of statistical de-anonymization attacks against high-dimensional micro-data, such as individual preferences, recommendations, transaction records and so on.
Cryptography and Security Databases
