Search Results for author:
Found 33 papers, 18 papers with code
Antidistillation Sampling
Frontier models that generate extended reasoning traces inadvertently produce rich token sequences that can facilitate model distillation.
Has My System Prompt Been Used? Large Language Model Prompt Membership Inference
Prompt engineering has emerged as a powerful technique for optimizing large language models (LLMs) for specific applications, enabling faster prototyping and improved performance, and giving rise to the interest of the community in protecting proprietary system prompts.
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges
Large language models often struggle with length generalization and solving complex problem instances beyond their training distribution.
Easy2Hard-Bench: Standardized Difficulty Labels for Profiling LLM Performance and Generalization
While generalization over tasks from easy to hard is crucial to profile language models (LLMs), the datasets with fine-grained difficulty annotations for each problem across a broad range of complexity are still blank.
Prompt Recovery for Image Generation Models: A Comparative Study of Discrete Optimizers
Recovering natural language prompts for image generation models, solely based on the generated images is a difficult discrete optimization problem.
The CLRS-Text Algorithmic Reasoning Language Benchmark
Three years ago, a similar issue was identified and rectified in the field of neural algorithmic reasoning, with the advent of the CLRS benchmark.
Transformers Can Do Arithmetic with the Right Embeddings
The poor performance of transformers on arithmetic tasks seems to stem in large part from their inability to keep track of the exact position of each digit inside of a large span of digits.
Rethinking LLM Memorization through the Lens of Adversarial Compression
The ACR overcomes the limitations of existing notions of memorization by (i) offering an adversarial view of measuring memorization, especially for monitoring unlearning and compliance; and (ii) allowing for the flexibility to measure memorization for arbitrary strings at a reasonably low compute.
Forcing Diffuse Distributions out of Language Models
Despite being trained specifically to follow user instructions, today's instructiontuned language models perform poorly when instructed to produce random outputs.
Benchmarking ChatGPT on Algorithmic Reasoning
We evaluate ChatGPT's ability to solve algorithm problems from the CLRS benchmark suite that is designed for GNNs.
Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text
Detecting text generated by modern large language models is thought to be hard, as both LLMs and humans can exhibit a wide range of complex behaviors.
TOFU: A Task of Fictitious Unlearning for LLMs
Large language models trained on massive corpora of data from the web can memorize and reproduce sensitive or private data raising both legal and ethical concerns.
Effective Backdoor Mitigation in Vision-Language Models Depends on the Pre-training Objective
We find that CleanCLIP, even with extensive hyperparameter tuning, is ineffective in poison removal when stronger pre-training objectives are used.
NEFTune: Noisy Embeddings Improve Instruction Finetuning
We show that language model finetuning can be improved, sometimes dramatically, with a simple augmentation.
Baseline Defenses for Adversarial Attacks Against Aligned Language Models
We find that the weakness of existing discrete optimizers for text, combined with the relatively high costs of optimization, makes standard adaptive attacks more challenging for LLMs.
A Cookbook of Self-Supervised Learning
Self-supervised learning, dubbed the dark matter of intelligence, is a promising path to advance machine learning.
Reckoning with the Disagreement Problem: Explanation Consensus as a Training Objective
We observe on three datasets that we can train a model with this loss term to improve explanation consensus on unseen data, and see improved consensus between explainers other than those used in the loss term.
Neural Auctions Compromise Bidder Information
It has been shown that neural networks can be used to approximate optimal mechanisms while satisfying the constraints that an auction be strategyproof and individually rational.
Universal Guidance for Diffusion Models
Typical diffusion models are trained to accept a particular form of conditioning, most commonly text, and cannot be conditioned on other modalities without retraining.
Transfer Learning with Deep Tabular Models
In this work, we demonstrate that upstream data gives tabular neural networks a decisive advantage over widely used GBDT models.
End-to-end Algorithm Synthesis with Recurrent Networks: Logical Extrapolation Without Overthinking
Algorithmic extrapolation can be achieved through recurrent systems, which can be iterated many times to solve difficult reasoning problems.
Thinking Deeper With Recurrent Networks: Logical Extrapolation Without Overthinking
Classical machine learning systems perform best when they are trained and tested on the same distribution, and they lack a mechanism to increase model power after training is complete.
Datasets for Studying Generalization from Easy to Hard Examples
We describe new datasets for studying generalization from easy to hard examples.
MetaBalance: High-Performance Neural Networks for Class-Imbalanced Data
Class-imbalanced data, in which some classes contain far more samples than others, is ubiquitous in real-world applications.
Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks
In this work, we show that recurrent networks trained to solve simple problems with few recurrent steps can indeed solve much more complex problems simply by performing additional recurrences during inference.
SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Training
We devise a hybrid deep learning approach to solving tabular data problems.
The Uncanny Similarity of Recurrence and Depth
It is widely believed that deep neural networks contain layer specialization, wherein neural networks extract hierarchical features representing edges and patterns in shallow layers and complete objects in deeper layers.
Just How Toxic is Data Poisoning? A Benchmark for Backdoor and Data Poisoning Attacks
Data poisoning and backdoor attacks manipulate training data in order to cause models to fail during inference.
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance.
Just How Toxic is Data Poisoning? A Unified Benchmark for Backdoor and Data Poisoning Attacks
Data poisoning and backdoor attacks manipulate training data in order to cause models to fail during inference.
Headless Horseman: Adversarial Attacks on Transfer Learning Models
We first demonstrate successful transfer attacks against a victim network using \textit{only} its feature extractor.
Adversarial Attacks on Machine Learning Systems for High-Frequency Trading
Algorithmic trading systems are often completely automated, and deep learning is increasingly receiving attention in this domain.
Truth or Backpropaganda? An Empirical Investigation of Deep Learning Theory
We empirically evaluate common assumptions about neural networks that are widely held by practitioners and theorists alike.
