Search Results for author:
Found 30 papers, 2 papers with code
Position: Model Collapse Does Not Mean What You Think
To assess how significantly different interpretations of model collapse threaten future generative models, we posit what we believe are realistic conditions for studying model collapse and then conduct a rigorous assessment of the literature's methodologies through this lens.
No, of course I can! Refusal Mechanisms Can Be Exploited Using Harmless Fine-Tuning Data
Against open-source models protected by simple defenses, we improve ASRs by an average of 3. 25 times compared to the best performing previous attacks that use only harmless data.
Correlating and Predicting Human Evaluations of Language Models from Natural Language Processing Benchmarks
The explosion of high-performing conversational language models (LMs) has spurred a shift from classic natural language processing (NLP) benchmarks to expensive, time-consuming and noisy human evaluations - yet the relationship between these two evaluation strategies remains hazy.
How Do Large Language Monkeys Get Their Power (Laws)?
Recent research across mathematical problem solving, proof assistant programming and multimodal jailbreaking documents a striking finding: when (multimodal) language model tackle a suite of tasks with multiple attempts per task -- succeeding if any attempt is correct -- then the negative log of the average success rate scales a power law in the number of attempts.
Best-of-N Jailbreaking
We find that BoN Jailbreaking achieves high attack success rates (ASRs) on closed-source language models, such as 89% on GPT-4o and 78% on Claude 3. 5 Sonnet when sampling 10, 000 augmented prompts.
Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach
Defending large language models against jailbreaks so that they never engage in a broadly-defined set of forbidden behaviors is an open problem.
ZIP-FIT: Embedding-Free Data Selection via Compression-Based Alignment
Data selection is crucial for optimizing language model (LM) performance on specific tasks, yet most existing methods fail to effectively consider the target task distribution.
Collapse or Thrive? Perils and Promises of Synthetic Data in a Self-Generating World
Others see collapse as avoidable; in an `{\it accumulate}' scenario, a sequence of models is trained, but each training uses all real and synthetic data generated so far.
Failures to Find Transferable Image Jailbreaks Between Vision-Language Models
These results stand in stark contrast to existing evidence of universal and transferable text jailbreaks against language models and transferable adversarial attacks against image classifiers, suggesting that VLMs may be more robust to gradient-based transfer attacks.
Uncovering Latent Memories: Assessing Data Leakage and Memorization Patterns in Frontier AI Models
Frontier AI systems are making transformative impacts across society, but such benefits are not without costs: models trained on web-scale datasets containing personal and private data raise profound concerns about data privacy and security.
In-Context Learning of Energy Functions
In-context learning is a powerful capability of certain machine learning models that arguably underpins the success of today's frontier AI models.
Quantifying Variance in Evaluation Benchmarks
Evaluation benchmarks are the cornerstone of measuring capabilities of large language models (LLMs), as well as driving progress in said capabilities.
Towards an Improved Understanding and Utilization of Maximum Manifold Capacity Representations
We then leverage tools from information theory to show that such embeddings maximize a well-known lower bound on mutual information between views, thereby connecting the geometric perspective of MMCR to the information-theoretic perspective commonly discussed in MVSSL.
Why Has Predicting Downstream Capabilities of Frontier AI Models with Scale Remained Elusive?
We then reveal the mechanism causing this degradation: downstream metrics require comparing the correct choice against a small number of specific incorrect choices, meaning accurately predicting downstream capabilities requires predicting not just how probability mass concentrates on the correct choice with scale, but also how probability mass fluctuates on specific incorrect choices with scale.
Is Model Collapse Inevitable? Breaking the Curse of Recursion by Accumulating Real and Synthetic Data
The proliferation of generative models, combined with pretraining on web-scale data, raises a timely question: what happens when these models are trained on their own generated outputs?
Bridging Associative Memory and Probabilistic Modeling
Based on the observation that associative memory's energy functions can be seen as probabilistic modeling's negative log likelihoods, we build a bridge between the two that enables useful flow of ideas in both directions.
Investigating Data Contamination for Pre-training Language Models
Language models pre-trained on web-scale corpora demonstrate impressive capabilities on diverse downstream tasks.
Disentangling Fact from Grid Cell Fiction in Trained Deep Path Integrators
Work on deep learning-based models of grid cells suggests that grid cells generically and robustly arise from optimizing networks to path integrate, i. e., track one's spatial position by integrating self-velocity signals.
What Causes Polysemanticity? An Alternative Origin Story of Mixed Selectivity from Incidental Causes
Using a combination of theory and experiments, we show that incidental polysemanticity can arise due to multiple reasons including regularization and neural noise; this incidental polysemanticity occurs because random initialization can, by chance alone, initially assign multiple features to the same neuron, and the training dynamics then strengthen such overlap.
Testing Assumptions Underlying a Unified Theory for the Origin of Grid Cells
At both the population and single-cell levels, we find evidence suggesting that neither of the assumptions are likely true in biological neural representations.
Pretraining on the Test Set Is All You Need
Inspired by recent work demonstrating the promise of smaller Transformer-based language models pretrained on carefully curated data, we supercharge such approaches by investing heavily in curating a novel, high quality, non-synthetic data mixture based solely on evaluation benchmarks.
FACADE: A Framework for Adversarial Circuit Anomaly Detection and Evaluation
We present FACADE, a novel probabilistic and geometric framework designed for unsupervised mechanistic anomaly detection in deep neural networks.
Deceptive Alignment Monitoring
As the capabilities of large machine learning models continue to grow, and as the autonomy afforded to such models continues to expand, the spectre of a new adversary looms: the models themselves.
Invalid Logic, Equivalent Gains: The Bizarreness of Reasoning in Language Model Prompting
We find that the logically \textit{invalid} reasoning prompts do indeed achieve similar performance gains on BBH tasks as logically valid reasoning prompts.
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Yet, while the literature on the trustworthiness of GPT models remains limited, practitioners have proposed employing capable GPT models for sensitive applications such as healthcare and finance -- where mistakes can be costly.
Are Emergent Abilities of Large Language Models a Mirage?
Recent work claims that large language models display emergent abilities, abilities not present in smaller-scale models that are present in larger-scale models.
Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle
Double descent is a surprising phenomenon in machine learning, in which as the number of model parameters grows relative to the number of data, test error drops as models grow ever larger into the highly overparameterized (data undersampled) regime.
Streaming Inference for Infinite Non-Stationary Clustering
Learning from a continuous stream of non-stationary data in an unsupervised manner is arguably one of the most common and most challenging settings facing intelligent agents.
An Algorithmic Theory of Metacognition in Minds and Machines
To the machine learning community, our proposed theory creates a novel interaction between the Actor and Critic in Actor-Critic agents and notes a novel connection between RL and Bayesian Optimization.
Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice
Third, the geometry of RNN dynamics reflects an induced coupling between the two separate inference processes necessary to solve the task.
