Search Results for author:
Found 73 papers, 36 papers with code
Towards Understanding Distilled Reasoning Models: A Representational Approach
In this paper, we investigate how model distillation impacts the development of reasoning features in large language models (LLMs).
Are Sparse Autoencoders Useful? A Case Study in Sparse Probing
Sparse autoencoders (SAEs) are a popular method for interpreting concepts represented in large language model (LLM) activations.
Harmonic Loss Trains Interpretable AI Models
In this paper, we introduce **harmonic loss** as an alternative to the standard cross-entropy loss for training neural networks and large language models (LLMs).
Language Models Use Trigonometry to Do Addition
By demonstrating that LLMs represent numbers on a helix and manipulate this helix to perform addition, we present the first representation-level explanation of an LLM's mathematical capability.
Low-Rank Adapting Models for Sparse Autoencoders
In these settings, our method reduces the cross entropy loss gap by 30% to 55% when SAEs are inserted during the forward pass.
Open Problems in Mechanistic Interpretability
Mechanistic interpretability aims to understand the computational mechanisms underlying neural networks' capabilities in order to accomplish concrete scientific and engineering goals.
Physics of Skill Learning
We aim to understand physics of skill learning, i. e., how skills are learned in neural networks during training.
Decomposing The Dark Matter of Sparse Autoencoders
Sparse autoencoders (SAEs) are a promising technique for decomposing language model activations into interpretable linear features.
Generalization from Starvation: Hints of Universality in LLM Knowledge Graph Learning
Motivated by interpretability and reliability, we investigate how neural networks represent knowledge during graph learning, We find hints of universality, where equivalent representations are learned across a range of model sizes (from $10^2$ to $10^9$ parameters) and contexts (MLP toy models, LLM in-context learning and LLM training).
The Geometry of Concepts: Sparse Autoencoder Feature Structure
Sparse autoencoders have recently produced dictionaries of high-dimensional vectors corresponding to the universe of concepts represented by large language models.
Efficient Dictionary Learning with Switch Sparse Autoencoders
We present experiments comparing Switch SAEs with other SAE architectures, and find that Switch SAEs deliver a substantial Pareto improvement in the reconstruction vs. sparsity frontier for a given fixed training compute budget.
KAN 2.0: Kolmogorov-Arnold Networks Meet Science
The synergy is bidirectional: science to KAN (incorporating scientific knowledge into KANs), and KAN to science (extracting scientific insights from KANs).
The Remarkable Robustness of LLMs: Stages of Inference?
We find that deleting and swapping interventions retain 72-95\% of the original model's prediction accuracy without fine-tuning, whereas models with more layers exhibit more robustness.
DafnyBench: A Benchmark for Formal Software Verification
We introduce DafnyBench, the largest benchmark of its kind for training and evaluating machine learning systems for formal software verification.
Survival of the Fittest Representation: A Case Study with Modular Addition
To investigate this Survival of the Fittest hypothesis, we conduct a case study on neural networks performing modular addition, and find that these networks' multiple circular representations at different Fourier frequencies undergo such competitive dynamics, with only a few circles surviving at the end.
How Do Transformers "Do" Physics? Investigating the Simple Harmonic Oscillator
We develop four criteria for the use of a method within the simple testbed of linear regression, where our method is $y = wx$ and our intermediate is $w$: (1) Can the intermediate be predicted from hidden states?
Not All Language Model Features Are Linear
Recent work has proposed that language models perform computation by manipulating one-dimensional representations of concepts ("features") in activation space.
Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems
Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts.
OptPDE: Discovering Novel Integrable Systems via AI-Human Collaboration
Integrable partial differential equation (PDE) systems are of great interest in natural science, but are exceedingly rare and difficult to discover.
KAN: Kolmogorov-Arnold Networks
Inspired by the Kolmogorov-Arnold representation theorem, we propose Kolmogorov-Arnold Networks (KANs) as promising alternatives to Multi-Layer Perceptrons (MLPs).
GenEFT: Understanding Statics and Dynamics of Model Generalization via Effective Theory
We present GenEFT: an effective theory framework for shedding light on the statics and dynamics of neural network generalization, and illustrate it with graph learning examples.
A Resource Model For Neural Scaling Law
A task is usually composite hence can be decomposed into many subtasks, which compete for resources (measured by the number of neurons allocated to subtasks).
Opening the AI black box: program synthesis via mechanistic interpretability
We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code.
Black-Box Access is Insufficient for Rigorous AI Audits
External audits of AI systems are increasingly recognized as a key mechanism for AI governance.
Generating Interpretable Networks using Hypernetworks
The hypernetwork is carefully designed such that it can control network complexity, leading to a diverse family of interpretable algorithms ranked by their complexity.
Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networks
Recurrent neural networks (RNNs) trained on compositional tasks can exhibit functional modularity, in which neurons can be clustered by activity similarity and participation in shared computational subtasks.
The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets
In this work, we use high-quality datasets of simple true/false statements to study in detail the structure of LLM representations of truth, drawing on three lines of evidence: 1.
Divide-and-Conquer Dynamics in AI-Driven Disempowerment
Myopic members prioritize their future well-being less than their present well-being, and are thus disinclined to solidarily support current victims today at personal cost, even if this is necessary to counter the shared threat of AI-driven disempowerment.
Grokking as Compression: A Nonlinear Complexity Perspective
To do so, we define linear mapping number (LMN) to measure network complexity, which is a generalized version of linear region number for ReLU networks.
A Neural Scaling Law from Lottery Ticket Ensembling
Neural scaling laws (NSL) refer to the phenomenon where model performance improves with scale.
Language Models Represent Space and Time
The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a set of more coherent and grounded representations that reflect the real world.
Provably safe systems: the only path to controllable AGI
We describe a path to humanity safely thriving with powerful Artificial General Intelligences (AGIs) by building them to provably satisfy human-specified requirements.
The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks
Do neural networks, trained on well-understood algorithmic tasks, reliably rediscover known algorithms for solving those tasks?
Discovering New Interpretable Conservation Laws as Sparse Invariants
Discovering conservation laws for a given dynamical system is important but challenging.
Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability
We introduce Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable.
GenPhys: From Physical Processes to Generative Models
We introduce a general family, Generative Models from Physical Processes (GenPhys), where we translate partial differential equations (PDEs) describing physical processes to generative models.
The Quantization Model of Neural Scaling
We tentatively find that the frequency at which these quanta are used in the training distribution roughly follows a power law corresponding with the empirical scaling exponent for language models, a prediction of our theory.
PFGM++: Unlocking the Potential of Physics-Inspired Generative Models
The new models reduce to PFGM when $D{=}1$ and to diffusion models when $D{\to}\infty$.
Ranked #1 on
Image Generation
on FFHQ 64x64 - 4x upscaling
Precision Machine Learning
We explore unique considerations involved in fitting ML models to data with very high precision, as is often required for science applications.
Omnigrok: Grokking Beyond Algorithmic Data
Grokking, the unusual phenomenon for algorithmic datasets where generalization happens long after overfitting the training data, has remained elusive.
Poisson Flow Generative Models
We interpret the data points as electrical charges on the $z=0$ hyperplane in a space augmented with an additional dimension $z$, generating a high-dimensional electric field (the gradient of the solution to Poisson equation).
Ranked #19 on
Image Generation
on CIFAR-10
Towards Understanding Grokking: An Effective Theory of Representation Learning
We aim to understand grokking, a phenomenon where models generalize long after overfitting their training set.
Pareto-optimal clustering with the primal deterministic information bottleneck
Our goal is to map out and study the Pareto frontier that quantifies this trade-off.
AI Poincaré 2.0: Machine Learning Conservation Laws from Differential Equations
We present a machine learning algorithm that discovers conservation laws from differential equations, both numerically (parametrized as neural networks) and symbolically, ensuring their functional independence (a non-linear generalization of linear independence).
Fault-Tolerant Neural Networks from Biological Error Correction Codes
It has been an open question in deep learning if fault-tolerant computation is possible: can arbitrarily reliable computation be achieved using only unreliable neurons?
Physics-Augmented Learning: A New Paradigm Beyond Physics-Informed Learning
Integrating physical inductive biases into machine learning can improve model generalizability.
Machine-learning hidden symmetries
We present an automated method for finding hidden symmetries, defined as symmetries that become manifest only in a new coordinate system that must be discovered.
Machine-Learning Non-Conservative Dynamics for New-Physics Detection
Energy conservation is a basic physics principle, the breakdown of which often implies new physics.
AI Poincaré: Machine Learning Conservation Laws from Trajectories
We present AI Poincar\'e, a machine learning algorithm for auto-discovering conserved quantities using trajectory data from unknown dynamical systems.
AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity
We present an improved method for symbolic regression that seeks to fit data to formulas that are Pareto-optimal, in the sense of having the best accuracy for a given complexity.
Symbolic Pregression: Discovering Physical Laws from Distorted Video
We present a method for unsupervised learning of equations of motion for objects in raw and optionally distorted unlabeled video.
Foreground modelling via Gaussian process regression: an application to HERA data
The key challenge in the observation of the redshifted 21-cm signal from cosmic reionization is its separation from the much brighter foreground emission.
Cosmology and Nongalactic Astrophysics
Pareto-optimal data compression for binary classification tasks
The goal of lossy data compression is to reduce the storage cost of a data set $X$ while retaining as much information as possible about something ($Y$) that you care about.
Learnability for the Information Bottleneck
However, in practice, not only is $\beta$ chosen empirically without theoretical guidance, there is also a lack of theoretical understanding between $\beta$, learnability, the intrinsic nature of the dataset and model capacity.
AI Feynman: a Physics-Inspired Method for Symbolic Regression
A core challenge for both physics and artificial intellicence (AI) is symbolic regression: finding a symbolic expression that matches data from an unknown function.
The role of artificial intelligence in achieving the Sustainable Development Goals
We find that AI can support the achievement of 128 targets across all SDGs, but it may also inhibit 58 targets.
Latent Representations of Dynamical Systems: When Two is Better Than One
A popular approach for predicting the future of dynamical systems involves mapping them into a lower-dimensional "latent space" where prediction is easier.
Toward an AI Physicist for Unsupervised Learning
We investigate opportunities and challenges for improving unsupervised machine learning using four common strategies with a long history in physics: divide-and-conquer, Occam's razor, unification and lifelong learning.
Meta-learning autoencoders for few-shot prediction
Compared to humans, machine learning models generally require significantly more training examples and fail to extrapolate from experience to solve previously unseen challenges.
Nanophotonic Particle Simulation and Inverse Design Using Artificial Neural Networks
We propose a method to use artificial neural networks to approximate light scattering by multilayer nanoparticles.
Computational Physics Applied Physics Optics
Gated Orthogonal Recurrent Units: On Learning to Forget
We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/irrelevant information in its memory.
Ranked #7 on
Question Answering
on bAbi
(Accuracy (trained on 1k) metric)
The power of deeper networks for expressing natural functions
It is well-known that neural networks are universal approximators, but that deeper networks tend in practice to be more powerful than shallower ones.
Tunable Efficient Unitary Neural Networks (EUNN) and their application to RNNs
Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data.
Why does deep and cheap learning work so well?
We show how the success of deep learning could depend not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can frequently be approximated through "cheap learning" with exponentially fewer parameters than generic ones.
Criticality in Formal Languages and Statistical Physics
We show that the mutual information between two symbols, as a function of the number of symbols between the two, decays exponentially in any probabilistic regular grammar, but can decay like a power law for a context-free grammar.
An Improved Model of Diffuse Galactic Radio Emission from 10 MHz to 5 THz
We present an improved Global Sky Model (GSM) of diffuse galactic radio emission from 10 MHz to 5 THz, whose uses include foreground modeling for CMB and 21 cm cosmology.
Cosmology and Nongalactic Astrophysics Astrophysics of Galaxies Instrumentation and Methods for Astrophysics
Research Priorities for Robust and Beneficial Artificial Intelligence
Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to investigate how to maximize these benefits while avoiding potential pitfalls.
Friendly Artificial Intelligence: the Physics Challenge
Relentless progress in artificial intelligence (AI) is increasingly raising concerns that machines will replace humans on the job market, and perhaps altogether.
A model of diffuse Galactic Radio Emission from 10 MHz to 100 GHz
Understanding diffuse Galactic radio emission is interesting both in its own right and for minimizing foreground contamination of cosmological measurements.
Optical Character Recognition
Optical Character Recognition (OCR)
Separating the Early Universe from the Late Universe: cosmological parameter estimation beyond the black box
We present a method for measuring the cosmic matter budget without assumptions about speculative Early Universe physics, and for measuring the primordial power spectrum P*(k) non-parametrically, either by combining CMB and LSS information or by using CMB polarization.
The importance of quantum decoherence in brain processes
Based on a calculation of neural decoherence rates, we argue that that the degrees of freedom of the human brain that relate to cognitive processes should be thought of as a classical rather than quantum system, i. e., that there is nothing fundamentally wrong with the current classical approach to neural network simulations.
On the dimensionality of spacetime
Some superstring theories have more than one effective low-energy limit, corresponding to classical spacetimes with different dimensionalities.
General Relativity and Quantum Cosmology High Energy Physics - Phenomenology High Energy Physics - Theory
