You need to log in to edit.

You can create a new account if you don't have one.

Or, discuss a change on Slack.

You can create a new account if you don't have one.

Or, discuss a change on Slack.

no code implementations • 8 Feb 2024 • David D. Baek, Ziming Liu, Max Tegmark

We present GenEFT: an effective theory framework for shedding light on the statics and dynamics of neural network generalization, and illustrate it with graph learning examples.

no code implementations • 7 Feb 2024 • Jinyeop Song, Ziming Liu, Max Tegmark, Jeff Gore

A task is usually composite hence can be decomposed into many subtasks, which compete for resources (measured by the number of neurons allocated to subtasks).

1 code implementation • 7 Feb 2024 • Eric J. Michaud, Isaac Liao, Vedang Lad, Ziming Liu, Anish Mudide, Chloe Loughridge, Zifan Carl Guo, Tara Rezaei Kheirkhah, Mateja Vukelić, Max Tegmark

We present MIPS, a novel method for program synthesis based on automated mechanistic interpretability of neural networks trained to perform the desired task, auto-distilling the learned algorithm into Python code.

no code implementations • 25 Jan 2024 • Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, Lee Sharkey, Satyapriya Krishna, Marvin Von Hagen, Silas Alberti, Alan Chan, Qinyi Sun, Michael Gerovitch, David Bau, Max Tegmark, David Krueger, Dylan Hadfield-Menell

The effectiveness of an audit, however, depends on the degree of system access granted to auditors.

no code implementations • 5 Dec 2023 • Isaac Liao, Ziming Liu, Max Tegmark

The hypernetwork is carefully designed such that it can control network complexity, leading to a diverse family of interpretable algorithms ranked by their complexity.

no code implementations • 11 Oct 2023 • Ziming Liu, Mikail Khona, Ila R. Fiete, Max Tegmark

Recurrent neural networks (RNNs) trained on compositional tasks can exhibit functional modularity, in which neurons can be clustered by activity similarity and participation in shared computational subtasks.

1 code implementation • 10 Oct 2023 • Samuel Marks, Max Tegmark

In this work, we curate high-quality datasets of true/false statements and use them to study in detail the structure of LLM representations of truth, drawing on three lines of evidence: 1.

no code implementations • 9 Oct 2023 • Ziming Liu, Ziqian Zhong, Max Tegmark

To do so, we define linear mapping number (LMN) to measure network complexity, which is a generalized version of linear region number for ReLU networks.

no code implementations • 9 Oct 2023 • Peter S. Park, Max Tegmark

Myopic members prioritize their future well-being less than their present well-being, and are thus disinclined to solidarily support current victims today at personal cost, even if this is necessary to counter the shared threat of AI-driven disempowerment.

1 code implementation • 3 Oct 2023 • Wes Gurnee, Max Tegmark

The capabilities of large language models (LLMs) have sparked debate over whether such systems just learn an enormous collection of superficial statistics or a coherent model of the data generation process -- a world model.

no code implementations • 3 Oct 2023 • Ziming Liu, Max Tegmark

Neural scaling laws (NSL) refer to the phenomenon where model performance improves with scale.

no code implementations • 5 Sep 2023 • Max Tegmark, Steve Omohundro

We describe a path to humanity safely thriving with powerful Artificial General Intelligences (AGIs) by building them to provably satisfy human-specified requirements.

1 code implementation • 31 May 2023 • Ziming Liu, Patrick Obin Sturm, Saketh Bharadwaj, Sam Silva, Max Tegmark

Discovering conservation laws for a given dynamical system is important but challenging.

1 code implementation • 4 May 2023 • Ziming Liu, Eric Gan, Max Tegmark

We introduce Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable.

no code implementations • 5 Apr 2023 • Ziming Liu, Di Luo, Yilun Xu, Tommi Jaakkola, Max Tegmark

We introduce a general family, Generative Models from Physical Processes (GenPhys), where we translate partial differential equations (PDEs) describing physical processes to generative models.

1 code implementation • NeurIPS 2023 • Eric J. Michaud, Ziming Liu, Uzay Girit, Max Tegmark

We tentatively find that the frequency at which these quanta are used in the training distribution roughly follows a power law corresponding with the empirical scaling exponent for language models, a prediction of our theory.

1 code implementation • 8 Feb 2023 • Yilun Xu, Ziming Liu, Yonglong Tian, Shangyuan Tong, Max Tegmark, Tommi Jaakkola

The new models reduce to PFGM when $D{=}1$ and to diffusion models when $D{\to}\infty$.

Ranked #1 on Image Generation on FFHQ 64x64 - 4x upscaling

1 code implementation • 24 Oct 2022 • Eric J. Michaud, Ziming Liu, Max Tegmark

We explore unique considerations involved in fitting ML models to data with very high precision, as is often required for science applications.

1 code implementation • 3 Oct 2022 • Ziming Liu, Eric J. Michaud, Max Tegmark

Grokking, the unusual phenomenon for algorithmic datasets where generalization happens long after overfitting the training data, has remained elusive.

1 code implementation • 22 Sep 2022 • Yilun Xu, Ziming Liu, Max Tegmark, Tommi Jaakkola

We interpret the data points as electrical charges on the $z=0$ hyperplane in a space augmented with an additional dimension $z$, generating a high-dimensional electric field (the gradient of the solution to Poisson equation).

Ranked #26 on Image Generation on CIFAR-10

1 code implementation • 20 May 2022 • Ziming Liu, Ouail Kitouni, Niklas Nolte, Eric J. Michaud, Max Tegmark, Mike Williams

We aim to understand grokking, a phenomenon where models generalize long after overfitting their training set.

1 code implementation • 5 Apr 2022 • Andrew K. Tan, Max Tegmark, Isaac L. Chuang

Our goal is to map out and study the Pareto frontier that quantifies this trade-off.

no code implementations • 23 Mar 2022 • Ziming Liu, Varun Madhavan, Max Tegmark

We present a machine learning algorithm that discovers conservation laws from differential equations, both numerically (parametrized as neural networks) and symbolically, ensuring their functional independence (a non-linear generalization of linear independence).

no code implementations • 25 Feb 2022 • Alexander Zlokapa, Andrew K. Tan, John M. Martyn, Ila R. Fiete, Max Tegmark, Isaac L. Chuang

It has been an open question in deep learning if fault-tolerant computation is possible: can arbitrarily reliable computation be achieved using only unreliable neurons?

no code implementations • NeurIPS Workshop AI4Scien 2021 • Ziming Liu, Yunyue Chen, Yuanqi Du, Max Tegmark

Integrating physical inductive biases into machine learning can improve model generalizability.

no code implementations • 20 Sep 2021 • Ziming Liu, Max Tegmark

We present an automated method for finding hidden symmetries, defined as symmetries that become manifest only in a new coordinate system that must be discovered.

no code implementations • 31 Aug 2021 • Samantha D'Alonzo, Max Tegmark

We present an automated method for measuring media bias.

no code implementations • 31 May 2021 • Ziming Liu, Bohan Wang, Qi Meng, Wei Chen, Max Tegmark, Tie-Yan Liu

Energy conservation is a basic physics principle, the breakdown of which often implies new physics.

no code implementations • 9 Nov 2020 • Ziming Liu, Max Tegmark

We present AI Poincar\'e, a machine learning algorithm for auto-discovering conserved quantities using trajectory data from unknown dynamical systems.

2 code implementations • NeurIPS 2020 • Silviu-Marian Udrescu, Andrew Tan, Jiahai Feng, Orisvaldo Neto, Tailin Wu, Max Tegmark

We present an improved method for symbolic regression that seeks to fit data to formulas that are Pareto-optimal, in the sense of having the best accuracy for a given complexity.

no code implementations • 19 May 2020 • Silviu-Marian Udrescu, Max Tegmark

We present a method for unsupervised learning of equations of motion for objects in raw and optionally distorted unlabeled video.

no code implementations • 13 Apr 2020 • Abhik Ghosh, Florent Mertens, Gianni Bernardi, Mário G. Santos, Nicholas S. Kern, Christopher L. Carilli, Trienko L. Grobler, Léon V. E. Koopmans, Daniel C. Jacobs, Adrian Liu, Aaron R. Parsons, Miguel F. Morales, James E. Aguirre, Joshua S. Dillon, Bryna J. Hazelton, Oleg M. Smirnov, Bharat K. Gehlot, Siyanda Matika, Paul Alexander, Zaki S. Ali, Adam P. Beardsley, Roshan K. Benefo, Tashalee S. Billings, Judd D. Bowman, Richard F. Bradley, Carina Cheng, Paul M. Chichura, David R. DeBoer, Eloy de Lera Acedo, Aaron Ewall-Wice, Gcobisa Fadana, Nicolas Fagnoni, Austin F. Fortino, Randall Fritz, Steve R. Furlanetto, Samavarti Gallardo, Brian Glendenning, Deepthi Gorthi, Bradley Greig, Jasper Grobbelaar, Jack Hickish, Alec Josaitis, Austin Julius, Amy S. Igarashi, MacCalvin Kariseb, Saul A. Kohn, Matthew Kolopanis, Telalo Lekalake, Anita Loots, David MacMahon, Lourence Malan, Cresshim Malgas, Matthys Maree, Zachary E. Martinot, Nathan Mathison, Eunice Matsetela, Andrei Mesinger, Abraham R. Neben, Bojan Nikolic, Chuneeta D. Nunhokee, Nipanjana Patra, Samantha Pieterse, Nima Razavi-Ghods, Jon Ringuette, James Robnett, Kathryn Rosie, Raddwine Sell, Craig Smith, Angelo Syce, Max Tegmark, Nithyanandan Thyagarajan, Peter K. G. Williams, Haoxuan Zheng

The key challenge in the observation of the redshifted 21-cm signal from cosmic reionization is its separation from the much brighter foreground emission.

Cosmology and Nongalactic Astrophysics

1 code implementation • 23 Aug 2019 • Max Tegmark, Tailin Wu

The goal of lossy data compression is to reduce the storage cost of a data set $X$ while retaining as much information as possible about something ($Y$) that you care about.

no code implementations • ICLR Workshop LLD 2019 • Tailin Wu, Ian Fischer, Isaac L. Chuang, Max Tegmark

However, in practice, not only is $\beta$ chosen empirically without theoretical guidance, there is also a lack of theoretical understanding between $\beta$, learnability, the intrinsic nature of the dataset and model capacity.

2 code implementations • 27 May 2019 • Silviu-Marian Udrescu, Max Tegmark

A core challenge for both physics and artificial intellicence (AI) is symbolic regression: finding a symbolic expression that matches data from an unknown function.

no code implementations • 30 Apr 2019 • Ricardo Vinuesa, Hossein Azizpour, Iolanda Leite, Madeline Balaam, Virginia Dignum, Sami Domisch, Anna Felländer, Simone Langhans, Max Tegmark, Francesco Fuso Nerini

We find that AI can support the achievement of 128 targets across all SDGs, but it may also inhibit 58 targets.

no code implementations • 9 Feb 2019 • Max Tegmark

A popular approach for predicting the future of dynamical systems involves mapping them into a lower-dimensional "latent space" where prediction is easier.

1 code implementation • 24 Oct 2018 • Tailin Wu, Max Tegmark

We investigate opportunities and challenges for improving unsupervised machine learning using four common strategies with a long history in physics: divide-and-conquer, Occam's razor, unification and lifelong learning.

1 code implementation • 26 Jul 2018 • Tailin Wu, John Peurifoy, Isaac L. Chuang, Max Tegmark

Compared to humans, machine learning models generally require significantly more training examples and fail to extrapolate from experience to solve previously unseen challenges.

1 code implementation • 18 Oct 2017 • John Peurifoy, Yichen Shen, Li Jing, Yi Yang, Fidel Cano-Renteria, Brendan Delacy, Max Tegmark, John D. Joannopoulos, Marin Soljacic

We propose a method to use artificial neural networks to approximate light scattering by multilayer nanoparticles.

Computational Physics Applied Physics Optics

1 code implementation • 8 Jun 2017 • Li Jing, Caglar Gulcehre, John Peurifoy, Yichen Shen, Max Tegmark, Marin Soljačić, Yoshua Bengio

We present a novel recurrent neural network (RNN) based model that combines the remembering ability of unitary RNNs with the ability of gated RNNs to effectively forget redundant/irrelevant information in its memory.

Ranked #7 on Question Answering on bAbi (Accuracy (trained on 1k) metric)

no code implementations • ICLR 2018 • David Rolnick, Max Tegmark

It is well-known that neural networks are universal approximators, but that deeper networks tend in practice to be more powerful than shallower ones.

4 code implementations • ICML 2017 • Li Jing, Yichen Shen, Tena Dubček, John Peurifoy, Scott Skirlo, Yann Lecun, Max Tegmark, Marin Soljačić

Using unitary (instead of general) matrices in artificial neural networks (ANNs) is a promising way to solve the gradient explosion/vanishing problem, as well as to enable ANNs to learn long-term correlations in the data.

no code implementations • 29 Aug 2016 • Henry W. Lin, Max Tegmark, David Rolnick

We show how the success of deep learning could depend not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can frequently be approximated through "cheap learning" with exponentially fewer parameters than generic ones.

no code implementations • 21 Jun 2016 • Henry W. Lin, Max Tegmark

We show that the mutual information between two symbols, as a function of the number of symbols between the two, decays exponentially in any probabilistic regular grammar, but can decay like a power law for a context-free grammar.

3 code implementations • 16 May 2016 • Haoxuan Zheng, Max Tegmark, Joshua S. Dillon, Doyeon A. Kim, Adrian Liu, Abraham Neben, Justin Jonas, Patricia Reich, Wolfgang Reich

We present an improved Global Sky Model (GSM) of diffuse galactic radio emission from 10 MHz to 5 THz, whose uses include foreground modeling for CMB and 21 cm cosmology.

Cosmology and Nongalactic Astrophysics Astrophysics of Galaxies Instrumentation and Methods for Astrophysics

no code implementations • 10 Feb 2016 • Stuart Russell, Daniel Dewey, Max Tegmark

Success in the quest for artificial intelligence has the potential to bring unprecedented benefits to humanity, and it is therefore worthwhile to investigate how to maximize these benefits while avoiding potential pitfalls.

no code implementations • 2 Sep 2014 • Max Tegmark

Relentless progress in artificial intelligence (AI) is increasingly raising concerns that machines will replace humans on the job market, and perhaps altogether.

2 code implementations • 12 Feb 2008 • Angelica de Oliveira-Costa, Max Tegmark, B. M. Gaensler, Justin Jonas, T. L. Landecker, Patricia Reich

Understanding diffuse Galactic radio emission is interesting both in its own right and for minimizing foreground contamination of cosmological measurements.

Optical Character Recognition Optical Character Recognition (OCR)

1 code implementation • 2 Jul 2002 • Max Tegmark, Matias Zaldarriaga

We present a method for measuring the cosmic matter budget without assumptions about speculative Early Universe physics, and for measuring the primordial power spectrum P*(k) non-parametrically, either by combining CMB and LSS information or by using CMB polarization.

no code implementations • 5 Jul 1999 • Max Tegmark

Based on a calculation of neural decoherence rates, we argue that that the degrees of freedom of the human brain that relate to cognitive processes should be thought of as a classical rather than quantum system, i. e., that there is nothing fundamentally wrong with the current classical approach to neural network simulations.

no code implementations • 25 Feb 1997 • Max Tegmark

Some superstring theories have more than one effective low-energy limit, corresponding to classical spacetimes with different dimensionalities.

General Relativity and Quantum Cosmology High Energy Physics - Phenomenology High Energy Physics - Theory

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.