no code implementations • 8 Aug 2024 • Alexander Atanasov, Jacob A. Zavatone-Veth, Cengiz Pehlevan

Recent years have seen substantial advances in our understanding of high-dimensional ridge regression, but existing theories assume that training examples are independent.

no code implementations • 7 Aug 2024 • Jacob A. Zavatone-Veth, Cengiz Pehlevan

We investigate the behavior of the Nadaraya-Watson kernel smoothing estimator in high dimensions using its relationship to the random energy model and to dense associative memories.

1 code implementation • 27 May 2024 • Sheng Yang, Jacob A. Zavatone-Veth, Cengiz Pehlevan

To this end, we propose a new spectral regularizer for representation learning that encourages black-box adversarial robustness in downstream classification tasks.

1 code implementation • 20 May 2024 • Yue M. Lu, Mary I. Letey, Jacob A. Zavatone-Veth, Anindita Maiti, Cengiz Pehlevan

Transformers have a remarkable ability to learn and execute tasks based on examples provided within the input itself, without explicit prior training.

1 code implementation • 1 May 2024 • Alexander Atanasov, Jacob A. Zavatone-Veth, Cengiz Pehlevan

This paper presents a succinct derivation of the training and generalization performance of a variety of high-dimensional ridge regression models using the basic tools of random matrix theory and free probability.

1 code implementation • NeurIPS 2023 • Hamza Tahir Chaudhry, Jacob A. Zavatone-Veth, Dmitry Krotov, Cengiz Pehlevan

Sequence memory is an essential attribute of natural and artificial intelligence that enables agents to encode, store, and retrieve complex sequences of stimuli and actions.

1 code implementation • 26 Jan 2023 • Jacob A. Zavatone-Veth, Sheng Yang, Julian A. Rubinfien, Cengiz Pehlevan

This holds in deep networks trained on high-dimensional image classification tasks, and even in self-supervised representation learning.

no code implementations • 1 Mar 2022 • Jacob A. Zavatone-Veth, William L. Tong, Cengiz Pehlevan

Moreover, we show that the leading-order correction to the kernel-limit learning curve cannot distinguish between random feature models and deep networks in which all layers are trained.

no code implementations • 12 Jan 2022 • Jacob A. Zavatone-Veth, Cengiz Pehlevan

In this short note, we reify the connection between work on the storage capacity problem in wide two-layer treelike neural networks and the rapidly-growing body of literature on kernel limits of wide neural networks.

no code implementations • 22 Dec 2021 • Ana I. Gonçalves, Jacob A. Zavatone-Veth, Megan R. Carey, Damon A. Clark

Our understanding of the neural basis of locomotor behavior can be informed by careful quantification of animal movement.

no code implementations • 23 Nov 2021 • Jacob A. Zavatone-Veth, Cengiz Pehlevan

Inference in deep Bayesian neural networks is only fully understood in the infinite-width limit, where the posterior flexibility afforded by increased depth washes out and the posterior predictive collapses to a shallow Gaussian process.

1 code implementation • NeurIPS 2021 • Jacob A. Zavatone-Veth, Abdulkadir Canatar, Benjamin S. Ruben, Cengiz Pehlevan

However, our theoretical understanding of how the learned hidden layer representations of finite networks differ from the fixed representations of infinite networks remains incomplete.

1 code implementation • NeurIPS 2021 • Jacob A. Zavatone-Veth, Cengiz Pehlevan

For deep linear networks, the prior has a simple expression in terms of the Meijer $G$-function.

no code implementations • 21 Jul 2020 • Jacob A. Zavatone-Veth, Cengiz Pehlevan

Though a wide variety of nonlinear activation functions have been proposed for use in artificial neural networks, a detailed understanding of their role in determining the expressive power of a network has not emerged.

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.