Search Results for author: Michael R. DeWeese

Found 11 papers, 6 papers with code

Reverse Engineering the Neural Tangent Kernel

2 code implementations6 Jun 2021 James B. Simon, Sajant Anand, Michael R. DeWeese

The development of methods to guide the design of neural networks is an important open challenge for deep learning theory.

Learning Theory Translation

Engineered swift equilibration for arbitrary geometries

no code implementations15 Dec 2020 Adam G. Frim, Adrianne Zhong, Shi-Fan Chen, Dibyendu Mandal, Michael R. DeWeese

Engineered swift equilibration (ESE) is a class of driving protocols that enforce an equilibrium distribution with respect to external control parameters at the beginning and end of rapid state transformations of open, classical non-equilibrium systems.

Statistical Mechanics

A new method for parameter estimation in probabilistic models: Minimum probability flow

1 code implementation17 Jul 2020 Jascha Sohl-Dickstein, Peter Battaglino, Michael R. DeWeese

Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function.

Critical Point-Finding Methods Reveal Gradient-Flat Regions of Deep Network Losses

no code implementations23 Mar 2020 Charles G. Frye, James Simon, Neha S. Wadia, Andrew Ligeralde, Michael R. DeWeese, Kristofer E. Bouchard

Despite the fact that the loss functions of deep neural networks are highly non-convex, gradient-based optimization algorithms converge to approximately the same performance from many random initial points.

Second-order methods

Design of optical neural networks with component imprecisions

1 code implementation13 Dec 2019 Michael Y. -S. Fang, Sasikanth Manipatruni, Casimir Wierzynski, Amir Khosrowshahi, Michael R. DeWeese

For the benefit of designing scalable, fault resistant optical neural networks (ONNs), we investigate the effects architectural designs have on the ONNs' robustness to imprecise components.

Numerically Recovering the Critical Points of a Deep Linear Autoencoder

no code implementations29 Jan 2019 Charles G. Frye, Neha S. Wadia, Michael R. DeWeese, Kristofer E. Bouchard

Numerically locating the critical points of non-convex surfaces is a long-standing problem central to many fields.

A Markov Jump Process for More Efficient Hamiltonian Monte Carlo

no code implementations13 Sep 2015 Andrew B. Berger, Mayur Mudigonda, Michael R. DeWeese, Jascha Sohl-Dickstein

In most sampling algorithms, including Hamiltonian Monte Carlo, transition rates between states correspond to the probability of making a transition in a single time step, and are constrained to be less than or equal to 1.

Time Resolution Dependence of Information Measures for Spiking Neurons: Atoms, Scaling, and Universality

no code implementations18 Apr 2015 Sarah E. Marzen, Michael R. DeWeese, James P. Crutchfield

A first step towards that larger goal is to develop information measures for individual output processes, including information generation (entropy rate), stored information (statistical complexity), predictable information (excess entropy), and active information accumulation (bound information rate).

Model Selection

Hamiltonian Monte Carlo Without Detailed Balance

2 code implementations18 Sep 2014 Jascha Sohl-Dickstein, Mayur Mudigonda, Michael R. DeWeese

We present a method for performing Hamiltonian Monte Carlo that largely eliminates sample rejection for typical hyperparameters.

Minimum Probability Flow Learning

1 code implementation25 Jun 2009 Jascha Sohl-Dickstein, Peter Battaglino, Michael R. DeWeese

Fitting probabilistic models to data is often difficult, due to the general intractability of the partition function and its derivatives.

Cannot find the paper you are looking for? You can Submit a new open access paper.