Search Results for author:
Found 17 papers, 3 papers with code
Population mobility, well-mixed clustering and disease spread: a look at COVID-19 Spread in the United States and preventive policy insights
The epidemiology of pandemics is classically viewed using geographical and political borders; however, these artificial divisions can result in a misunderstanding of the current epidemiological state within a given region.
Bandit-Driven Batch Selection for Robust Learning under Label Noise
We introduce a novel approach for batch selection in Stochastic Gradient Descent (SGD) training, leveraging combinatorial bandit algorithms.
Differential Equation Scaling Limits of Shaped and Unshaped Neural Networks
Secondly, for an unshaped MLP at initialization, we derive the first order asymptotic correction to the layerwise correlation.
Diffusion on the Probability Simplex
However, the desired continuous nature of the noising process can be at odds with discrete data.
Network Degeneracy as an Indicator of Training Performance: Comparing Finite and Infinite Width Angle Predictions
We observe this degeneracy in the sense that on initialization, inputs tend to become more and more correlated as they travel through the layers of the network.
Dynamic Sparse Training with Structured Sparsity
Dynamic Sparse Training (DST) methods achieve state-of-the-art results in sparse neural network training, matching the generalization of dense models while enabling sparse training and inference.
Depth Degeneracy in Neural Networks: Vanishing Angles in Fully Connected ReLU Networks on Initialization
In this paper, we examine the evolution of the angle between two inputs to a ReLU neural network as a function of the number of layers.
Bounding generalization error with input compression: An empirical study with infinite-width networks
Estimating the Generalization Error (GE) of Deep Neural Networks (DNNs) is an important task that often relies on availability of held-out data.
The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization
In this work, we study the distribution of this random matrix.
The Exponentially Tilted Gaussian Prior for Variational Autoencoders
An important property for deep neural networks is the ability to perform robust out-of-distribution detection on previously unseen data.
Finding Critical Scenarios for Automated Driving Systems: A Systematic Literature Review
The main contributions are: (i) introducing a comprehensive taxonomy for critical scenario identification methods; (ii) giving an overview of the state-of-the-art research based on the taxonomy encompassing 86 papers between 2017 and 2020; and (iii) identifying open issues and directions for further research.
The Future is Log-Gaussian: ResNets and Their Infinite-Depth-and-Width Limit at Initialization
To provide a better approximation, we study ReLU ResNets in the infinite-depth-and-width limit, where both depth and width tend to infinity as their ratio, $d/n$, remains constant.
A Derivative-Free Method for Solving Elliptic Partial Differential Equations with Deep Neural Networks
We introduce a deep neural network based method for solving a class of elliptic partial differential equations.
Finite Depth and Width Corrections to the Neural Tangent Kernel
Moreover, we prove that for such deep and wide networks, the NTK has a non-trivial evolution during training by showing that the mean of its first SGD update is also exponential in the ratio of network depth to width.
Products of Many Large Random Matrices and Gradients in Deep Neural Networks
The fluctuations we find can be thought of as a finite temperature correction to the limit in which first the size and then the number of matrices tend to infinity.
The landscape of the spiked tensor model
We compute the expected number of critical points and local maxima of this objective function and show that it is exponential in the dimensions $n$, and give exact formulas for the exponential growth rate.
Optimal Strategy in "Guess Who?": Beyond Binary Search
Instead, the optimal strategy for the player who trails is to make certain bold plays in an attempt catch up.
Probability Computer Science and Game Theory Optimization and Control 91A15, 60G40, 62L15, 91A60
