Search Results for author:
Found 14 papers, 4 papers with code
Optimization Over Trained Neural Networks: Taking a Relaxing Walk
Besides training, mathematical optimization is also used in deep learning to model and solve formulations over trained neural networks for purposes such as verification, compression, and optimization with learned constraints.
Computational Tradeoffs of Optimization-Based Bound Tightening in ReLU Networks
The use of Mixed-Integer Linear Programming (MILP) models to represent neural networks with Rectified Linear Unit (ReLU) activations has become increasingly widespread in the last decade.
When Deep Learning Meets Polyhedral Theory: A Survey
In the past decade, deep learning became the prevalent methodology for predictive modeling thanks to the remarkable accuracy of deep neural networks in tasks such as computer vision and natural language processing.
Getting Away with More Network Pruning: From Sparsity to Geometry and Linear Regions
One surprising trait of neural networks is the extent to which their connections can be pruned with little to no effect on accuracy.
Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm
In this work, we study such relative distortions in recall by hypothesizing an intensification effect that is inherent to the model.
Optimal Decision Diagrams for Classification
Decision diagrams for classification have some notable advantages over decision trees, as their internal connections can be determined at training time and their width is not bound to grow exponentially with their depth.
The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks
We propose a tractable heuristic for solving the combinatorial extension of OBS, in which we select weights for simultaneous removal, as well as a systematic update of the remaining weights.
Training Thinner and Deeper Neural Networks: Jumpstart Regularization
Neural networks are more expressive when they have multiple layers.
Scaling Up Exact Neural Network Compression by ReLU Stability
We can compress a rectifier network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable.
Lossless Compression of Deep Neural Networks
Deep neural networks have been successful in many predictive modeling tasks, such as image and language recognition, where large neural networks are often used to obtain good accuracy.
Equivalent and Approximate Transformations of Deep Neural Networks
On the practical side, we show that certain rectified linear units (ReLUs) can be safely removed from a network if they are always active or inactive for any valid input.
Empirical Bounds on Linear Regions of Deep Rectifier Networks
Our first contribution is a method to sample the activation patterns defined by ReLUs using universal hash functions.
How Could Polyhedral Theory Harness Deep Learning?
The holy grail of deep learning is to come up with an automatic method to design optimal architectures for different applications.
Bounding and Counting Linear Regions of Deep Neural Networks
We investigate the complexity of deep neural networks (DNN) that represent piecewise linear (PWL) functions.
