Search Results for author:
Found 15 papers, 9 papers with code
When less is more: Simplifying inputs aids neural network understanding
To answer these questions, we need a clear measure of input simplicity (or inversely, complexity), an optimization objective that correlates with simplification, and a framework to incorporate such objective into training and inference.
The Low-Resource Double Bind: An Empirical Study of Pruning for Low-Resource Machine Translation
However, evaluation of the trade-offs incurred by popular compression techniques has been centered on high-resource datasets.
A Tale Of Two Long Tails
As machine learning models are increasingly employed to assist human decision-makers, it becomes critical to communicate the uncertainty associated with these model predictions.
When does loss-based prioritization fail?
Each example is propagated forward and backward through the network the same amount of times, independent of how much the example contributes to the learning protocol.
Randomness In Neural Network Training: Characterizing The Impact of Tooling
However, we also find that the cost of ensuring determinism varies dramatically between neural network architectures and hardware types, e. g., with overhead up to $746\%$, $241\%$, and $196\%$ on a spectrum of widely used GPU accelerator architectures, relative to non-deterministic training.
Keep the Gradients Flowing: Using Gradient Flow to Study Sparse Network Optimization
Based upon these findings, we show that gradient flow in sparse networks can be improved by reconsidering aspects of the architecture design and the training regime.
Characterising Bias in Compressed Models
However, overall accuracy hides disproportionately high errors on a small subset of examples; we call this subset Compression Identified Exemplars (CIE).
The Hardware Lottery
Hardware, systems and algorithms research communities have historically had different incentive structures and fluctuating motivation to engage with each other explicitly.
Estimating Example Difficulty Using Variance of Gradients
In this work, we propose Variance of Gradients (VoG) as a valuable and efficient metric to rank data by difficulty and to surface a tractable subset of the most challenging examples for human-in-the-loop auditing.
Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims
With the recent wave of progress in artificial intelligence (AI) has come a growing awareness of the large-scale impacts of AI systems, and recognition that existing regulations and norms in industry and academia are insufficient to ensure responsible AI development.
Computers and Society
What Do Compressed Deep Neural Networks Forget?
However, this measure of performance conceals significant differences in how different classes and images are impacted by model compression techniques.
Selective Brain Damage: Measuring the Disparate Impact of Model Pruning
Neural network pruning techniques have demonstrated it is possible to remove the majority of weights in a network with surprisingly little degradation to top-1 test set accuracy.
The State of Sparsity in Deep Neural Networks
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet.
A Benchmark for Interpretability Methods in Deep Neural Networks
We propose an empirical measure of the approximate accuracy of feature importance estimates in deep neural networks.
The (Un)reliability of saliency methods
Saliency methods aim to explain the predictions of deep neural networks.
