Search Results for author:
Found 13 papers, 6 papers with code
Consensus, dissensus and synergy between clinicians and specialist foundation models in radiology report generation
Radiology reports are an instrumental part of modern medicine, informing key clinical decisions such as diagnosis and treatment.
Why neural networks find simple solutions: the many regularizers of geometric complexity
Using a combination of theoretical arguments and empirical results, we show that many common training heuristics such as parameter norm regularization, spectral norm regularization, flatness regularization, implicit gradient regularization, noise regularization and the choice of parameter initialization all act to control geometric complexity, providing a unifying framework in which to characterize the behavior of deep learning models.
The Geometric Occam's Razor Implicit in Deep Learning
We argue that over-parameterized neural networks trained with stochastic gradient descent are subject to a Geometric Occam's Razor; that is, these networks are implicitly regularized by the geometric model complexity.
Discretization Drift in Two-Player Games
Gradient-based methods for two-player games produce rich dynamics that can solve challenging problems, yet can be difficult to stabilize and understand.
On the Origin of Implicit Regularization in Stochastic Gradient Descent
To interpret this phenomenon we prove that for SGD with random shuffling, the mean SGD iterate also stays close to the path of gradient flow if the learning rate is small and finite, but on a modified loss.
Implicit Gradient Regularization
We call this Implicit Gradient Regularization (IGR) and we use backward error analysis to calculate the size of this regularization.
Learning to Make Analogies by Contrasting Abstract Relational Structure
Here, we study how analogical reasoning can be induced in neural networks that learn to perceive and reason about raw visual data.
Analyzing biological and artificial neural networks: challenges with opportunities for synergy?
We explore opportunities for synergy between the two fields, such as the use of DNNs as in-silico model systems for neuroscience, and how this synergy can lead to new hypotheses about the operating principles of biological neural networks.
Measuring abstract reasoning in neural networks
To succeed at this challenge, models must cope with various generalisation `regimes' in which the training and test data differ in clearly-defined ways.
Spectral Inference Networks: Unifying Deep and Spectral Learning
We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization.
On the importance of single directions for generalization
Finally, we find that class selectivity is a poor predictor of task importance, suggesting not only that networks which generalize well minimize their dependence on individual units by reducing their selectivity, but also that individually selective units may not be necessary for strong network performance.
Cognitive Psychology for Deep Neural Networks: A Shape Bias Case Study
To explore the potential value of these tools, we chose a well-established analysis from developmental psychology that explains how children learn word labels for objects, and applied that analysis to DNNs.
A simple neural network module for relational reasoning
Relational reasoning is a central component of generally intelligent behavior, but has proven difficult for neural networks to learn.
Image Retrieval with Multi-Modal Query
Question Answering
+2
