Search Results for author:
Found 26 papers, 8 papers with code
Maximal Initial Learning Rates in Deep ReLU Networks
Training a neural network requires choosing a suitable learning rate, involving a trade-off between speed and effectiveness of convergence.
PhAST: Physics-Aware, Scalable, and Task-specific GNNs for Accelerated Catalyst Design
Catalyst materials play a crucial role in the electrochemical reactions involved in a great number of industrial processes key to this transition, such as renewable energy storage and electrofuel synthesis.
Understanding the Evolution of Linear Regions in Deep Reinforcement Learning
Empirically, we find that the region density increases only moderately throughout training, as measured along fixed trajectories coming from the final policy.
Bugs in the Data: How ImageNet Misrepresents Biodiversity
We find that many of the classes are ill-defined or overlapping, and that 12% of the images are incorrectly labeled, with some classes having >90% of images incorrect.
Physics-Constrained Deep Learning for Climate Downscaling
In order to conserve physical quantities, we develop methods that guarantee physical constraints are satisfied by a deep learning downscaling model while also improving their performance according to traditional metrics.
On Neural Architecture Inductive Biases for Relational Tasks
Recent work has explored how forcing relational representations to remain distinct from sensory representations, as it seems to be the case in the brain, can help artificial systems.
TIML: Task-Informed Meta-Learning for Agriculture
When developing algorithms for data-sparse regions, a natural approach is to use transfer learning from data-rich regions.
ClimART: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models
Numerical simulations of Earth's weather and climate require substantial amounts of computation.
Techniques for Symbol Grounding with SATNet
Many experts argue that the future of artificial intelligence is limited by the field's ability to integrate symbolic logical reasoning into deep learning architectures.
DC3: A learning method for optimization with hard constraints
Large optimization problems with hard constraints arise in many settings, yet classical solvers are often prohibitively slow, motivating the use of deep networks as cheap "approximate solvers."
Geo-Spatiotemporal Features and Shape-Based Prior Knowledge for Fine-grained Imbalanced Data Classification
Fine-grained classification aims at distinguishing between items with similar global perception and patterns, but that differ by minute details.
Deep ReLU Networks Preserve Expected Length
Assessing the complexity of functions computed by a neural network helps us understand how the network will learn and generalize.
Reverse-Engineering Deep ReLU Networks
It has been widely assumed that a neural network cannot be recovered from its outputs, as the network depends on its parameters in a highly nonlinear way.
Identifying Weights and Architectures of Unknown ReLU Networks
The output of a neural network depends on its parameters in a highly nonlinear way, and it is widely assumed that a network's parameters cannot be identified from its outputs.
Tackling Climate Change with Machine Learning
Climate change is one of the greatest challenges facing humanity, and we, as machine learning experts, may wonder how we can help.
Deep ReLU Networks Have Surprisingly Few Activation Patterns
The success of deep networks has been attributed in part to their expressivity: per parameter, deep networks can approximate a richer class of functions than shallow networks.
Complexity of Linear Regions in Deep Networks
It is well-known that the expressivity of a neural network depends on its architecture, with deeper networks expressing more complex functions.
Cross-Classification Clustering: An Efficient Multi-Object Tracking Technique for 3-D Instance Segmentation in Connectomics
Pixel-accurate tracking of objects is a key element in many computer vision applications, often solved by iterated individual object tracking or instance segmentation followed by object matching.
Experience Replay for Continual Learning
We examine this issue in the context of reinforcement learning, in a setting where an agent is exposed to tasks in a sequence.
Measuring and regularizing networks in function space
To optimize a neural network one often thinks of optimizing its parameters, but it is ultimately a matter of optimizing the function that maps inputs to outputs.
How to Start Training: The Effect of Initialization and Architecture
We identify and study two common failure modes for early training in deep ReLU nets.
Deep Learning is Robust to Massive Label Noise
Deep neural networks trained on large supervised datasets have led to impressive results in image classification and other tasks.
Morphological Error Detection in 3D Segmentations
Deep learning algorithms for connectomics rely upon localized classification, rather than overall morphology.
The power of deeper networks for expressing natural functions
It is well-known that neural networks are universal approximators, but that deeper networks tend in practice to be more powerful than shallower ones.
A Multi-Pass Approach to Large-Scale Connectomics
The field of connectomics faces unprecedented "big data" challenges.
Why does deep and cheap learning work so well?
We show how the success of deep learning could depend not only on mathematics but also on physics: although well-known mathematical theorems guarantee that neural networks can approximate arbitrary functions well, the class of functions of practical interest can frequently be approximated through "cheap learning" with exponentially fewer parameters than generic ones.
