Search Results for author:
Found 32 papers, 15 papers with code
Implicit Bias of Policy Gradient in Linear Quadratic Control: Extrapolation to Unseen Initial States
This paper theoretically studies the implicit bias of policy gradient in terms of extrapolation to unseen initial states.
Data-Driven Strategies for Coping with Incomplete DVL Measurements
Autonomous underwater vehicles are specialized platforms engineered for deep underwater operations.
A-KIT: Adaptive Kalman-Informed Transformer
In this paper, we derive and introduce A-KIT, an adaptive Kalman-informed transformer to learn the varying process noise covariance online.
Inertial Navigation Meets Deep Learning: A Survey of Current Trends and Future Directions
The latter is done by learning some of the fusion filter parameters.
What Makes Data Suitable for a Locally Connected Neural Network? A Necessary and Sufficient Condition Based on Quantum Entanglement
Focusing on locally connected neural networks (a prevalent family of architectures that includes convolutional and recurrent neural networks as well as local self-attention models), we address this problem by adopting theoretical tools from quantum physics.
Set-Transformer BeamsNet for AUV Velocity Forecasting in Complete DVL Outage Scenarios
Our ST-BeamsNet estimated the AUV velocity vector with an 8. 547% speed error, which is 26% better than the MA approach.
On the Ability of Graph Neural Networks to Model Interactions Between Vertices
Formalizing strength of interactions through an established measure known as separation rank, we quantify the ability of certain GNNs to model interaction between a given subset of vertices and its complement, i. e. between the sides of a given partition of input vertices.
Learning Low Dimensional State Spaces with Overparameterized Recurrent Neural Nets
Overparameterization in deep learning typically refers to settings where a trained neural network (NN) has representational capacity to fit the training data in many ways, some of which generalize well, while others do not.
Deep Linear Networks for Matrix Completion -- An Infinite Depth Limit
The deep linear network (DLN) is a model for implicit regularization in gradient based optimization of overparametrized learning architectures.
LiBeamsNet: AUV Velocity Vector Estimation in Situations of Limited DVL Beam Measurements
In such conditions, the vehicle's velocity vector could not be estimated leading to a navigation solution drift and in some situations the AUV is required to abort the mission and return to the surface.
BeamsNet: A data-driven Approach Enhancing Doppler Velocity Log Measurements for Autonomous Underwater Vehicle Navigation
Both simulation and sea experiments were made to validate the proposed learning approach relative to the model-based approach.
Semantic Segmentation in Art Paintings
In this paper, we tackle the problem of semantic segmentation of artistic paintings, an even more challenging task because of a much larger diversity in colors, textures, and shapes and because there are no ground truth annotations available for segmentation.
On the Implicit Bias of Gradient Descent for Temporal Extrapolation
When using recurrent neural networks (RNNs) it is common practice to apply trained models to sequences longer than those seen in training.
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks
In the pursuit of explaining implicit regularization in deep learning, prominent focus was given to matrix and tensor factorizations, which correspond to simplified neural networks.
Continuous vs. Discrete Optimization of Deep Neural Networks
The extent to which it represents gradient descent is an open question in the theory of deep learning.
Implicit Regularization in Tensor Factorization
Recent efforts to unravel the mystery of implicit regularization in deep learning have led to a theoretical focus on matrix factorization -- matrix completion via linear neural network.
Implicit Regularization in Deep Learning May Not Be Explainable by Norms
Mathematically characterizing the implicit regularization induced by gradient-based optimization is a longstanding pursuit in the theory of deep learning.
Implicit Regularization in Deep Matrix Factorization
Efforts to understand the generalization mystery in deep learning have led to the belief that gradient-based optimization induces a form of implicit regularization, a bias towards models of low "complexity."
A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks
We analyze speed of convergence to global optimum for gradient descent training a deep linear neural network (parameterized as $x \mapsto W_N W_{N-1} \cdots W_1 x$) by minimizing the $\ell_2$ loss over whitened data.
âZero-Shotâ Super-Resolution Using Deep Internal Learning
On such images, our method outperforms SotA CNN-based SR methods, as well as previous unsupervised SR methods.
Quantum Entanglement in Deep Learning Architectures
Modern deep learning has enabled unprecedented achievements in various domains.
On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization
The effect of depth on optimization is decoupled from expressiveness by focusing on settings where additional layers amount to overparameterization - linear neural networks, a well-studied model.
"Zero-Shot" Super-Resolution using Deep Internal Learning
On such images, our method outperforms SotA CNN-based SR methods, as well as previous unsupervised SR methods.
Ranked #46 on
Image Super-Resolution
on BSD100 - 4x upscaling
Analysis and Design of Convolutional Networks via Hierarchical Tensor Decompositions
Expressive efficiency refers to the ability of a network architecture to realize functions that require an alternative architecture to be much larger.
Deep Learning and Quantum Entanglement: Fundamental Connections with Implications to Network Design
This description enables us to carry a graph-theoretic analysis of a convolutional network, with which we demonstrate a direct control over the inductive bias of the deep network via its channel numbers, that are related to the min-cut in the underlying graph.
Boosting Dilated Convolutional Networks with Mixed Tensor Decompositions
By introducing and analyzing the concept of mixed tensor decompositions, we prove that interconnecting dilated convolutional networks can lead to expressive efficiency.
Tensorial Mixture Models
Other methods, based on arithmetic circuits and sum-product networks, do allow tractable marginalization, but their performance is challenged by the need to learn the structure of a circuit.
Inductive Bias of Deep Convolutional Networks through Pooling Geometry
In addition to analyzing deep networks, we show that shallow ones support only linear separation ranks, and by this gain insight into the benefit of functions brought forth by depth - they are able to efficiently model strong correlation under favored partitions of the input.
Convolutional Rectifier Networks as Generalized Tensor Decompositions
Second, and more importantly, we show that depth efficiency is weaker with convolutional rectifier networks than it is with convolutional arithmetic circuits.
On the Expressive Power of Deep Learning: A Tensor Analysis
In this work we derive a deep network architecture based on arithmetic circuits that inherently employs locality, sharing and pooling.
Deep SimNets
We present a deep layered architecture that generalizes convolutional neural networks (ConvNets).
SimNets: A Generalization of Convolutional Networks
We present a deep layered architecture that generalizes classical convolutional neural networks (ConvNets).
