Search Results for author:
Found 16 papers, 6 papers with code
Understanding Counting in Small Transformers: The Interplay between Attention and Feed-Forward Layers
From a broader perspective, our analysis offers a framework to understand how the interaction of different architectural components of transformer models shapes diverse algorithmic solutions and approximations.
Harnessing Synthetic Datasets: The Role of Shape Bias in Deep Neural Network Generalization
In this study, we investigate how neural networks exhibit shape bias during training on synthetic datasets, serving as an indicator of the synthetic data quality.
Gemtelligence: Accelerating Gemstone classification with Deep Learning
The value of luxury goods, particularly investment-grade gemstones, is greatly influenced by their origin and authenticity, sometimes resulting in differences worth millions of dollars.
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
Autoregressive Transformers adopted in Large Language Models (LLMs) are hard to scale to long sequences.
Uncertainty Quantification in Machine Learning for Engineering Design and Health Prognostics: A Tutorial
In this tutorial, we aim to provide a holistic lens on emerging UQ methods for ML models with a particular focus on neural networks and the applications of these UQ methods in tackling engineering design as well as prognostics and health management problems.
Controllable Neural Symbolic Regression
In symbolic regression, the goal is to find an analytical expression that accurately fits experimental data with the minimal use of mathematical symbols such as operators, variables, and constants.
An SDE for Modeling SAM: Theory and Insights
We study the SAM (Sharpness-Aware Minimization) optimizer which has recently attracted a lot of interest due to its increased performance over more classical variants of stochastic gradient descent.
Cosmology from Galaxy Redshift Surveys with PointNet
In this work, we aim to improve upon two-point statistics by employing a \textit{PointNet}-like neural network to regress the values of the cosmological parameters directly from point cloud data.
Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse
First, we show that rank collapse of the tokens' representations hinders training by causing the gradients of the queries and keys to vanish at initialization.
Dynaformer: A Deep Learning Model for Ageing-aware Battery Discharge Prediction
Electrochemical batteries are ubiquitous devices in our society.
FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control
Generating music with deep neural networks has been an area of active research in recent years.
On the effectiveness of Randomized Signatures as Reservoir for Learning Rough Dynamics
Many finance, physics, and engineering phenomena are modeled by continuous-time dynamical systems driven by highly irregular (stochastic) inputs.
Empirics on the expressiveness of Randomized Signature
Time series analysis is a widespread task in Natural Sciences, Social Sciences and Engineering.
Neural Symbolic Regression that Scales
We procedurally generate an unbounded set of equations, and simultaneously pre-train a Transformer to predict the symbolic equation from a corresponding set of input-output-pairs.
Uncertainty-aware Remaining Useful Life predictor
Remaining Useful Life (RUL) estimation is the problem of inferring how long a certain industrial asset can be expected to operate within its defined specifications.
A Seq2Seq approach to Symbolic Regression
Deep neural networks have proved to be powerful function approximators.
