Search Results for author:
Found 12 papers, 6 papers with code
Online Learning and Information Exponents: On The Importance of Batch size, and Time/Complexity Tradeoffs
We study the impact of the batch size $n_b$ on the iteration time $T$ of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target functions of isotropic covariates.
Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions
Here, we investigate the training dynamics of two-layer shallow neural networks trained with gradient-based algorithms, and discuss how they learn pertinent features in multi-index models, that is target functions with low-dimensional relevant directions.
NLP Verification: Towards a General Methodology for Certifying Robustness
We propose a number of practical NLP methods that can help to quantify the effects of the embedding gap; and in particular we propose the metric of falsifiability of semantic subspaces as another fundamental metric to be reported as part of the NLP verification pipeline.
The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents
In particular, multi-pass GD with finite stepsize is found to overcome the limitations of gradient flow and single-pass GD given by the information exponent (Ben Arous et al., 2021) and leap exponent (Abbe et al., 2023) of the target function.
Vehicle: Bridging the Embedding Gap in the Verification of Neuro-Symbolic Programs
Neuro-symbolic programs -- programs containing both machine learning components and traditional symbolic code -- are becoming increasingly widespread.
Escaping mediocrity: how two-layer networks learn hard generalized linear models with SGD
These insights are grounded in the reduction of SGD dynamics to a stochastic process in lower dimensions, where escaping mediocrity equates to calculating an exit time.
ANTONIO: Towards a Systematic Method of Generating NLP Benchmarks for Verification
In particular, many known neural network verification methods that work for computer vision and other numeric datasets do not work for NLP.
From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks
This manuscript investigates the one-pass stochastic gradient descent (SGD) dynamics of a two-layer neural network trained on Gaussian data and labels generated by a similar, though not necessarily identical, target function.
Why Robust Natural Language Understanding is a Challenge
With the proliferation of Deep Machine Learning into real-life applications, a particular property of this technology has been brought to attention: robustness Neural Networks notoriously present low robustness and can be highly sensitive to small input perturbations.
Vehicle: Interfacing Neural Network Verifiers with Interactive Theorem Provers
However, although work has managed to incorporate the results of these verifiers to prove larger properties of individual systems, there is currently no general methodology for bridging the gap between verifiers and interactive theorem provers (ITPs).
Volume-Centred Range Bars: Novel Interpretable Representation of Financial Markets Designed for Machine Learning Applications
In this study, we propose a volume-price-based market representation for making financial time series more suitable for machine learning pipelines.
A Generic Methodology for the Statistically Uniform & Comparable Evaluation of Automated Trading Platform Components
This work is one of the first in applying this rigorous statistically-backed approach to the field of financial markets and we hope this may be a springboard for more research.
