no code implementations • 25 Feb 2024 • Ilan Price, Nicholas Daultry Ball, Samuel C. H. Lam, Adam C. Jones, Jared Tanner

Inducing and leveraging sparse activations during training and inference is a promising avenue for improving the computational efficiency of deep networks, which is increasingly important as network sizes continue to grow and their application becomes more widespread.

no code implementations • 25 Oct 2023 • Thiziri Nait-Saada, Alireza Naderi, Jared Tanner

The infinitely wide neural network has been proven a useful and manageable mathematical model that enables the understanding of many phenomena appearing in deep learning.

1 code implementation • 13 Oct 2023 • Yuxin Zhang, Lirui Zhao, Mingbao Lin, Yunyun Sun, Yiwu Yao, Xingjia Han, Jared Tanner, Shiwei Liu, Rongrong Ji

Inspired by the Dynamic Sparse Training, DSnoT minimizes the reconstruction error between the dense and sparse LLMs, in the fashion of performing iterative weight pruning-and-growing on top of sparse LLMs.

no code implementations • 31 Jan 2023 • Thiziri Nait Saada, Jared Tanner

The edge-of-chaos dynamics of wide randomly initialized low-rank feedforward networks are analyzed.

no code implementations • 30 Jan 2023 • Vincent P. H. Goverse, Jad Hamdan, Jared Tanner

We investigate properties of neural networks that use both ReLU and $x^2$ as activation functions and build upon previous results to show that both analytic functions and functions in Sobolev spaces can be approximated by such networks of constant depth to arbitrary accuracy, demonstrating optimal order approximation rates across all nonlinear approximators, including standard ReLU networks.

no code implementations • 27 Oct 2022 • Ilan Price, Jared Tanner

The requirement to repeatedly move large feature maps off- and on-chip during inference with convolutional neural networks (CNNs) imposes high costs in terms of both energy and time.

1 code implementation • 8 Mar 2022 • Charles Millard, Mark Chiew, Jared Tanner, Aaron T. Hess, Boris Mailhe

To our knowledge, P-VDAMP is the first algorithm for multi-coil MRI data that obeys a state evolution with accurately tracked parameters.

1 code implementation • 17 May 2021 • Michael Murray, Vinayak Abrol, Jared Tanner

The activation function deployed in a deep neural network has great influence on the performance of the network at initialisation, which in turn has implications for training.

2 code implementations • 12 Feb 2021 • Ilan Price, Jared Tanner

We show that standard training of networks built with these layers, and pruned at initialization, achieves state-of-the-art accuracy for extreme sparsities on a variety of benchmark network architectures and datasets.

no code implementations • 8 Feb 2021 • Jared Tanner, Giuseppe Ughi

The ability to train randomly initialised deep neural networks is known to depend strongly on the variance of the weight matrices and biases as well as the choice of nonlinear activation.

Information Theory Information Theory

no code implementations • 3 Dec 2020 • Giuseppe Ughi, Vinayak Abrol, Jared Tanner

We perform a comprehensive study on the performance of derivative free optimization (DFO) algorithms for the generation of targeted black-box adversarial attacks on Deep Neural Network (DNN) classifiers assuming the perturbation energy is bounded by an $\ell_\infty$ constraint and the number of queries to the network is limited.

2 code implementations • 18 Jul 2020 • Jared Tanner, Simon Vary

This manuscript develops similar guarantees showing that $m\times n$ matrices that can be expressed as the sum of a rank-$r$ matrix and a $s$-sparse matrix can be recovered by computationally tractable methods from $\mathcal{O}(r(m+n-r)+s)\log(mn/s)$ linear measurements.

no code implementations • 10 Apr 2020 • Michael Murray, Jared Tanner

In this paper we consider the problem of designing a decoder to recover a set of sparse codes from their linear measurements alone, that is without access to encoder matrix.

1 code implementation • 3 Mar 2020 • Charles Millard, Aaron T Hess, Boris Mailhé, Jared Tanner

Central to AMP is its "state evolution", which guarantees that the difference between the current estimate and ground truth (the "aliasing") at every iteration obeys a Gaussian distribution that can be fully characterized by a scalar.

1 code implementation • 24 Feb 2020 • Giuseppe Ughi, Vinayak Abrol, Jared Tanner

We demonstrate that model-based derivative free optimisation algorithms can generate adversarial targeted misclassification of deep networks using fewer network queries than non-model-based methods.

no code implementations • 25 Nov 2019 • Ilan Price, Jared Tanner

This paper considers the growth in the length of one-dimensional trajectories as they are passed through deep ReLU neural networks, which, among other things, is one measure of the expressivity of deep networks.

1 code implementation • 4 Nov 2019 • Charles Millard, Aaron T Hess, Boris Mailhé, Jared Tanner

In response we present an algorithm based on Orthogonal AMP constructed specifically for variable density partial Fourier sensing matrices.

no code implementations • 25 Sep 2019 • Ilan Price, Jared Tanner

This paper considers the growth in the length of one-dimensional trajectories as they are passed through deep ReLU neural networks, which, among other things, is one measure of the expressivity of deep networks.

1 code implementation • 25 Aug 2019 • Bernadette J. Stolz, Jared Tanner, Heather A. Harrington, Vidit Nanda

This paper describes the systematic application of local topological methods for detecting interfaces and related anomalies in complicated high-dimensional data.

Algebraic Topology Algebraic Geometry 57N80

no code implementations • 28 Feb 2019 • Giancarlo A. Antonucci, Simon Vary, David Humphreys, Robert A. Lamb, Jonathan Piper, Jared Tanner

Snapshot mosaic multispectral imagery acquires an undersampled data cube by acquiring a single spectral measurement per spatial pixel.

no code implementations • 26 Jun 2018 • Michael Murray, Jared Tanner

Deep Convolutional Sparse Coding (D-CSC) is a framework reminiscent of deep convolutional neural networks (DCNNs), but by omitting the learning of the dictionaries one can more transparently analyse the role of the activation function and its ability to recover activation paths through the layers.

1 code implementation • 15 Aug 2017 • Rodrigo Mendoza-Smith, Jared Tanner

The persistent homology pipeline includes the reduction of a, so-called, boundary matrix.

Algebraic Topology

Cannot find the paper you are looking for? You can
Submit a new open access paper.

Contact us on:
hello@paperswithcode.com
.
Papers With Code is a free resource with all data licensed under CC-BY-SA.