Search Results for author:
Found 24 papers, 7 papers with code
Harmonic Retrieval Using Weighted Lifted-Structure Low-Rank Matrix Completion
In this paper, we investigate the problem of recovering the frequency components of a mixture of $K$ complex sinusoids from a random subset of $N$ equally-spaced time-domain samples.
M22: A Communication-Efficient Algorithm for Federated Learning Inspired by Rate-Distortion
This paper proposes ``\emph{${\bf M}$-magnitude weighted $L_{\bf 2}$ distortion + $\bf 2$ degrees of freedom''} (M22) algorithm, a rate-distortion inspired approach to gradient compression for federated training of deep neural networks (DNNs).
Empirical Risk Minimization with Relative Entropy Regularization
Among these properties, the solution to this problem, if it exists, is shown to be a unique probability measure, often mutually absolutely continuous with the reference measure.
Sharp asymptotics on the compression of two-layer neural networks
In this paper, we study the compression of a target two-layer neural network with N nodes into a compressed network with M<N nodes.
How to Attain Communication-Efficient DNN Training? Convert, Compress, Correct
Namely: (i) gradient quantization through floating-point conversion, (ii) lossless compression of the quantized gradient, and (iii) quantization error correction.
A Perspective on Neural Capacity Estimation: Viability and Reliability
These estimators ar referred to as neural mutual information estimation (NMIE)s. NMIEs differ from other approaches as they are data-driven estimators.
Convert, compress, correct: Three steps toward communication-efficient DNN training
In this paper, we introduce a novel algorithm, $\mathsf{CO}_3$, for communication-efficiency distributed Deep Neural Network (DNN) training.
Two-snapshot DOA Estimation via Hankel-structured Matrix Completion
In this paper, we study the problem of estimating the direction of arrival (DOA) using a sparsely sampled uniform linear array (ULA).
Empirical Risk Minimization with Relative Entropy Regularization: Optimality and Sensitivity Analysis
The optimality and sensitivity of the empirical risk minimization problem with relative entropy regularization (ERM-RER) are investigated for the case in which the reference is a sigma-finite measure instead of a probability measure.
Lossy Gradient Compression: How Much Accuracy Can One Bit Buy?
Under this assumption on the DNN gradient distribution, we propose a class of distortion measures to aid the design of quantizers for the compression of the model updates.
DNN gradient lossless compression: Can GenNorm be the answer?
In this paper we argue that, for some networks of practical interest, the gradient entries can be well modelled as having a generalized normal (GenNorm) distribution.
Neural Capacity Estimators: How Reliable Are They?
Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks and without the knowing closed form distribution of the data.
Speeding-Up Back-Propagation in DNN: Approximate Outer Product with Memory
In order to correct for the inherent bias in this approximation, the algorithm retains in memory an accumulation of the outer products that are not used in the approximation.
Wireless Federated Learning with Limited Communication and Differential Privacy
This paper investigates the role of dimensionality reduction in efficient communication and differential privacy (DP) of the local datasets at the remote users for over-the-air computation (AirComp)-based federated learning (FL) model.
Hierarchical Causal Bandit
Causal bandit is a nascent learning model where an agent sequentially experiments in a causal network of variables, in order to identify the reward-maximizing intervention.
Straggler Mitigation through Unequal Error Protection for Distributed Approximate Matrix Multiplication
Large-scale machine learning and data mining methods routinely distribute computations across multiple agents to parallelize processing.
Image Classification
Distributed, Parallel, and Cluster Computing
Information Theory
Information Theory
Multi-Class Unsourced Random Access via Coded Demixing
Unsourced random access (URA) is a recently proposed communication paradigm attuned to machine-driven data transfers.
Information Theory Information Theory
An Exploration of the Heterogeneous Unsourced MAC
In the cluster with higher power, devices transmit using a two-layer superposition modulation.
Information Theory Information Theory
An efficient label-free analyte detection algorithm for time-resolved spectroscopy
Time-resolved spectral techniques play an important analysis tool in many contexts, from physical chemistry to biomedicine.
Efficient Federated Learning over Multiple Access Channel with Differential Privacy Constraints
Accordingly, the objective of the clients is to minimize the training loss subject to (i) rate constraints for reliable communication over the MAC and (ii) DP constraint over the local datasets.
The Information & Mutual Information Ratio for Counting Image Features and Their Matches
The IR is a feature of a single image, while the MIR describes features common across two or more images. We begin by introducing the IR and the MIR and motivate these features in an information theoretical context as the ratio of the self-information of an intensity level over the information contained over the pixels of the same intensity.
Decentralized SGD with Over-the-Air Computation
We study the performance of decentralized stochastic gradient descent (DSGD) in a wireless network, where the nodes collaboratively optimize an objective function using their local datasets.
Compressibility Measures for Affinely Singular Random Vectors
When considering discrete-domain moving-average processes with non-Gaussian excitation noise, the above results allow us to evaluate the block-average RID and DRB, as well as to determine a relationship between these parameters and other existing compressibility measures.
Distributed Convex Optimization With Limited Communications
In this paper, a distributed convex optimization algorithm, termed \emph{distributed coordinate dual averaging} (DCDA) algorithm, is proposed.
