Search Results for author:
Found 45 papers, 5 papers with code
Performance Guarantees for Schatten-$p$ Quasi-Norm Minimization in Recovery of Low-Rank Matrices
We address some theoretical guarantees for Schatten-$p$ quasi-norm minimization ($p \in (0, 1]$) in recovering low-rank matrices from compressed linear measurements.
Progressive Learning for Systematic Design of Large Neural Networks
The developed network is expected to show good generalization power due to appropriate regularization and use of random weights in the layers.
A Unified Framework for Training Neural Networks
The lack of mathematical tractability of Deep Neural Networks (DNNs) has hindered progress towards having a unified convergence analysis of training algorithms, in the general setting.
Learning Kolmogorov Models for Binary Random Variables
We summarize our recent findings, where we proposed a framework for learning a Kolmogorov model, for a collection of binary random variables.
BIG-bench Machine Learning
Interpretable Machine Learning
+1
Generic Variance Bounds on Estimation and Prediction Errors in Time Series Analysis: An Entropy Perspective
In this paper, we obtain generic bounds on the variances of estimation and prediction errors in time series analysis via an information-theoretic approach.
Decentralized Multi-Task Learning Based on Extreme Learning Machines
We first present the ELM based MTL problem in the centralized setting, which is solved by the proposed MTL-ELM algorithm.
SSFN -- Self Size-estimating Feed-forward Network with Low Complexity, Limited Need for Human Intervention, and Consistent Behaviour across Trials
We design a self size-estimating feed-forward network (SSFN) using a joint optimization approach for estimation of number of layers, number of nodes and learning of weight matrices.
Mobility-aware Content Preference Learning in Decentralized Caching Networks
To determine the caching scheme for decentralized caching networks, the content preference learning problem based on mobility prediction is studied.
Hidden Markov Models for sepsis detection in preterm infants
We explore the use of traditional and contemporary hidden Markov models (HMMs) for sequential physiological data analysis and sepsis prediction in preterm infants.
Conditional Mutual Information Neural Estimator
Several recent works in communication systems have proposed to leverage the power of neural networks in the design of encoders and decoders.
The Convex Information Bottleneck Lagrangian
In this paper, we (i) present a general family of Lagrangians which allow for the exploration of the IB curve in all scenarios; (ii) provide the exact one-to-one mapping between the Lagrange multiplier and the desired compression rate $r$ for known IB curve shapes; and (iii) show we can approximately obtain a specific compression level with the convex IB Lagrangian for both known and unknown IB curve shapes.
A Hybrid Model-based and Data-driven Approach to Spectrum Sharing in mmWave Cellular Networks
Inter-operator spectrum sharing in millimeter-wave bands has the potential of substantially increasing the spectrum utilization and providing a larger bandwidth to individual user equipment at the expense of increasing inter-operator interference.
High-dimensional Neural Feature Design for Layer-wise Reduction of Training Cost
We show that the proposed architecture is norm-preserving and provides an invertible feature vector, and therefore, can be used to reduce the training cost of any other learning method which employs linear projection to estimate the target.
Asynchronous Decentralized Learning of a Neural Network
In this work, we exploit an asynchronous computing framework namely ARock to learn a deep neural network called self-size estimating feedforward neural network (SSFN) in a decentralized scenario.
Upper Bounds on the Generalization Error of Private Algorithms for Discrete Data
In this work, we study the generalization capability of algorithms from an information-theoretic perspective.
A Variational Approach to Privacy and Fairness
In this article, we propose a new variational approach to learn private and/or fair representations.
Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling
The estimation of mutual information (MI) or conditional mutual information (CMI) from a set of samples is a long-standing problem.
Decentralized Beamforming Design for Intelligent Reflecting Surface-enhanced Cell-free Networks
Cell-free networks are considered as a promising distributed network architecture to satisfy the increasing number of users and high rate expectations in beyond-5G systems.
A Low Complexity Decentralized Neural Net with Centralized Equivalence using Layer-wise Learning
We design a low complexity decentralized learning algorithm to train a recently proposed large neural network in distributed processing nodes (workers).
Coded Stochastic ADMM for Decentralized Consensus Optimization with Edge Computing
A class of mini-batch stochastic alternating direction method of multipliers (ADMM) algorithms is explored to develop the distributed learning model.
On Random Subset Generalization Error Bounds and the Stochastic Gradient Langevin Dynamics Algorithm
In this work, we unify several expected generalization error bounds based on random subsets using the framework developed by Hellstr\"om and Durisi [1].
A ReLU Dense Layer to Improve the Performance of Neural Networks
We use a combination of random weights and rectified linear unit (ReLU) activation function to add a ReLU dense (ReDense) layer to the trained neural network such that it can achieve a lower training loss.
Causality Graph of Vehicular Traffic Flow
In an intelligent transportation system, the effects and relations of traffic flow at different points in a network are valuable features which can be exploited for control system design and traffic forecasting.
Tighter expected generalization error bounds via Wasserstein distance
This work presents several expected generalization error bounds based on the Wasserstein distance.
Quadratic Signaling Games with Channel Combining Ratio
Regarding the Nash equilibrium, we explicitly characterize affine equilibria for the single-stage setup and show that the optimal encoder (resp.
Optimization and Control Information Theory Information Theory
A Multi-Objective Optimization Framework for URLLC with Decoding Complexity Constraints
In this paper, we introduce a multi-objective optimization framework for the optimal design of URLLC in the presence of decoding complexity constraints.
Information Theory Information Theory
A Learning-Based Approach to Address Complexity-Reliability Tradeoff in OS Decoders
We show that using artificial neural networks to predict the required order of an ordered statistics based decoder helps in reducing the average complexity and hence the latency of the decoder.
A Model Randomization Approach to Statistical Parameter Privacy
To ensure parameter privacy, we propose a filter design framework which consists of two components: a randomizer and a nonlinear transformation.
Adaptive Stochastic ADMM for Decentralized Reinforcement Learning in Edge Industrial IoT
Alternating direction method of multipliers (ADMM) has a structure that allows for decentralized implementation, and has shown faster convergence than gradient descent based methods.
Generalized Talagrand Inequality for Sinkhorn Distance using Entropy Power Inequality
We evaluate for a wide variety of distributions this term whereas for Gaussian and i. i. d.
Federated Learning over Wireless IoT Networks with Optimized Communication and Resources
Hence, we investigate the problem of jointly optimized communication efficiency and resources for FL over wireless Internet of things (IoT) networks.
Asynchronous Parallel Incremental Block-Coordinate Descent for Decentralized Machine Learning
This paper studies the problem of training an ML model over decentralized systems, where data are distributed over many user devices and the learning algorithm run on-device, with the aim of relaxing the burden at a central entity/server.
An Information-Theoretic Analysis of Bayesian Reinforcement Learning
Building on the framework introduced by Xu and Raginksy [1] for supervised learning problems, we study the best achievable performance for model-based Bayesian reinforcement learning problems.
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization
To date, no "information-theoretic" frameworks for reasoning about generalization error have been shown to establish minimax rates for gradient descent in the setting of stochastic convex optimization.
Thompson Sampling Regret Bounds for Contextual Bandits with sub-Gaussian rewards
In this work, we study the performance of the Thompson Sampling algorithm for Contextual Bandit problems based on the framework introduced by Neu et al. and their concept of lifted information ratio.
Blind Asynchronous Goal-Oriented Detection for Massive Connectivity
Resource allocation and multiple access schemes are instrumental for the success of communication networks, which facilitate seamless wireless connectivity among a growing population of uncoordinated and non-synchronized users.
More PAC-Bayes bounds: From bounded losses, to losses with general tail behaviors, to anytime-validity
Firstly, for losses with a bounded range, we recover a strengthened version of Catoni's bound that holds uniformly for all parameter values.
Towards Optimal Pilot Spacing and Power Control in Multi-Antenna Systems Operating Over Non-Stationary Rician Aging Channels
In particular, when the channel changes rapidly in time, channel aging degrades the performance in terms of spectral efficiency without proper pilot spacing and power control.
Gradient Coding in Decentralized Learning for Evading Stragglers
In this paper, we consider a decentralized learning problem in the presence of stragglers.
Malicious Reconfigurable Intelligent Surfaces: How Impactful can Destructive Beamforming be?
Reconfigurable intelligent surfaces (RISs) have demonstrated significant potential for enhancing communication system performance if properly configured.
Chained Information-Theoretic bounds and Tight Regret Rate for Linear Bandit Problems
This paper studies the Bayesian regret of a variant of the Thompson-Sampling algorithm for bandit problems.
Distributed Learning based on 1-Bit Gradient Coding in the Presence of Stragglers
For this problem, DL methods based on gradient coding have been widely investigated, which redundantly distribute the training data to the workers to guarantee convergence when some workers are stragglers.
Adaptive Coded Federated Learning: Privacy Preservation and Straggler Mitigation
In ACFL, before the training, each device uploads a coded local dataset with additive noise to the central server to generate a global coded dataset under privacy preservation requirements.
A note on generalization bounds for losses with finite moments
Moreover, the paper derives a high-probability PAC-Bayes bound for losses with a bounded variance.
Cooperative Gradient Coding for Collaborative Federated Learning
We investigate federated learning (FL) in the presence of stragglers, with emphasis on wireless scenarios where the power-constrained edge devices collaboratively train a global model on their local datasets and transmit local model updates through fading channels.
