Search Results for author:
Found 52 papers, 17 papers with code
A Unified Convergence Analysis of Block Successive Minimization Methods for Nonsmooth Optimization
The block coordinate descent (BCD) method is widely used for minimizing a continuous function f of several block variables.
Optimization and Control
On the Linear Convergence of the Proximal Gradient Method for Trace Norm Regularization
Motivated by various applications in machine learning, the problem of minimizing a convex smooth loss function with trace norm regularization has received much attention lately.
Parallel Successive Convex Approximation for Nonsmooth Nonconvex Optimization
In this work, we propose an inexact parallel BCD approach where at each iteration, a subset of the variables is updated in parallel by minimizing convex approximations of the original objective function.
Optimization and Control
Guaranteed Matrix Completion via Non-convex Factorization
In this paper, we establish a theoretical guarantee for the factorization formulation to correctly recover the underlying low-rank matrix.
Parallel Direction Method of Multipliers
In this paper, we propose a parallel randomized block coordinate method named Parallel Direction Method of Multipliers (PDMM) to solve the optimization problems with multi-block linear constraints.
Computational Intractability of Dictionary Learning for Sparse Representation
In this paper we consider the dictionary learning problem for sparse representation.
Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications
The heavy-tailed distributions of corrupted outliers and singular values of all channels in low-level vision have proven effective priors for many applications such as background modeling, photometric stereo and image alignment.
Inexact Block Coordinate Descent Algorithms for Nonsmooth Nonconvex Optimization
Interestingly, when the approximation subproblem is solved by a descent algorithm, convergence of a subsequence to a stationary point is still guaranteed even if the approximation subproblem is solved inexactly by terminating the descent algorithm after a finite number of iterations.
A General $\mathcal{O}(n^2)$ Hyper-Parameter Optimization for Gaussian Process Regression with Cross-Validation and Non-linearly Constrained ADMM
Hyper-parameter optimization remains as the core issue of Gaussian process (GP) for machine learning nowadays.
Optimally Combining Classifiers for Semi-Supervised Learning
In this paper, we find these two methods have complementary properties and larger diversity, which motivates us to propose a new semi-supervised learning method that is able to adaptively combine the strengths of Xgboost and transductive support vector machine.
Improved RIP-Based Bounds for Guaranteed Performance of two Compressed Sensing Algorithms
The purpose of this paper is to affirmatively answer this question and rigorously show that the RIP-based bounds for guaranteed performance of IHT can be significantly improved to $ \delta_{3k} < (\sqrt{5}-1)/2 \approx 0. 618, $ and the bound for CoSaMP can be improved and pushed to $ \delta_{4k}< 0. 5102.
Pushing The Limit of Type I Codebook For FDD Massive MIMO Beamforming: A Channel Covariance Reconstruction Approach
There is a fundamental trade-off between the channel representation resolution of codebooks and the overheads of feedback communications in the fifth generation new radio (5G NR) frequency division duplex (FDD) massive multiple-input and multiple-output (MIMO) systems.
A Single-Loop Smoothed Gradient Descent-Ascent Algorithm for Nonconvex-Concave Min-Max Problems
We prove that the stabilized GDA algorithm can achieve an $O(1/\epsilon^2)$ iteration complexity for minimizing the pointwise maximum of a finite collection of nonconvex functions.
Distributed Stochastic Consensus Optimization with Momentum for Nonconvex Nonsmooth Problems
While many distributed optimization algorithms have been proposed for solving smooth or convex problems over the networks, few of them can handle non-convex and non-smooth problems.
Disentangling Adversarial Robustness in Directions of the Data Manifold
Standard adversarial training increases model robustness by extending the data manifold boundary in the small variance directions, while on the contrary, adversarial training with generative adversarial examples increases model robustness by extending the data manifold boundary in the large variance directions.
An efficient linear programming rounding-and-refinement algorithm for large-scale network slicing problem
In this paper, we consider the network slicing problem which attempts to map multiple customized virtual network requests (also called services) to a common shared network infrastructure and allocate network resources to meet diverse service requirements, and propose an efficient two-stage algorithm for solving this NP-hard problem.
Networking and Internet Architecture Information Theory Signal Processing Information Theory Optimization and Control
Resource Reservation in Backhaul and Radio Access Network with Uncertain User Demands
In this paper, we consider the problem of joint resource reservation in the backhaul and Radio Access Network (RAN) based on the statistics of user demands and channel states, and also network availability.
Decentralized Non-Convex Learning with Linearly Coupled Constraints
Motivated by the need for decentralized learning, this paper aims at designing a distributed algorithm for solving nonconvex problems with general linear constraints over a multi-agent network.
Optimization and Control Systems and Control Systems and Control
Data-Driven Adaptive Network Slicing for Multi-Tenant Networks
We propose a Frank-Wolfe algorithm to iteratively solve approximated problems in long time-scales.
On Generalization of Adversarial Imitation Learning and Beyond
For some MDPs, we show that vanilla AIL has a worse sample complexity than BC.
Efficient Estimation of Sensor Biases for the 3-Dimensional Asynchronous Multi-Sensor System
There are generally two difficulties in this bias estimation problem: one is the unknown target states which serve as the nuisance variables in the estimation problem, and the other is the highly nonlinear coordinate transformation between the local and global coordinate systems of the sensors.
HyperDQN: A Randomized Exploration Method for Deep Reinforcement Learning
However, it is limited to the case where 1) a good feature is known in advance and 2) this feature is fixed during the training: if otherwise, RLSVI suffers an unbearable computational burden to obtain the posterior samples of the parameter in the $Q$-value function.
Fast Generic Interaction Detection for Model Interpretability and Compression
The ability of discovering feature interactions in a black-box model is vital to explainable deep learning.
Rethinking ValueDice: Does It Really Improve Performance?
First, we show that ValueDice could reduce to BC under the offline setting.
Downlink Channel Covariance Matrix Reconstruction for FDD Massive MIMO Systems with Limited Feedback
The downlink channel covariance matrix (CCM) acquisition is the key step for the practical performance of massive multiple-input and multiple-output (MIMO) systems, including beamforming, channel tracking, and user scheduling.
Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation
In this paper, we propose the Differentiation on Spike Representation (DSR) method, which could achieve high performance that is competitive to ANNs yet with low latency.
Rethinking WMMSE: Can Its Complexity Scale Linearly With the Number of BS Antennas?
Precoding design for maximizing weighted sum-rate (WSR) is a fundamental problem for downlink of massive multi-user multiple-input multiple-output (MU-MIMO) systems.
Efficient-Adam: Communication-Efficient Distributed Adam
Distributed adaptive stochastic gradient methods have been widely used for large-scale nonconvex optimization, such as training deep learning models.
Robust Adaptive Beamforming via Worst-Case SINR Maximization with Nonconvex Uncertainty Sets
Then a linear matrix inequality (LMI) relaxation for the QMI problem is proposed, with an additional valid linear constraint.
Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis
Imitation learning learns a policy from expert trajectories.
Adam Can Converge Without Any Modification On Update Rules
We point out there is a mismatch between the settings of theory and practice: Reddi et al. 2018 pick the problem after picking the hyperparameters of Adam, i. e., $(\beta_1, \beta_2)$; while practical applications often fix the problem first and then tune $(\beta_1, \beta_2)$.
Adaptive Smoothness-weighted Adversarial Training for Multiple Perturbations with Its Stability Analysis
Therefore, adversarial training for multiple perturbations (ATMP) is proposed to generalize the adversarial robustness over different perturbation types (in $\ell_1$, $\ell_2$, and $\ell_\infty$ norm-bounded perturbations).
Understanding Adversarial Robustness Against On-manifold Adversarial Examples
On synthetic datasets, theoretically, We prove that on-manifold adversarial examples are powerful, yet adversarial training focuses on off-manifold directions and ignores the on-manifold adversarial examples.
Stability Analysis and Generalization Bounds of Adversarial Training
In adversarial machine learning, deep neural networks can fit the adversarial examples on the training dataset but have poor generalization ability on the test set.
When Expressivity Meets Trainability: Fewer than $n$ Neurons Can Work
Third, we consider a constrained optimization formulation where the feasible region is the nice local region, and prove that every KKT point is a nearly global minimizer.
Bridging Distributional and Risk-sensitive Reinforcement Learning with Provable Regret Bounds
We study the regret guarantee for risk-sensitive reinforcement learning (RSRL) via distributional reinforcement learning (DRL) methods.
Computational Efficiency
Distributional Reinforcement Learning
+2
Adversarial Rademacher Complexity of Deep Neural Networks
Specifically, we provide the first bound of adversarial Rademacher complexity of deep neural networks.
A Data Quality Assessment Framework for AI-enabled Wireless Communication
Using artificial intelligent (AI) to re-design and enhance the current wireless communication system is a promising pathway for the future sixth-generation (6G) wireless network.
Theoretical Analysis of Offline Imitation With Supplementary Dataset
This paper considers a situation where, besides the small amount of expert data, a supplementary dataset is available, which can be collected cheaply from sub-optimal policies.
Invariant Layers for Graphs with Nodes of Different Types
We further narrow the invariant network design space by addressing a question about the sizes of tensor layers necessary for function approximation on graph data.
Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks
In particular, our method achieves state-of-the-art accuracy on ImageNet, while the memory cost and training time are reduced by more than 70% and 50%, respectively, compared with BPTT.
A Physics-based and Data-driven Approach for Localized Statistical Channel Modeling
Localized channel modeling is crucial for offline performance optimization of 5G cellular networks, but the existing channel models are for general scenarios and do not capture local geographical structures.
Regret Bounds for Risk-sensitive Reinforcement Learning with Lipschitz Dynamic Risk Measures
We study finite episodic Markov decision processes incorporating dynamic risk measures to capture risk sensitivity.
Provably Efficient Adversarial Imitation Learning with Unknown Transitions
Adversarial imitation learning (AIL), a subset of IL methods, is particularly promising, but its theoretical foundation in the presence of unknown transitions has yet to be fully developed.
A Distribution Optimization Framework for Confidence Bounds of Risk Measures
Unlike traditional approaches that add or subtract a confidence radius from the empirical risk measures, our proposed schemes evaluate a specific transformation of the empirical distribution based on the distance.
ReMax: A Simple, Effective, and Efficient Reinforcement Learning Method for Aligning Large Language Models
Based on these properties, we develop ReMax, a new algorithm tailored for RLHF.
TeleQnA: A Benchmark Dataset to Assess Large Language Models Telecommunications Knowledge
Afterwards, using the provided dataset, an evaluation is conducted to assess the capabilities of LLMs, including GPT-3. 5 and GPT-4.
HyperAgent: A Simple, Scalable, Efficient and Provable Reinforcement Learning Framework for Complex Environments
To solve complex tasks under resource constraints, reinforcement learning (RL) agents need to be simple, efficient, and scalable, addressing (1) large state spaces and (2) the continuous accumulation of interaction data.
Optimistic Thompson Sampling for No-Regret Learning in Unknown Games
This work tackles the complexities of multi-player scenarios in \emph{unknown games}, where the primary challenge lies in navigating the uncertainty of the environment through bandit feedback alongside strategic decision-making.
Why Transformers Need Adam: A Hessian Perspective
SGD performs worse than Adam by a significant margin on Transformers, but the reason remains unclear.
Radar Anti-jamming Strategy Learning via Domain-knowledge Enhanced Online Convex Optimization
The dynamic competition between radar and jammer systems presents a significant challenge for modern Electronic Warfare (EW), as current active learning approaches still lack sample efficiency and fail to exploit jammer's characteristics.
Prior-dependent analysis of posterior sampling reinforcement learning with function approximation
This work advances randomized exploration in reinforcement learning (RL) with function approximation modeled by linear mixture MDPs.
