Search Results for author:
Found 19 papers, 6 papers with code
Distribution Aligning Refinery of Pseudo-label for Imbalanced Semi-supervised Learning
While semi-supervised learning (SSL) has proven to be a promising way for leveraging unlabeled data when labeled data is scarce, the existing SSL algorithms typically assume that training class distributions are balanced.
Layer-adaptive sparsity for the Magnitude-based Pruning
Recent discoveries on neural network pruning reveal that, with a carefully chosen layerwise sparsity, a simple magnitude-based pruning achieves state-of-the-art tradeoff between sparsity and performance.
Lookahead: a Far-Sighted Alternative of Magnitude-based Pruning
Magnitude-based pruning is one of the simplest methods for pruning neural networks.
SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Robustness
Randomized smoothing is currently a state-of-the-art method to construct a certifiably robust classifier from neural networks against $\ell_2$-adversarial perturbations.
Guiding Energy-based Models via Contrastive Latent Variables
To enable the joint training of EBM and CRL, we also design a new class of latent-variable EBMs for learning the joint density of data and the contrastive latent variable.
Learning Bounds for Risk-sensitive Learning
The second result, based on a novel variance-based characterization of OCE, gives an expected loss guarantee with a suppressed dependence on the smoothness of the selected OCE.
Max-Product Belief Propagation for Linear Programming: Applications to Combinatorial Optimization
The max-product {belief propagation} (BP) is a popular message-passing heuristic for approximating a maximum-a-posteriori (MAP) assignment in a joint distribution represented by a graphical model (GM).
Rapid Mixing Swendsen-Wang Sampler for Stochastic Partitioned Attractive Models
The Gibbs sampler is a particularly popular Markov chain used for learning and inference problems in Graphical Models (GMs).
Sequential Local Learning for Latent Graphical Models
Learning parameters of latent graphical models (GM) is inherently much harder than that of no-latent ones since the latent variables make the corresponding log-likelihood non-concave.
Minimum Weight Perfect Matching via Blossom Belief Propagation
Max-product Belief Propagation (BP) is a popular message-passing algorithm for computing a Maximum-A-Posteriori (MAP) assignment over a distribution represented by a Graphical Model (GM).
Spectral Approximate Inference
Our contribution is two-fold: (a) we first propose a fully polynomial-time approximation scheme (FPTAS) for approximating the partition function of GM associating with a low-rank coupling matrix; (b) for general high-rank GMs, we design a spectral mean-field scheme utilizing (a) as a subroutine, where it approximates a high-rank GM into a product of rank-1 GMs for an efficient approximation of the partition function.
Minimum Width for Universal Approximation
In this work, we provide the first definitive result in this direction for networks using the ReLU activation functions: The minimum width required for the universal approximation of the $L^p$ functions is exactly $\max\{d_x+1, d_y\}$.
Provable Memorization via Deep Neural Networks using Sub-linear Parameters
It is known that $O(N)$ parameters are sufficient for neural networks to memorize arbitrary $N$ input-label pairs.
SmoothMix: Training Confidence-calibrated Smoothed Classifiers for Certified Adversarial Robustness
Randomized smoothing is currently a state-of-the-art method to construct a certifiably robust classifier from neural networks against $\ell_2$-adversarial perturbations.
Generalization Bounds for Stochastic Gradient Descent via Localized $\varepsilon$-Covers
In this paper, we propose a new covering technique localized for the trajectories of SGD.
Neural Networks Efficiently Learn Low-Dimensional Representations with SGD
We further demonstrate that, SGD-trained ReLU NNs can learn a single-index target of the form $y=f(\langle\boldsymbol{u},\boldsymbol{x}\rangle) + \epsilon$ by recovering the principal direction, with a sample complexity linear in $d$ (up to log factors), where $f$ is a monotonic function with at most polynomial growth, and $\epsilon$ is the noise.
On the Correctness of Automatic Differentiation for Neural Networks with Machine-Representable Parameters
For a neural network with bias parameters, we first prove that the incorrect set is always empty.
Minimum width for universal approximation using ReLU networks on compact domain
We next prove a lower bound on $w_{\min}$ for uniform approximation using general activation functions including ReLU: $w_{\min}\ge d_y+1$ if $d_x<d_y\le2d_x$.
Expressive Power of ReLU and Step Networks under Floating-Point Operations
In this work, we analyze the expressive power of neural networks under a more realistic setup: when we use floating-point numbers and operations.
