Search Results for author: Furong Huang

Found 44 papers, 12 papers with code

Understanding the Generalization Benefit of Model Invariance from a Data Perspective

1 code implementation NeurIPS 2021 Sicheng Zhu, Bang An, Furong Huang

In experiments on multiple datasets, we evaluate sample covering numbers for some commonly used transformations and show that the smaller sample covering number for a set of transformations (e. g., the 3D-view transformation) indicates a smaller gap between the test and training error for invariant models, which verifies our propositions.

Generalization Bounds

Comfetch: Federated Learning of Large Networks on Memory-Constrained Clients via Sketching

no code implementations17 Sep 2021 Tahseen Rabbani, Brandon Feng, Yifan Yang, Arjun Rajkumar, Amitabh Varshney, Furong Huang

A popular application of federated learning is using many clients to train a deep neural network, the parameters of which are maintained on a central server.

Federated Learning

Practical and Fast Momentum-Based Power Methods

no code implementations20 Aug 2021 Tahseen Rabbani, Apollo Jain, Arjun Rajkumar, Furong Huang

The power method is a classical algorithm with broad applications in machine learning tasks, including streaming PCA, spectral clustering, and low-rank matrix approximation.

Certified Defense via Latent Space Randomized Smoothing with Orthogonal Encoders

no code implementations1 Aug 2021 Huimin Zeng, Jiahao Su, Furong Huang

Randomized Smoothing (RS), being one of few provable defenses, has been showing great effectiveness and scalability in terms of defending against $\ell_2$-norm adversarial perturbations.

Scaling-up Diverse Orthogonal Convolutional Networks with a Paraunitary Framework

no code implementations16 Jun 2021 Jiahao Su, Wonmin Byeon, Furong Huang

To address this problem, we propose a theoretical framework for orthogonal convolutional layers, which establishes the equivalence between various orthogonal convolutional layers in the spatial domain and the paraunitary systems in the spectral domain.

Who Is the Strongest Enemy? Towards Optimal and Efficient Evasion Attacks in Deep RL

no code implementations9 Jun 2021 Yanchao Sun, Ruijie Zheng, Yongyuan Liang, Furong Huang

Evaluating the worst-case performance of a reinforcement learning (RL) agent under the strongest/optimal adversarial perturbations on state observations (within some constraints) is crucial for understanding the robustness of RL agents.

Can You Learn an Algorithm? Generalizing from Easy to Hard Problems with Recurrent Networks

1 code implementation NeurIPS 2021 Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, Tom Goldstein

In this work, we show that recurrent networks trained to solve simple problems with few recurrent steps can indeed solve much more complex problems simply by performing additional recurrences during inference.

Guided Hyperparameter Tuning Through Visualization and Inference

no code implementations24 May 2021 Hyekang Joo, Calvin Bao, Ishan Sen, Furong Huang, Leilani Battle

Moreover, an analysis on the variance in a selected performance metric in the context of the model hyperparameters shows the impact that certain hyperparameters have on the performance metric.

Insta-RS: Instance-wise Randomized Smoothing for Improved Robustness and Accuracy

no code implementations7 Mar 2021 Chen Chen, Kezhi Kong, Peihong Yu, Juan Luque, Tom Goldstein, Furong Huang

Randomized smoothing (RS) is an effective and scalable technique for constructing neural network classifiers that are certifiably robust to adversarial perturbations.

DP-InstaHide: Provably Defusing Poisoning and Backdoor Attacks with Differentially Private Data Augmentations

no code implementations2 Mar 2021 Eitan Borgnia, Jonas Geiping, Valeriia Cherepanova, Liam Fowl, Arjun Gupta, Amin Ghiasi, Furong Huang, Micah Goldblum, Tom Goldstein

The InstaHide method has recently been proposed as an alternative to DP training that leverages supposed privacy properties of the mixup augmentation, although without rigorous guarantees.

Data Poisoning

Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks

no code implementations24 Oct 2020 Huimin Zeng, Chen Zhu, Tom Goldstein, Furong Huang

Adversarial Training is proved to be an efficient method to defend against adversarial examples, being one of the few defenses that withstand strong attacks.

Vulnerability-Aware Poisoning Mechanism for Online RL with Unknown Dynamics

no code implementations ICLR 2021 Yanchao Sun, Da Huo, Furong Huang

Poisoning attacks on Reinforcement Learning (RL) systems could take advantage of RL algorithm's vulnerabilities and cause failure of the learning.

MaxVA: Fast Adaptation of Step Sizes by Maximizing Observed Variance of Gradients

1 code implementation21 Jun 2020 Chen Zhu, Yu Cheng, Zhe Gan, Furong Huang, Jingjing Liu, Tom Goldstein

Adaptive gradient methods such as RMSProp and Adam use exponential moving estimate of the squared gradient to compute adaptive step sizes, achieving better convergence than SGD in face of noisy objectives.

Image Classification Language understanding +4

Using Wavelets and Spectral Methods to Study Patterns in Image-Classification Datasets

no code implementations17 Jun 2020 Roozbeh Yousefzadeh, Furong Huang

We show that each image can be written as the summation of a finite number of rank-1 patterns in the wavelet space, providing a low rank approximation that captures the structures and patterns essential for learning.

Adversarial Robustness General Classification +1

Improving the Tightness of Convex Relaxation Bounds for Training Certifiably Robust Classifiers

no code implementations22 Feb 2020 Chen Zhu, Renkun Ni, Ping-Yeh Chiang, Hengduo Li, Furong Huang, Tom Goldstein

Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical robustness.

Convolutional Tensor-Train LSTM for Spatio-temporal Learning

2 code implementations NeurIPS 2020 Jiahao Su, Wonmin Byeon, Jean Kossaifi, Furong Huang, Jan Kautz, Animashree Anandkumar

Learning from spatio-temporal data has numerous applications such as human-behavior analysis, object tracking, video compression, and physics simulation. However, existing methods still perform poorly on challenging video tasks such as long-term forecasting.

Activity Recognition Video Compression +1

TempLe: Learning Template of Transitions for Sample Efficient Multi-task RL

no code implementations16 Feb 2020 Yanchao Sun, Xiangyu Yin, Furong Huang

Transferring knowledge among various environments is important to efficiently learn multiple tasks online.

ARMA Nets: Expanding Receptive Field for Dense Prediction

no code implementations NeurIPS 2020 Jiahao Su, Shiqi Wang, Furong Huang

In this work, we propose to replace any traditional convolutional layer with an autoregressive moving-average (ARMA) layer, a novel module with an adjustable receptive field controlled by the learnable autoregressive coefficients.

Image Classification Semantic Segmentation +1

Understanding Generalization in Deep Learning via Tensor Methods

no code implementations14 Jan 2020 Jingling Li, Yanchao Sun, Jiahao Su, Taiji Suzuki, Furong Huang

Recently proposed complexity measures have provided insights to understanding the generalizability in neural networks from perspectives of PAC-Bayes, robustness, overparametrization, compression and so on.

Can Agents Learn by Analogy? An Inferable Model for PAC Reinforcement Learning

1 code implementation21 Dec 2019 Yanchao Sun, Furong Huang

We propose a new model-based method called Greedy Inference Model (GIM) that infers the unknown dynamics from known dynamics based on the internal spectral properties of the environment.

Model-based Reinforcement Learning

Sampling-Free Learning of Bayesian Quantized Neural Networks

no code implementations ICLR 2020 Jiahao Su, Milan Cvitkovic, Furong Huang

Bayesian learning of model parameters in neural networks is important in scenarios where estimates with well-calibrated uncertainty are important.

Image Classification

Label Smoothing and Logit Squeezing: A Replacement for Adversarial Training?

no code implementations25 Oct 2019 Ali Shafahi, Amin Ghiasi, Furong Huang, Tom Goldstein

Adversarial training is one of the strongest defenses against adversarial attacks, but it requires adversarial examples to be generated for every mini-batch during optimization.

Adversarial Robustness

Improved Training of Certifiably Robust Models

no code implementations25 Sep 2019 Chen Zhu, Renkun Ni, Ping-Yeh Chiang, Hengduo Li, Furong Huang, Tom Goldstein

Convex relaxations are effective for training and certifying neural networks against norm-bounded adversarial attacks, but they leave a large gap between certifiable and empirical (PGD) robustness.

Convolutional Tensor-Train LSTM for Long-Term Video Prediction

no code implementations25 Sep 2019 Jiahao Su, Wonmin Byeon, Furong Huang, Jan Kautz, Animashree Anandkumar

Long-term video prediction is highly challenging since it entails simultaneously capturing spatial and temporal information across a long range of image frames. Standard recurrent models are ineffective since they are prone to error propagation and cannot effectively capture higher-order correlations.

Video Prediction

Understanding Generalization through Visualizations

2 code implementations NeurIPS Workshop ICBINB 2020 W. Ronny Huang, Zeyad Emam, Micah Goldblum, Liam Fowl, Justin K. Terry, Furong Huang, Tom Goldstein

The power of neural networks lies in their ability to generalize to unseen data, yet the underlying reasons for this phenomenon remain elusive.

Guaranteed Simultaneous Asymmetric Tensor Decomposition via Orthogonalized Alternating Least Squares

no code implementations25 May 2018 Furong Huang, Jialin Li, Xuchen You

We propose a Slicing Initialized Alternating Subspace Iteration (s-ASI) method that is guaranteed to recover top $r$ components ($\epsilon$-close) simultaneously for (a)symmetric tensors almost surely under the noiseless case (with high probability for a bounded noise) using $O(\log(\log \frac{1}{\epsilon}))$ steps of tensor subspace iterations.

Tensor Decomposition

Tensorial Neural Networks: Generalization of Neural Networks and Application to Model Compression

no code implementations25 May 2018 Jiahao Su, Jingling Li, Bobby Bhattacharjee, Furong Huang

We propose tensorial neural networks (TNNs), a generalization of existing neural networks by extending tensor operations on low order operands to those on high order ones.

Model Compression Tensor Decomposition

An end-to-end Differentially Private Latent Dirichlet Allocation Using a Spectral Algorithm

no code implementations ICML 2020 Christopher DeCarolis, Mukul Ram, Seyed A. Esmaeili, Yu-Xiang Wang, Furong Huang

Overall, by combining the sensitivity and utility characterization, we obtain an end-to-end differentially private spectral algorithm for LDA and identify the corresponding configuration that outperforms others in any specific regime.

Variational Inference

Learning Deep ResNet Blocks Sequentially using Boosting Theory

no code implementations ICML 2018 Furong Huang, Jordan Ash, John Langford, Robert Schapire

We prove that the training error decays exponentially with the depth $T$ if the \emph{weak module classifiers} that we train perform slightly better than some weak baseline.

Non-negative Factorization of the Occurrence Tensor from Financial Contracts

1 code implementation10 Dec 2016 Zheng Xu, Furong Huang, Louiqa Raschid, Tom Goldstein

We propose an algorithm for the non-negative factorization of an occurrence tensor built from heterogeneous networks.

Unsupervised learning of transcriptional regulatory networks via latent tree graphical models

no code implementations20 Sep 2016 Anthony Gitter, Furong Huang, Ragupathyraj Valluvan, Ernest Fraenkel, Animashree Anandkumar

We use a latent tree graphical model to analyze gene expression without relying on transcription factor expression as a proxy for regulator activity.

Unsupervised Learning of Word-Sequence Representations from Scratch via Convolutional Tensor Decomposition

no code implementations10 Jun 2016 Furong Huang, Animashree Anandkumar

More importantly, it is challenging for pre-trained models to obtain word-sequence embeddings that are universally good for all downstream tasks or for any new datasets.

Dictionary Learning Tensor Decomposition

Discovery of Latent Factors in High-dimensional Data Using Tensor Methods

no code implementations10 Jun 2016 Furong Huang

This thesis presents theoretical results on convergence to globally optimal solution of tensor decomposition using the stochastic gradient descent, despite non-convexity of the objective.

Dimensionality Reduction Latent Variable Models +2

Discovering Neuronal Cell Types and Their Gene Expression Profiles Using a Spatial Point Process Mixture Model

no code implementations4 Feb 2016 Furong Huang, Animashree Anandkumar, Christian Borgs, Jennifer Chayes, Ernest Fraenkel, Michael Hawrylycz, Ed Lein, Alessandro Ingrosso, Srinivas Turaga

Single-cell RNA sequencing can now be used to measure the gene expression profiles of individual neurons and to categorize neurons based on their gene expression profiles.

Convolutional Dictionary Learning through Tensor Factorization

no code implementations10 Jun 2015 Furong Huang, Animashree Anandkumar

Tensor methods have emerged as a powerful paradigm for consistent learning of many latent variable models such as topic models, independent component analysis and dictionary learning.

Dictionary Learning Latent Variable Models +2

Escaping From Saddle Points --- Online Stochastic Gradient for Tensor Decomposition

1 code implementation6 Mar 2015 Rong Ge, Furong Huang, Chi Jin, Yang Yuan

To the best of our knowledge this is the first work that gives global convergence guarantees for stochastic gradient descent on non-convex functions with exponentially many local minima and saddle points.

Latent Variable Models Tensor Decomposition

Guaranteed Scalable Learning of Latent Tree Models

no code implementations18 Jun 2014 Furong Huang, Niranjan U. N., Ioakeim Perros, Robert Chen, Jimeng Sun, Anima Anandkumar

We present an integrated approach for structure and parameter estimation in latent tree graphical models.

Online Tensor Methods for Learning Latent Variable Models

1 code implementation3 Sep 2013 Furong Huang, U. N. Niranjan, Mohammad Umar Hakeem, Animashree Anandkumar

We introduce an online tensor decomposition based approach for two latent variable modeling problems namely, (1) community detection, in which we learn the latent communities that the social actors in social networks belong to, and (2) topic modeling, in which we infer hidden topics of text articles.

Community Detection Latent Variable Models +1

Learning Mixtures of Tree Graphical Models

no code implementations NeurIPS 2012 Anima Anandkumar, Daniel J. Hsu, Furong Huang, Sham M. Kakade

We consider unsupervised estimation of mixtures of discrete graphical models, where the class variable is hidden and each mixture component can have a potentially different Markov graph structure and parameters over the observed variables.

Cannot find the paper you are looking for? You can Submit a new open access paper.