Search Results for author: Heng Huang

Found 133 papers, 23 papers with code

Adversarial Nonnegative Matrix Factorization

no code implementations ICML 2020 lei luo, yanfu Zhang, Heng Huang

Nonnegative Matrix Factorization (NMF) has become an increasingly important research topic in machine learning.

Bilevel Optimization

Sparse Shrunk Additive Models

no code implementations ICML 2020 Hong Chen, Guodong Liu, Heng Huang

Meanwhile, in these feature selection models, the interactions between features are often ignored or just discussed under prior structure information.

Additive models feature selection

Can Stochastic Zeroth-Order Frank-Wolfe Method Converge Faster for Non-Convex Problems?

no code implementations ICML 2020 Hongchang Gao, Heng Huang

To address the problem of lacking gradient in many applications, we propose two new stochastic zeroth-order Frank-Wolfe algorithms and theoretically proved that they have a faster convergence rate than existing methods for non-convex problems.

Fast OSCAR and OWL with Safe Screening Rules

no code implementations ICML 2020 Runxue Bao, Bin Gu, Heng Huang

Ordered Weight $L_{1}$-Norms (OWL) is a new family of regularizers for high-dimensional sparse regression.

Functional2Structural: Cross-Modality Brain Networks Representation Learning

no code implementations6 May 2022 Haoteng Tang, Xiyao Fu, Lei Guo, Yalin Wang, Scott Mackin, Olusola Ajilore, Alex Leow, Paul Thompson, Heng Huang, Liang Zhan

Since brain networks derived from functional and structural MRI describe the brain topology from different perspectives, exploring a representation that combines these cross-modality brain networks is non-trivial.

Disease Prediction Graph Learning +2

Local Stochastic Bilevel Optimization with Momentum-Based Variance Reduction

no code implementations3 May 2022 Junyi Li, Feihu Huang, Heng Huang

Specifically, we first propose the FedBiO, a deterministic gradient-based algorithm and we show it requires $O(\epsilon^{-2})$ number of iterations to reach an $\epsilon$-stationary point.

Bilevel Optimization Federated Learning +2

Distributed Dynamic Safe Screening Algorithms for Sparse Regularization

no code implementations23 Apr 2022 Runxue Bao, Xidong Wu, Wenhan Xian, Heng Huang

To the best of our knowledge, this is the first work of distributed safe dynamic screening method.

Distributed Optimization

Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm

no code implementations19 Mar 2022 Qingsong Zhang, Bin Gu, Zhiyuan Dang, Cheng Deng, Heng Huang

Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO.

Federated Learning

Closing the Generalization Gap of Cross-silo Federated Medical Image Segmentation

no code implementations18 Mar 2022 An Xu, Wenqi Li, Pengfei Guo, Dong Yang, Holger Roth, Ali Hatamizadeh, Can Zhao, Daguang Xu, Heng Huang, Ziyue Xu

In this work, we propose a novel training framework FedSM to avoid the client drift issue and successfully close the generalization gap compared with the centralized training for medical image segmentation tasks for the first time.

Federated Learning Medical Image Segmentation +1

Towards Bi-directional Skip Connections in Encoder-Decoder Architectures and Beyond

no code implementations11 Mar 2022 Tiange Xiang, Chaoyi Zhang, Xinyi Wang, Yang song, Dongnan Liu, Heng Huang, Weidong Cai

With the backward skip connections, we propose a U-Net based network family, namely Bi-directional O-shape networks, which set new benchmarks on multiple public medical imaging segmentation datasets.

Medical Image Segmentation Neural Architecture Search

How Many Data Are Needed for Robust Learning?

no code implementations23 Feb 2022 Hongyang Zhang, Yihan Wu, Heng Huang

We show that the sample complexity of robust interpolation problem could be exponential in the input dimensionality and discover a phase transition phenomenon when the data are in a unit ball.

HNF-Netv2 for Brain Tumor Segmentation using multi-modal MR Imaging

no code implementations10 Feb 2022 Haozhe Jia, Chao Bai, Weidong Cai, Heng Huang, Yong Xia

In our previous work, $i. e.$, HNF-Net, high-resolution feature representation and light-weight non-local self-attention mechanism are exploited for brain tumor segmentation using multi-modal MR imaging.

Brain Tumor Segmentation Tumor Segmentation

Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement

1 code implementation6 Jan 2022 Dongnan Liu, Chaoyi Zhang, Yang song, Heng Huang, Chenyu Wang, Michael Barnett, Weidong Cai

Recent advances in unsupervised domain adaptation (UDA) techniques have witnessed great success in cross-domain computer vision tasks, enhancing the generalization ability of data-driven deep learning architectures by bridging the domain distribution gaps.

Disentanglement Object Detection +1

A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse

no code implementations9 Dec 2021 Junyi Li, Bin Gu, Heng Huang

Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm.

Bilevel Optimization

Optimal Underdamped Langevin MCMC Method

no code implementations NeurIPS 2021 Zhengmian Hu, Feihu Huang, Heng Huang

In the paper, we study the underdamped Langevin diffusion (ULD) with strongly-convex potential consisting of finite summation of $N$ smooth components, and propose an efficient discretization method, which requires $O(N+d^\frac{1}{3}N^\frac{2}{3}/\varepsilon^\frac{2}{3})$ gradient evaluations to achieve $\varepsilon$-error (in $\sqrt{\mathbb{E}{\lVert{\cdot}\rVert_2^2}}$ distance) for approximating $d$-dimensional ULD.

Efficient Mirror Descent Ascent Methods for Nonsmooth Minimax Problems

no code implementations NeurIPS 2021 Feihu Huang, Xidong Wu, Heng Huang

For our stochastic algorithms, we first prove that the mini-batch stochastic mirror descent ascent (SMDA) method obtains a sample complexity of $O(\kappa^3\epsilon^{-4})$ for finding an $\epsilon$-stationary point, where $\kappa$ denotes the condition number.

A Faster Decentralized Algorithm for Nonconvex Minimax Problems

no code implementations NeurIPS 2021 Wenhan Xian, Feihu Huang, yanfu Zhang, Heng Huang

We prove that our DM-HSGD algorithm achieves stochastic first-order oracle (SFO) complexity of $O(\kappa^3 \epsilon^{-3})$ for decentralized stochastic nonconvex-strongly-concave problem to search an $\epsilon$-stationary point, which improves the exiting best theoretical results.

Fast Training Method for Stochastic Compositional Optimization Problems

no code implementations NeurIPS 2021 Hongchang Gao, Heng Huang

The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning.

Additive models Meta-Learning

Adaptive Hierarchical Similarity Metric Learning with Noisy Labels

no code implementations29 Oct 2021 Jiexi Yan, Lei Luo, Cheng Deng, Heng Huang

Since these noisy labels often cause severe performance degradation, it is crucial to enhance the robustness and generalization ability of DML.

Learning with noisy labels Metric Learning

Adversarial Fairness Network

no code implementations29 Sep 2021 Taeuk Jang, Xiaoqian Wang, Heng Huang

To achieve this goal, we reformulate the data input by eliminating the sensitive information and strengthen model fairness by minimizing the marginal contribution of the sensitive feature.

Fairness

Efficient Bi-level Optimization for Non-smooth Optimization

no code implementations29 Sep 2021 Wanli Shi, Heng Huang, Bin Gu

Then, we transform the smoothed bi-level optimization to an unconstrained penalty problem by replacing the smoothed sub-problem with its first-order necessary conditions.

Efficient Semi-Supervised Adversarial Training without Guessing Labels

no code implementations29 Sep 2021 Huimin Wu, Heng Huang, Bin Gu

To adapt to semi-supervised learning problems, they need to estimate labels for unlabeled data in advance, which inevitably degenerates the performance of the learned model due to the bias on the estimation of labels for unlabeled data.

Adversarial Weight Perturbation Improves Generalization in Graph Neural Networks

no code implementations29 Sep 2021 Yihan Wu, Aleksandar Bojchevski, Heng Huang

Along the way, we identify a vanishing-gradient issue with all existing formulations of AWP and we propose Weighted Truncated AWP (WT-AWP) to alleviate this issue.

Graph Learning

Understanding Metric Learning on Unit Hypersphere and Generating Better Examples for Adversarial Training

no code implementations29 Sep 2021 Yihan Wu, Heng Huang

In this paper, we boost the performance of deep metric learning (DML) models with adversarial examples generated by attacking two new objective functions: \textit{intra-class alignment} and \textit{hyperspherical uniformity}.

Metric Learning Representation Learning

AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization

no code implementations26 Sep 2021 Qingsong Zhang, Bin Gu, Cheng Deng, Songxiang Gu, Liefeng Bo, Jian Pei, Heng Huang

To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i. e. AsySQN-SGD, -SVRG and -SAGA, are proposed.

Federated Learning

An Accelerated Variance-Reduced Conditional Gradient Sliding Algorithm for First-order and Zeroth-order Optimization

no code implementations18 Sep 2021 Xiyuan Wei, Bin Gu, Heng Huang

The conditional gradient algorithm (also known as the Frank-Wolfe algorithm) has recently regained popularity in the machine learning community due to its projection-free property to solve constrained problems.

DSNet: A Dual-Stream Framework for Weakly-Supervised Gigapixel Pathology Image Analysis

no code implementations13 Sep 2021 Tiange Xiang, Yang song, Chaoyi Zhang, Dongnan Liu, Mei Chen, Fan Zhang, Heng Huang, Lauren O'Donnell, Weidong Cai

With image-level labels only, patch-wise classification would be sub-optimal due to inconsistency between the patch appearance and image-level label.

Classification whole slide images

Boundary-aware Graph Reasoning for Semantic Segmentation

no code implementations9 Aug 2021 Haoteng Tang, Haozhe Jia, Weidong Cai, Heng Huang, Yong Xia, Liang Zhan

In this paper, we propose a Boundary-aware Graph Reasoning (BGR) module to learn long-range contextual features for semantic segmentation.

graph construction Semantic Segmentation

PSGR: Pixel-wise Sparse Graph Reasoning for COVID-19 Pneumonia Segmentation in CT Images

no code implementations9 Aug 2021 Haozhe Jia, Haoteng Tang, Guixiang Ma, Weidong Cai, Heng Huang, Liang Zhan, Yong Xia

In the PSGR module, a graph is first constructed by projecting each pixel on a node based on the features produced by the segmentation backbone, and then converted into a sparsely-connected graph by keeping only K strongest connections to each uncertain pixel.

Computed Tomography (CT) graph construction +1

Enhanced Bilevel Optimization via Bregman Distance

no code implementations26 Jul 2021 Feihu Huang, Junyi Li, Heng Huang

Specifically, we propose a bilevel optimization method based on Bregman distance (BiO-BreD) for solving deterministic bilevel problems, which reaches a lower computational complexity than the best known results.

Bilevel Optimization Hyperparameter Optimization +1

Fedlearn-Algo: A flexible open-source privacy-preserving machine learning platform

3 code implementations8 Jul 2021 Bo Liu, Chaowei Tan, Jiazhou Wang, Tao Zeng, Huasong Shan, Houpu Yao, Heng Huang, Peng Dai, Liefeng Bo, Yanqing Chen

We use this platform to demonstrate our research and development results on privacy preserving machine learning algorithms.

Federated Learning

AdaGDA: Faster Adaptive Gradient Descent Ascent Methods for Minimax Optimization

no code implementations30 Jun 2021 Feihu Huang, Heng Huang

Specifically, we propose a fast Adaptive Gradient Descent Ascent (AdaGDA) method based on the basic momentum technique, which reaches a lower gradient complexity of $O(\kappa^4\epsilon^{-4})$ for finding an $\epsilon$-stationary point without large batches, which improves the existing results of the adaptive GDA methods by a factor of $O(\sqrt{\kappa})$.

BiX-NAS: Searching Efficient Bi-directional Architecture for Medical Image Segmentation

1 code implementation26 Jun 2021 Xinyi Wang, Tiange Xiang, Chaoyi Zhang, Yang song, Dongnan Liu, Heng Huang, Weidong Cai

We evaluate BiX-NAS on two segmentation tasks using three different medical image datasets, and the experimental results show that our BiX-NAS searched architecture achieves the state-of-the-art performance with significantly lower computational cost.

Medical Image Segmentation Neural Architecture Search +1

Bregman Gradient Policy Optimization

1 code implementation ICLR 2022 Feihu Huang, Shangqian Gao, Heng Huang

In the paper, we design a novel Bregman gradient policy optimization framework for reinforcement learning based on Bregman divergences and momentum techniques.

reinforcement-learning

BiAdam: Fast Adaptive Bilevel Optimization Methods

no code implementations21 Jun 2021 Feihu Huang, Heng Huang

At the same time, we propose an accelerated version of BiAdam algorithm (VR-BiAdam) by using variance reduced technique, which reaches the best known sample complexity of $\tilde{O}(\epsilon^{-3})$.

Bilevel Optimization

Nearest Neighbor Matching for Deep Clustering

1 code implementation CVPR 2021 Zhiyuan Dang, Cheng Deng, Xu Yang, Kun Wei, Heng Huang

Specifically, for the local level, we match the nearest neighbors based on batch embedded features, as for the global one, we match neighbors from overall embedded features.

Deep Clustering Frame

Unsupervised Hyperbolic Metric Learning

no code implementations CVPR 2021 Jiexi Yan, Lei Luo, Cheng Deng, Heng Huang

Learning feature embedding directly from images without any human supervision is a very challenging and essential task in the field of computer vision and machine learning.

Metric Learning

Network Pruning via Performance Maximization

1 code implementation CVPR 2021 Shangqian Gao, Feihu Huang, Weidong Cai, Heng Huang

Specifically, we train a stand-alone neural network to predict sub-networks' performance and then maximize the output of the network as a proxy of accuracy to guide pruning.

Model Compression Network Pruning

SUPER-ADAM: Faster and Universal Framework of Adaptive Gradients

1 code implementation NeurIPS 2021 Feihu Huang, Junyi Li, Heng Huang

To fill this gap, we propose a faster and universal framework of adaptive gradients (i. e., SUPER-ADAM) by introducing a universal adaptive matrix that includes most existing adaptive gradient forms.

Fast Training Method for Stochastic Compositional Optimization Problems

no code implementations NeurIPS 2021 Hongchang Gao, Heng Huang

The stochastic compositional optimization problem covers a wide range of machine learning models, such as sparse additive models and model-agnostic meta-learning.

Additive models Meta-Learning

Learning Sampling Policy for Faster Derivative Free Optimization

no code implementations9 Apr 2021 Zhou Zhai, Bin Gu, Heng Huang

To explore this problem, in this paper, we propose a new reinforcement learning based ZO algorithm (ZO-RL) with learning the sampling policy for generating the perturbations in ZO optimization instead of using random sampling.

reinforcement-learning

Doubly Contrastive Deep Clustering

1 code implementation9 Mar 2021 Zhiyuan Dang, Cheng Deng, Xu Yang, Heng Huang

In this paper, we present a novel Doubly Contrastive Deep Clustering (DCDC) framework, which constructs contrastive loss over both sample and class views to obtain more discriminative features and competitive results.

Contrastive Learning Data Augmentation +1

Data Augmentation for Object Detection via Differentiable Neural Rendering

1 code implementation4 Mar 2021 Guanghan Ning, Guang Chen, Chaowei Tan, Si Luo, Liefeng Bo, Heng Huang

We propose a new offline data augmentation method for object detection, which semantically interpolates the training data with novel views.

Data Augmentation Neural Rendering +2

Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating

no code implementations1 Mar 2021 Qingsong Zhang, Bin Gu, Cheng Deng, Heng Huang

Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage.

Federated Learning

Optimizing Large-Scale Hyperparameters via Automated Learning Algorithm

1 code implementation17 Feb 2021 Bin Gu, Guodong Liu, yanfu Zhang, Xiang Geng, Heng Huang

Modern machine learning algorithms usually involve tuning multiple (from one to thousands) hyperparameters which play a pivotal role in terms of model generalizability.

Hyperparameter Optimization

A New Framework for Variance-Reduced Hamiltonian Monte Carlo

no code implementations9 Feb 2021 Zhengmian Hu, Feihu Huang, Heng Huang

Moreover, our HMC methods with biased gradient estimators, such as SARAH and SARGE, require $\tilde{O}(N+\sqrt{N} \kappa^2 d^{\frac{1}{2}} \varepsilon^{-1})$ gradient complexity, which has the same dependency on condition number $\kappa$ and dimension $d$ as full gradient method, but improves the dependency of sample size $N$ for a factor of $N^\frac{1}{2}$.

Coordinating Momenta for Cross-silo Federated Learning

no code implementations8 Feb 2021 An Xu, Heng Huang

In this work, we propose a new method to improve the training performance in cross-silo FL via maintaining double momentum buffers.

Federated Learning

A Bayesian Federated Learning Framework with Online Laplace Approximation

no code implementations3 Feb 2021 Liangxi Liu, Feng Zheng, Hong Chen, Guo-Jun Qi, Heng Huang, Ling Shao

On the client side, a prior loss that uses the global posterior probabilistic parameters delivered from the server is designed to guide the local training.

Federated Learning

Exploration and Estimation for Model Compression

no code implementations ICCV 2021 yanfu Zhang, Shangqian Gao, Heng Huang

In this paper, we focus on the discrimination-aware compression of Convolutional Neural Networks (CNNs).

Model Compression

Delay-Tolerant Local SGD for Efficient Distributed Training

no code implementations1 Jan 2021 An Xu, Xiao Yan, Hongchang Gao, Heng Huang

The heavy communication for model synchronization is a major bottleneck for scaling up the distributed deep neural network training to many workers.

Federated Learning

Adversarial Attack on Deep Cross-Modal Hamming Retrieval

no code implementations ICCV 2021 Chao Li, Shangqian Gao, Cheng Deng, Wei Liu, Heng Huang

Specifically, given a target model, we first construct its substitute model to exploit cross-modal correlations within hamming space, with which we create adversarial examples by limitedly querying from a target model.

Adversarial Attack Cross-Modal Retrieval

Learning Better Visual Data Similarities via New Grouplet Non-Euclidean Embedding

no code implementations ICCV 2021 yanfu Zhang, Lei Luo, Wenhan Xian, Heng Huang

However, pair-wise methods involve expensive training costs, while proxy-based methods are less accurate in characterizing the relationships between data points.

Metric Learning

Model Compression via Hyper-Structure Network

no code implementations1 Jan 2021 Shangqian Gao, Feihu Huang, Heng Huang

In this paper, we propose a novel channel pruning method to solve the problem of compression and acceleration of Convolutional Neural Networks (CNNs).

Model Compression

CommPOOL: An Interpretable Graph Pooling Framework for Hierarchical Graph Representation Learning

no code implementations10 Dec 2020 Haoteng Tang, Guixiang Ma, Lifang He, Heng Huang, Liang Zhan

In this paper, we propose a new interpretable graph pooling framework - CommPOOL, that can capture and preserve the hierarchical community structure of graphs in the graph representation learning process.

Graph Classification Graph Representation Learning

A Deep Drift-Diffusion Model for Image Aesthetic Score Distribution Prediction

no code implementations15 Oct 2020 Xin Jin, Xiqiao Li, Heng Huang, XiaoDong Li, Xinghui Zhou

In this paper, we propose a Deep Drift-Diffusion (DDD) model inspired by psychologists to predict aesthetic score distribution from images.

PDAM: A Panoptic-Level Feature Alignment Framework for Unsupervised Domain Adaptive Instance Segmentation in Microscopy Images

1 code implementation11 Sep 2020 Dongnan Liu, Donghao Zhang, Yang song, Fan Zhang, Lauren O'Donnell, Heng Huang, Mei Chen, Weidong Cai

In this work, we present an unsupervised domain adaptation (UDA) method, named Panoptic Domain Adaptive Mask R-CNN (PDAM), for unsupervised instance segmentation in microscopy images.

Instance Segmentation Semantic Segmentation +1

Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee

no code implementations1 Sep 2020 Junyi Li, Bin Gu, Heng Huang

In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation.

Representation Learning

Adaptive Serverless Learning

no code implementations24 Aug 2020 Hongchang Gao, Heng Huang

To the best of our knowledge, this is the first adaptive decentralized training approach.

Periodic Stochastic Gradient Descent with Momentum for Decentralized Training

no code implementations24 Aug 2020 Hongchang Gao, Heng Huang

The condition for achieving the linear speedup is also provided for this variant.

Accelerated Zeroth-Order and First-Order Momentum Methods from Mini to Minimax Optimization

no code implementations18 Aug 2020 Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang

Our Acc-MDA achieves a low gradient complexity of $\tilde{O}(\kappa_y^{4. 5}\epsilon^{-3})$ without requiring large batches for finding an $\epsilon$-stationary point.

Adversarial Attack

Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning

no code implementations14 Aug 2020 Bin Gu, An Xu, Zhouyuan Huo, Cheng Deng, Heng Huang

To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.

Federated Learning

Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data

1 code implementation14 Aug 2020 Bin Gu, Zhiyuan Dang, Xiang Li, Heng Huang

In this paper, we focus on nonlinear learning with kernels, and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data.

Federated Learning

Step-Ahead Error Feedback for Distributed Training with Compressed Gradient

no code implementations13 Aug 2020 An Xu, Zhouyuan Huo, Heng Huang

Both our theoretical and empirical results show that our new methods can handle the "gradient mismatch" problem.

Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization

no code implementations4 Aug 2020 Feihu Huang, Songcan Chen, Heng Huang

Our theoretical analysis shows that the online SPIDER-ADMM has the IFO complexity of $\mathcal{O}(\epsilon^{-\frac{3}{2}})$, which improves the existing best results by a factor of $\mathcal{O}(\epsilon^{-\frac{1}{2}})$.

Momentum-Based Policy Gradient Methods

1 code implementation ICML 2020 Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang

In particular, we present a non-adaptive version of IS-MBPG method, i. e., IS-MBPG*, which also reaches the best known sample complexity of $O(\epsilon^{-3})$ without any large batches.

Policy Gradient Methods

BiO-Net: Learning Recurrent Bi-directional Connections for Encoder-Decoder Architecture

1 code implementation1 Jul 2020 Tiange Xiang, Chaoyi Zhang, Dongnan Liu, Yang song, Heng Huang, Weidong Cai

U-Net has become one of the state-of-the-art deep learning-based approaches for modern computer vision tasks such as semantic segmentation, super resolution, image denoising, and inpainting.

Image Denoising Semantic Segmentation +1

Fast OSCAR and OWL Regression via Safe Screening Rules

1 code implementation29 Jun 2020 Runxue Bao, Bin Gu, Heng Huang

Moreover, we prove that the algorithms with our screening rule are guaranteed to have identical results with the original algorithms.

Sparse Learning

Faster Secure Data Mining via Distributed Homomorphic Encryption

no code implementations17 Jun 2020 Junyi Li, Heng Huang

Due to the rising privacy demand in data mining, Homomorphic Encryption (HE) is receiving more and more attention recently for its capability to do computations over the encrypted field.

Exploit Where Optimizer Explores via Residuals

no code implementations11 Apr 2020 An Xu, Zhouyuan Huo, Heng Huang

In order to train the neural networks faster, many efforts have been devoted to exploring a better solution trajectory, but few have been put into exploiting the existing solution trajectory.

Image Classification Language Modelling

Optimal Gradient Quantization Condition for Communication-Efficient Distributed Training

no code implementations25 Feb 2020 An Xu, Zhouyuan Huo, Heng Huang

The communication of gradients is costly for training deep neural networks with multiple devices in computer vision applications.

Quantization

Panoptic Feature Fusion Net: A Novel Instance Segmentation Paradigm for Biomedical and Biological Images

1 code implementation15 Feb 2020 Dongnan Liu, Donghao Zhang, Yang song, Heng Huang, Weidong Cai

Specifically, our proposed PFFNet contains a residual attention feature fusion mechanism to incorporate the instance prediction with the semantic features, in order to facilitate the semantic contextual information learning in the instance branch.

Instance Segmentation Medical Image Segmentation +1

Large Batch Training Does Not Need Warmup

no code implementations4 Feb 2020 Zhouyuan Huo, Bin Gu, Heng Huang

Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications.

Safe Sample Screening for Robust Support Vector Machine

no code implementations24 Dec 2019 Zhou Zhai, Bin Gu, Xiang Li, Heng Huang

To address this challenge, in this paper, we propose two safe sample screening rules for RSVM based on the framework of concave-convex procedure (CCCP).

Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization

no code implementations24 Dec 2019 Wanli Shi, Bin Gu, Xinag Li, Heng Huang

Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled.

Region and Object based Panoptic Image Synthesis through Conditional GANs

no code implementations14 Dec 2019 Heng Wang, Donghao Zhang, Yang song, Heng Huang, Mei Chen, Weidong Cai

Our contribution consists of the proposal of a significant task worth investigating and a naive baseline of solving it.

Image-to-Image Translation Translation

Curvilinear Distance Metric Learning

1 code implementation NeurIPS 2019 Shuo Chen, Lei Luo, Jian Yang, Chen Gong, Jun Li, Heng Huang

To address this issue, we first reveal that the traditional linear distance metric is equivalent to the cumulative arc length between the data pair's nearest points on the learned straight measurer lines.

Metric Learning

Straggler-Agnostic and Communication-Efficient Distributed Primal-Dual Algorithm for High-Dimensional Data Mining

no code implementations9 Oct 2019 Zhouyuan Huo, Heng Huang

Recently, reducing communication time between machines becomes the main focus of distributed data mining.

Deep Relational Factorization Machines

no code implementations25 Sep 2019 Hongchang Gao, Gang Wu, Ryan Rossi, Viswanathan Swaminathan, Heng Huang

Factorization Machines (FMs) is an important supervised learning approach due to its unique ability to capture feature interactions when dealing with high-dimensional sparse data.

Approaching Machine Learning Fairness through Adversarial Network

no code implementations6 Sep 2019 Xiaoqian Wang, Heng Huang

In order to achieve this goal, we reformulate the data input by removing the sensitive information and strengthen model fairness by minimizing the marginal contribution of the sensitive feature.

Fairness

On the Acceleration of Deep Learning Model Parallelism with Staleness

no code implementations CVPR 2020 An Xu, Zhouyuan Huo, Heng Huang

Training the deep convolutional neural network for computer vision problems is slow and inefficient, especially when it is large and distributed across multiple devices.

Nonconvex Zeroth-Order Stochastic ADMM Methods with Lower Function Query Complexity

no code implementations30 Jul 2019 Feihu Huang, Shangqian Gao, Jian Pei, Heng Huang

Zeroth-order methods powerful optimization tools for solving many machine learning problems because it only need function values (not gradient) in the optimization.

Adversarial Attack

Quadruply Stochastic Gradients for Large Scale Nonlinear Semi-Supervised AUC Optimization

no code implementations29 Jul 2019 Wanli Shi, Bin Gu, Xiang Li, Xiang Geng, Heng Huang

To address this problem, in this paper, we propose a novel scalable quadruply stochastic gradient algorithm (QSG-S2AUC) for nonlinear semi-supervised AUC optimization.

Stochastic Optimization

Scalable Semi-Supervised SVM via Triply Stochastic Gradients

no code implementations26 Jul 2019 Xiang Geng, Bin Gu, Xiang Li, Wanli Shi, Guansheng Zheng, Heng Huang

Specifically, to handle two types of data instances involved in S$^3$VM, TSGS$^3$VM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient.

An Iteratively Re-weighted Method for Problems with Sparsity-Inducing Norms

no code implementations2 Jul 2019 Feiping Nie, Zhanxuan Hu, Xiaoqian Wang, Rong Wang, Xuelong. Li, Heng Huang

This work aims at solving the problems with intractable sparsity-inducing norms that are often encountered in various machine learning tasks, such as multi-task learning, subspace clustering, feature selection, robust principal component analysis, and so on.

feature selection Multi-Task Learning

Robust Linear Discriminant Analysis Using Ratio Minimization of L1,2-Norms

no code implementations29 Jun 2019 Feiping Nie, Hua Wang, Zheng Wang, Heng Huang

In this paper, we propose a novel robust linear discriminant analysis method based on the L1, 2-norm ratio minimization.

Zeroth-Order Stochastic Alternating Direction Method of Multipliers for Nonconvex Nonsmooth Optimization

no code implementations29 May 2019 Feihu Huang, Shangqian Gao, Songcan Chen, Heng Huang

In particular, our methods not only reach the best convergence rate $O(1/T)$ for the nonconvex optimization, but also are able to effectively solve many complex machine learning problems with multiple regularized penalties and constraints.

Adversarial Attack

LightTrack: A Generic Framework for Online Top-Down Human Pose Tracking

2 code implementations7 May 2019 Guanghan Ning, Heng Huang

To the best of our knowledge, this is the first paper to propose an online human pose tracking framework in a top-down fashion.

Pose Estimation Pose Tracking +1

Heterogeneous Memory Enhanced Multimodal Attention Model for Video Question Answering

1 code implementation CVPR 2019 Chenyou Fan, Xiaofan Zhang, Shu Zhang, Wensheng Wang, Chi Zhang, Heng Huang

In this paper, we propose a novel end-to-end trainable Video Question Answering (VideoQA) framework with three major components: 1) a new heterogeneous memory which can effectively learn global context information from appearance and motion features; 2) a redesigned question memory which helps understand the complex semantics of question and highlights queried subjects; and 3) a new multimodal fusion layer which performs multi-step reasoning by attending to relevant visual and textual hints with self-updated attention.

Question Answering Video Question Answering +1

Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization

no code implementations16 Feb 2019 Feihu Huang, Bin Gu, Zhouyuan Huo, Songcan Chen, Heng Huang

Proximal gradient method has been playing an important role to solve many machine learning tasks, especially for the nonsmooth problems.

Bilevel Distance Metric Learning for Robust Image Recognition

no code implementations NeurIPS 2018 Jie Xu, Lei Luo, Cheng Deng, Heng Huang

Metric learning, aiming to learn a discriminative Mahalanobis distance matrix M that can effectively reflect the similarity between data samples, has been widely studied in various image recognition problems.

Metric Learning

3D Global Convolutional Adversarial Network\\ for Prostate MR Volume Segmentation

no code implementations18 Jul 2018 Haozhe Jia, Yang song, Donghao Zhang, Heng Huang, Dagan Feng, Michael Fulham, Yong Xia, Weidong Cai

In this paper, we propose a 3D Global Convolutional Adversarial Network (3D GCA-Net) to address efficient prostate MR volume segmentation.

General Classification

Training Neural Networks Using Features Replay

no code implementations NeurIPS 2018 Zhouyuan Huo, Bin Gu, Heng Huang

Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network.

Faster Derivative-Free Stochastic Algorithm for Shared Memory Machines

no code implementations ICML 2018 Bin Gu, Zhouyuan Huo, Cheng Deng, Heng Huang

Asynchronous parallel stochastic gradient optimization has been playing a pivotal role to solve large-scale machine learning problems in big data applications.

Ensemble Learning

Direct Shape Regression Networks for End-to-End Face Alignment

no code implementations CVPR 2018 Xin Miao, Xian-Tong Zhen, Xianglong Liu, Cheng Deng, Vassilis Athitsos, Heng Huang

In this paper, we propose the direct shape regression network (DSRN) for end-to-end face alignment by jointly handling the aforementioned challenges in a unified framework.

Face Alignment Structured Prediction

Decoupled Parallel Backpropagation with Convergence Guarantee

3 code implementations ICML 2018 Zhouyuan Huo, Bin Gu, Qian Yang, Heng Huang

The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully leveraging the computing resources.

Learning A Structured Optimal Bipartite Graph for Co-Clustering

no code implementations NeurIPS 2017 Feiping Nie, Xiaoqian Wang, Cheng Deng, Heng Huang

In graph based co-clustering methods, a bipartite graph is constructed to depict the relation between features and samples.

Group Sparse Additive Machine

no code implementations NeurIPS 2017 Hong Chen, Xiaoqian Wang, Cheng Deng, Heng Huang

Among them, learning models with grouped variables have shown competitive performance for prediction and variable selection.

Additive models Classification +2

Accelerated Method for Stochastic Composition Optimization with Nonsmooth Regularization

no code implementations10 Nov 2017 Zhouyuan Huo, Bin Gu, Ji Liu, Heng Huang

To the best of our knowledge, our method admits the fastest convergence rate for stochastic composition optimization: for strongly convex composition problem, our algorithm is proved to admit linear convergence; for general composition problem, our algorithm significantly improves the state-of-the-art convergence rate from $O(T^{-1/2})$ to $O((n_1+n_2)^{{2}/{3}}T^{-1})$.

reinforcement-learning

Fast and Scalable Distributed Deep Convolutional Autoencoder for fMRI Big Data Analytics

no code implementations24 Oct 2017 Milad Makkie, Heng Huang, Yu Zhao, Athanasios V. Vasilakos, Tianming Liu

In recent years, analyzing task-based fMRI (tfMRI) data has become an essential tool for understanding brain function and networks.

Dictionary Learning Time Series

Deep Clustering via Joint Convolutional Autoencoder Embedding and Relative Entropy Minimization

1 code implementation ICCV 2017 Kamran Ghasedi Dizaji, Amirhossein Herandi, Cheng Deng, Weidong Cai, Heng Huang

We define a clustering objective function using relative entropy (KL divergence) minimization, regularized by a prior for the frequency of cluster assignments.

Deep Clustering Image Clustering

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

no code implementations18 Dec 2016 Bin Gu, De Wang, Zhouyuan Huo, Heng Huang

The theoretical results show that our inexact proximal gradient algorithms can have the same convergence rates as the ones of exact proximal gradient algorithms in the non-convex setting.

Zeroth-order Asynchronous Doubly Stochastic Algorithm with Variance Reduction

no code implementations5 Dec 2016 Bin Gu, Zhouyuan Huo, Heng Huang

The convergence rate of existing asynchronous doubly stochastic zeroth order algorithms is $O(\frac{1}{\sqrt{T}})$ (also for the sequential stochastic zeroth-order optimization algorithms).

Error Analysis of Generalized Nyström Kernel Regression

no code implementations NeurIPS 2016 Hong Chen, Haifeng Xia, Heng Huang, Weidong Cai

Nystr\"{o}m method has been used successfully to improve the computational efficiency of kernel ridge regression (KRR).

Asynchronous Stochastic Block Coordinate Descent with Variance Reduction

no code implementations29 Oct 2016 Bin Gu, Zhouyuan Huo, Heng Huang

In this paper, we focus on a composite objective function consisting of a smooth convex function $f$ and a block separable convex function, which widely exists in machine learning and computer vision.

Stochastic Optimization

A Closed Form Solution to Multi-View Low-Rank Regression

no code implementations14 Oct 2016 Shuai Zheng, Xiao Cai, Chris Ding, Feiping Nie, Heng Huang

Real life data often includes information from different channels.

A Harmonic Mean Linear Discriminant Analysis for Robust Image Classification

no code implementations14 Oct 2016 Shuai Zheng, Feiping Nie, Chris Ding, Heng Huang

In null space based LDA (NLDA), a well-known LDA extension, between-class distance is maximized in the null space of the within-class scatter matrix.

Classification General Classification +2

Decoupled Asynchronous Proximal Stochastic Gradient Descent with Variance Reduction

no code implementations22 Sep 2016 Zhouyuan Huo, Bin Gu, Heng Huang

In this paper, we propose a faster method, decoupled asynchronous proximal stochastic variance reduced gradient descent method (DAP-SVRG).

Distributed Asynchronous Dual Free Stochastic Dual Coordinate Ascent

no code implementations29 May 2016 Zhouyuan Huo, Heng Huang

Our method does not need the dual formulation of the target problem in the optimization.

Distributed Optimization

Asynchronous Stochastic Gradient Descent with Variance Reduction for Non-Convex Optimization

no code implementations12 Apr 2016 Zhouyuan Huo, Heng Huang

We provide the first theoretical analysis on the convergence rate of the asynchronous stochastic variance reduced gradient (SVRG) descent algorithm on non-convex optimization.

Enhancing Sentence Relation Modeling with Auxiliary Character-level Embedding

no code implementations30 Mar 2016 Peng Li, Heng Huang

Neural network based approaches for sentence relation modeling automatically generate hidden matching features from raw sentence pairs.

Clinical Information Extraction via Convolutional Neural Network

no code implementations30 Mar 2016 Peng Li, Heng Huang

We report an implementation of a clinical information extraction tool that leverages deep neural network to annotate event spans and their attributes from raw clinical notes and pathology reports.

Non-Greedy L21-Norm Maximization for Principal Component Analysis

no code implementations28 Mar 2016 Feiping Nie, Heng Huang

In this paper, we propose to maximize the L21-norm based robust PCA objective, which is theoretically connected to the minimization of reconstruction error.

Theoretic Analysis and Extremely Easy Algorithms for Domain Adaptive Feature Learning

no code implementations5 Sep 2015 Wenhao Jiang, Cheng Deng, Wei Liu, Feiping Nie, Fu-Lai Chung, Heng Huang

Domain adaptation problems arise in a variety of applications, where a training dataset from the \textit{source} domain and a test dataset from the \textit{target} domain typically follow different distributions.

Domain Adaptation

Improved Spectral Clustering via Embedded Label Propagation

no code implementations23 Nov 2014 Xiaojun Chang, Feiping Nie, Yi Yang, Heng Huang

Our algorithm is built upon two advancements of the state of the art:1) label propagation, which propagates a node\'s labels to neighboring nodes according to their proximity; and 2) manifold learning, which has been widely used in its capacity to leverage the manifold structure of data points.

A Convex Sparse PCA for Feature Analysis

no code implementations23 Nov 2014 Xiaojun Chang, Feiping Nie, Yi Yang, Heng Huang

In addition, based on the sparse model used in CSPCA, an optimal weight is assigned to each of the original feature, which in turn provides the output with good interpretability.

Dimensionality Reduction feature selection

Video Motion Segmentation Using New Adaptive Manifold Denoising Model

no code implementations CVPR 2014 Dijun Luo, Heng Huang

After that, we employ an embedded manifold denoising approach with the adaptive kernel to segment the motion of rigid and non-rigid objects.

Denoising Motion Segmentation

Heterogeneous Visual Features Fusion via Sparse Multimodal Machine

no code implementations CVPR 2013 Hua Wang, Feiping Nie, Heng Huang, Chris Ding

We applied our SMML method to five broadly used object categorization and scene understanding image data sets for both singlelabel and multi-label image classification tasks.

Feature Importance Multi-Label Image Classification +2

Efficient and Robust Feature Selection via Joint ℓ2,1-Norms Minimization

no code implementations NeurIPS 2010 Feiping Nie, Heng Huang, Xiao Cai, Chris H. Ding

The ℓ2, 1-norm based loss function is robust to outliers in data points and the ℓ2, 1-norm regularization selects features across all data points with joint sparsity.

feature selection

Cannot find the paper you are looking for? You can Submit a new open access paper.