Search Results for author: Bin Gu

Found 37 papers, 5 papers with code

Fast OSCAR and OWL with Safe Screening Rules

no code implementations ICML 2020 Runxue Bao, Bin Gu, Heng Huang

Ordered Weight $L_{1}$-Norms (OWL) is a new family of regularizers for high-dimensional sparse regression.

Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm

no code implementations19 Mar 2022 Qingsong Zhang, Bin Gu, Zhiyuan Dang, Cheng Deng, Heng Huang

Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO.

Federated Learning

Multi-Level Contrastive Learning for Cross-Lingual Alignment

no code implementations26 Feb 2022 Beiduo Chen, Wu Guo, Bin Gu, Quan Liu, Yongchao Wang

Cross-language pre-trained models such as multilingual BERT (mBERT) have achieved significant performance in various cross-lingual downstream NLP tasks.

Contrastive Learning Cross-Lingual Transfer

A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse

no code implementations9 Dec 2021 Junyi Li, Bin Gu, Heng Huang

Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm.

Bilevel Optimization

Efficient Semi-Supervised Adversarial Training without Guessing Labels

no code implementations29 Sep 2021 Huimin Wu, Heng Huang, Bin Gu

To adapt to semi-supervised learning problems, they need to estimate labels for unlabeled data in advance, which inevitably degenerates the performance of the learned model due to the bias on the estimation of labels for unlabeled data.

Perturbation Diversity Certificates Robust Generalisation

no code implementations29 Sep 2021 Zhuang Qian, Shufei Zhang, Kaizhu Huang, Qiufeng Wang, Bin Gu, Huan Xiong, Xinping Yi

It is possibly due to the fact that the conventional adversarial training methods generate adversarial perturbations usually in a supervised way, so that the adversarial samples are highly biased towards the decision boundary, resulting in an inhomogeneous data distribution.

Efficient Bi-level Optimization for Non-smooth Optimization

no code implementations29 Sep 2021 Wanli Shi, Heng Huang, Bin Gu

Then, we transform the smoothed bi-level optimization to an unconstrained penalty problem by replacing the smoothed sub-problem with its first-order necessary conditions.

Accelerated Gradient-Free Method for Heavily Constrained Nonconvex Optimization

no code implementations29 Sep 2021 Wanli Shi, Hongchang Gao, Bin Gu

In this paper, to solve the nonconvex problem with a large number of white/black-box constraints, we proposed a doubly stochastic zeroth-order gradient method (DSZOG).

AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization

no code implementations26 Sep 2021 Qingsong Zhang, Bin Gu, Cheng Deng, Songxiang Gu, Liefeng Bo, Jian Pei, Heng Huang

To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i. e. AsySQN-SGD, -SVRG and -SAGA, are proposed.

Federated Learning

An Accelerated Variance-Reduced Conditional Gradient Sliding Algorithm for First-order and Zeroth-order Optimization

no code implementations18 Sep 2021 Xiyuan Wei, Bin Gu, Heng Huang

The conditional gradient algorithm (also known as the Frank-Wolfe algorithm) has recently regained popularity in the machine learning community due to its projection-free property to solve constrained problems.

Fast and Scalable Adversarial Training of Kernel SVM via Doubly Stochastic Gradients

1 code implementation21 Jul 2021 Huimin Wu, Zhengmian Hu, Bin Gu

Although a wide range of researches have been done in recent years to improve the adversarial robustness of learning models, but most of them are limited to deep neural networks (DNNs) and the work for kernel SVM is still vacant.

Adversarial Robustness

Topic Classification on Spoken Documents Using Deep Acoustic and Linguistic Features

no code implementations16 Jun 2021 Tan Liu, Wu Guo, Bin Gu

In this paper, instead of using the ASR transcripts, the fusion of deep acoustic and linguistic features is used for topic classification on spoken documents.

Automatic Speech Recognition Topic Classification

Learning Sampling Policy for Faster Derivative Free Optimization

no code implementations9 Apr 2021 Zhou Zhai, Bin Gu, Heng Huang

To explore this problem, in this paper, we propose a new reinforcement learning based ZO algorithm (ZO-RL) with learning the sampling policy for generating the perturbations in ZO optimization instead of using random sampling.


Bidirectional Multiscale Feature Aggregation for Speaker Verification

no code implementations1 Apr 2021 Jiajun Qi, Wu Guo, Bin Gu

In this paper, we propose a novel bidirectional multiscale feature aggregation (BMFA) network with attentional fusion modules for text-independent speaker verification.

Text-Independent Speaker Verification

Improved Meta-learning training for Speaker Verification

no code implementations29 Mar 2021 Yafeng Chen, Wu Guo, Bin Gu

By combining these two methods, we can observe further improvements on these two databases.

Data Augmentation Meta-Learning +2

Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating

no code implementations1 Mar 2021 Qingsong Zhang, Bin Gu, Cheng Deng, Heng Huang

Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage.

Federated Learning

Optimizing Large-Scale Hyperparameters via Automated Learning Algorithm

1 code implementation17 Feb 2021 Bin Gu, Guodong Liu, yanfu Zhang, Xiang Geng, Heng Huang

Modern machine learning algorithms usually involve tuning multiple (from one to thousands) hyperparameters which play a pivotal role in terms of model generalizability.

Hyperparameter Optimization

Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee

no code implementations1 Sep 2020 Junyi Li, Bin Gu, Heng Huang

In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation.

Representation Learning

Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data

1 code implementation14 Aug 2020 Bin Gu, Zhiyuan Dang, Xiang Li, Heng Huang

In this paper, we focus on nonlinear learning with kernels, and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data.

Federated Learning

Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning

no code implementations14 Aug 2020 Bin Gu, An Xu, Zhouyuan Huo, Cheng Deng, Heng Huang

To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.

Federated Learning

Fast OSCAR and OWL Regression via Safe Screening Rules

1 code implementation29 Jun 2020 Runxue Bao, Bin Gu, Heng Huang

Moreover, we prove that the algorithms with our screening rule are guaranteed to have identical results with the original algorithms.

Sparse Learning

Faster On-Device Training Using New Federated Momentum Algorithm

no code implementations6 Feb 2020 Zhouyuan Huo, Qian Yang, Bin Gu, Lawrence Carin. Heng Huang

Mobile crowdsensing has gained significant attention in recent years and has become a critical paradigm for emerging Internet of Things applications.

Federated Learning

Large Batch Training Does Not Need Warmup

no code implementations4 Feb 2020 Zhouyuan Huo, Bin Gu, Heng Huang

Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications.

Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization

no code implementations24 Dec 2019 Wanli Shi, Bin Gu, Xinag Li, Heng Huang

Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled.

Safe Sample Screening for Robust Support Vector Machine

no code implementations24 Dec 2019 Zhou Zhai, Bin Gu, Xiang Li, Heng Huang

To address this challenge, in this paper, we propose two safe sample screening rules for RSVM based on the framework of concave-convex procedure (CCCP).

Quadruply Stochastic Gradients for Large Scale Nonlinear Semi-Supervised AUC Optimization

no code implementations29 Jul 2019 Wanli Shi, Bin Gu, Xiang Li, Xiang Geng, Heng Huang

To address this problem, in this paper, we propose a novel scalable quadruply stochastic gradient algorithm (QSG-S2AUC) for nonlinear semi-supervised AUC optimization.

Stochastic Optimization

Scalable Semi-Supervised SVM via Triply Stochastic Gradients

no code implementations26 Jul 2019 Xiang Geng, Bin Gu, Xiang Li, Wanli Shi, Guansheng Zheng, Heng Huang

Specifically, to handle two types of data instances involved in S$^3$VM, TSGS$^3$VM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient.

Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization

no code implementations16 Feb 2019 Feihu Huang, Bin Gu, Zhouyuan Huo, Songcan Chen, Heng Huang

Proximal gradient method has been playing an important role to solve many machine learning tasks, especially for the nonsmooth problems.

Training Neural Networks Using Features Replay

no code implementations NeurIPS 2018 Zhouyuan Huo, Bin Gu, Heng Huang

Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network.

Faster Derivative-Free Stochastic Algorithm for Shared Memory Machines

no code implementations ICML 2018 Bin Gu, Zhouyuan Huo, Cheng Deng, Heng Huang

Asynchronous parallel stochastic gradient optimization has been playing a pivotal role to solve large-scale machine learning problems in big data applications.

Ensemble Learning

Decoupled Parallel Backpropagation with Convergence Guarantee

3 code implementations ICML 2018 Zhouyuan Huo, Bin Gu, Qian Yang, Heng Huang

The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully leveraging the computing resources.

Accelerated Method for Stochastic Composition Optimization with Nonsmooth Regularization

no code implementations10 Nov 2017 Zhouyuan Huo, Bin Gu, Ji Liu, Heng Huang

To the best of our knowledge, our method admits the fastest convergence rate for stochastic composition optimization: for strongly convex composition problem, our algorithm is proved to admit linear convergence; for general composition problem, our algorithm significantly improves the state-of-the-art convergence rate from $O(T^{-1/2})$ to $O((n_1+n_2)^{{2}/{3}}T^{-1})$.


Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

no code implementations18 Dec 2016 Bin Gu, De Wang, Zhouyuan Huo, Heng Huang

The theoretical results show that our inexact proximal gradient algorithms can have the same convergence rates as the ones of exact proximal gradient algorithms in the non-convex setting.

Zeroth-order Asynchronous Doubly Stochastic Algorithm with Variance Reduction

no code implementations5 Dec 2016 Bin Gu, Zhouyuan Huo, Heng Huang

The convergence rate of existing asynchronous doubly stochastic zeroth order algorithms is $O(\frac{1}{\sqrt{T}})$ (also for the sequential stochastic zeroth-order optimization algorithms).

Asynchronous Stochastic Block Coordinate Descent with Variance Reduction

no code implementations29 Oct 2016 Bin Gu, Zhouyuan Huo, Heng Huang

In this paper, we focus on a composite objective function consisting of a smooth convex function $f$ and a block separable convex function, which widely exists in machine learning and computer vision.

Stochastic Optimization

Decoupled Asynchronous Proximal Stochastic Gradient Descent with Variance Reduction

no code implementations22 Sep 2016 Zhouyuan Huo, Bin Gu, Heng Huang

In this paper, we propose a faster method, decoupled asynchronous proximal stochastic variance reduced gradient descent method (DAP-SVRG).

Cannot find the paper you are looking for? You can Submit a new open access paper.