Search Results for author: Bin Gu

Found 64 papers, 8 papers with code

Fast OSCAR and OWL with Safe Screening Rules

no code implementations ICML 2020 Runxue Bao, Bin Gu, Heng Huang

Ordered Weight $L_{1}$-Norms (OWL) is a new family of regularizers for high-dimensional sparse regression.

regression

FTBC: Forward Temporal Bias Correction for Optimizing ANN-SNN Conversion

no code implementations27 Mar 2024 Xiaofeng Wu, Velibor Bojkovic, Bin Gu, Kun Suo, Kai Zou

Spiking Neural Networks (SNNs) offer a promising avenue for energy-efficient computing compared with Artificial Neural Networks (ANNs), closely mirroring biological neural processes.

Federated Causal Discovery from Heterogeneous Data

1 code implementation20 Feb 2024 Loka Li, Ignavier Ng, Gongxu Luo, Biwei Huang, Guangyi Chen, Tongliang Liu, Bin Gu, Kun Zhang

This discrepancy has motivated the development of federated causal discovery (FCD) approaches.

Causal Discovery

Limited Memory Online Gradient Descent for Kernelized Pairwise Learning with Dynamic Averaging

no code implementations2 Feb 2024 Hilal AlQuabeh, William de Vazelhes, Bin Gu

Recently, an OGD algorithm emerged, employing gradient computation involving prior and most recent examples, a step that effectively reduces algorithmic complexity to $O(T)$, with $T$ being the number of received examples.

Metric Learning

DevEval: Evaluating Code Generation in Practical Software Projects

no code implementations12 Jan 2024 Jia Li, Ge Li, YunFei Zhao, Yongmin Li, Zhi Jin, Hao Zhu, Huanyu Liu, Kaibo Liu, Lecheng Wang, Zheng Fang, Lanshen Wang, Jiazheng Ding, Xuanming Zhang, Yihong Dong, Yuqi Zhu, Bin Gu, Mengfei Yang

Compared to previous benchmarks, DevEval aligns to practical projects in multiple dimensions, e. g., real program distributions, sufficient dependencies, and enough-scale project contexts.

Code Generation

Iterative Regularization with k-support Norm: An Important Complement to Sparse Recovery

1 code implementation19 Dec 2023 William de Vazelhes, Bhaskar Mukhoty, Xiao-Tong Yuan, Bin Gu

However, most of those iterative methods are based on the $\ell_1$ norm which requires restrictive applicability conditions and could fail in many cases.

Dynamic Spiking Graph Neural Networks

no code implementations15 Dec 2023 Nan Yin, Mengzhu Wang, Zhenghan Chen, Giulia De Masi, Bin Gu, Huan Xiong

Current work often uses SNNs instead of Recurrent Neural Networks (RNNs) by using binary features instead of continuous ones for efficient training, which would overlooks graph structure information and leads to the loss of details during propagation.

Dynamic Node Classification Graph Representation Learning

Rethinking the Instruction Quality: LIFT is What You Need

no code implementations12 Dec 2023 Yang Xu, Yongqiang Yao, Yufan Huang, MengNan Qi, Maoquan Wang, Bin Gu, Neel Sundaresan

Instruction tuning, a specialized technique to enhance large language model (LLM) performance via instruction datasets, relies heavily on the quality of employed data.

Code Generation Instruction Following +3

SUT: Active Defects Probing for Transcompiler Models

no code implementations22 Oct 2023 MengNan Qi, Yufan Huang, Maoquan Wang, Yongqiang Yao, Zihan Liu, Bin Gu, Colin Clement, Neel Sundaresan

In this paper we introduce a new metrics for programming language translation and these metrics address these basic syntax errors.

Translation

Program Translation via Code Distillation

no code implementations17 Oct 2023 Yufan Huang, MengNan Qi, Yongqiang Yao, Maoquan Wang, Bin Gu, Colin Clement, Neel Sundaresan

Distilled code serves as a translation pivot for any programming language, leading by construction to parallel corpora which scale to all available source code by simply applying the distillation compiler.

Machine Translation Translation

Variance Reduced Online Gradient Descent for Kernelized Pairwise Learning with Limited Memory

1 code implementation10 Oct 2023 Hilal AlQuabeh, Bhaskar Mukhoty, Bin Gu

Specifically, we establish a clear connection between the variance of online gradients and the regret, and construct online gradients using the most recent stratified samples with a limited buffer of size of $s$ representing all past data, which have a complexity of $O(sT)$ and employs $O(\sqrt{T}\log{T})$ random Fourier features for kernel approximation.

Correct-by-Construction for Hybrid Systems by Synthesizing Reset Controller

no code implementations12 Sep 2023 Jiang Liu, Han Su, Yunjun Bai, Bin Gu, Bai Xue, Mengfei Yang, Naijun Zhan

Controller synthesis, including reset controller, feedback controller, and switching logic controller, provides an essential mechanism to guarantee the correctness and reliability of hybrid systems in a correct-by-construction manner.

Secure and Fast Asynchronous Vertical Federated Learning via Cascaded Hybrid Optimization

no code implementations28 Jun 2023 Ganyu Wang, Qingsong Zhang, Li Xiang, Boyu Wang, Bin Gu, Charles Ling

Meanwhile, the upstream model (server) is updated with first-order optimization (FOO) locally, which significantly improves the convergence rate, making it feasible to train the large models without compromising privacy and security.

Privacy Preserving Vertical Federated Learning

Advancing Counterfactual Inference through Nonlinear Quantile Regression

no code implementations9 Jun 2023 Shaoan Xie, Biwei Huang, Bin Gu, Tongliang Liu, Kun Zhang

Traditional counterfactual inference, under Pearls' counterfactual framework, typically depends on having access to or estimating a structural causal model.

counterfactual Counterfactual Inference +2

Language-Universal Adapter Learning with Knowledge Distillation for End-to-End Multilingual Speech Recognition

1 code implementation28 Feb 2023 Zhijie Shen, Wu Guo, Bin Gu

In this paper, we propose a language-universal adapter learning framework based on a pre-trained model for end-to-end multilingual automatic speech recognition (ASR).

Automatic Speech Recognition Automatic Speech Recognition (ASR) +2

On the Stability and Generalization of Triplet Learning

no code implementations20 Feb 2023 Jun Chen, Hong Chen, Xue Jiang, Bin Gu, Weifu Li, Tieliang Gong, Feng Zheng

Triplet learning, i. e. learning from triplet data, has attracted much attention in computer vision tasks with an extremely large number of categories, e. g., face recognition and person re-identification.

Face Recognition Metric Learning +1

Stability-based Generalization Analysis for Mixtures of Pointwise and Pairwise Learning

no code implementations20 Feb 2023 Jiahuan Wang, Jun Chen, Hong Chen, Bin Gu, Weifu Li, Xin Tang

Recently, some mixture algorithms of pointwise and pairwise learning (PPL) have been formulated by employing the hybrid error metric of "pointwise loss + pairwise loss" and have shown empirical effectiveness on feature selection, ranking and recommendation tasks.

feature selection Generalization Bounds +1

Energy Efficient Training of SNN using Local Zeroth Order Method

no code implementations2 Feb 2023 Bhaskar Mukhoty, Velibor Bojkovic, William de Vazelhes, Giulia De Masi, Huan Xiong, Bin Gu

To circumvent the problem surrogate method uses a differentiable approximation of the Heaviside in the backward pass, while the forward pass uses the Heaviside as the spiking function.

Denoising Multi-Similarity Formulation: A Self-paced Curriculum-Driven Approach for Robust Metric Learning

no code implementations19 Nov 2022 Chenkang Zhang, Lei Luo, Bin Gu

To efficiently solve this nonconvex quadratic problem, we propose a doubly stochastic projection coordinate gradient algorithm.

Denoising Metric Learning

Zeroth-Order Hard-Thresholding: Gradient Error vs. Expansivity

no code implementations11 Oct 2022 William de Vazelhes, Hualin Zhang, Huimin Wu, Xiao-Tong Yuan, Bin Gu

To solve this puzzle, in this paper, we focus on the $\ell_0$ constrained black-box stochastic optimization problems, and propose a new stochastic zeroth-order gradient hard-thresholding (SZOHT) algorithm with a general ZO gradient estimator powered by a novel random support sampling.

Portfolio Optimization Sparse Learning +1

Zeroth-Order Negative Curvature Finding: Escaping Saddle Points without Gradients

no code implementations4 Oct 2022 Hualin Zhang, Huan Xiong, Bin Gu

We consider escaping saddle points of nonconvex problems where only the function evaluations can be accessed.

GAGA: Deciphering Age-path of Generalized Self-paced Regularizer

no code implementations15 Sep 2022 Xingyu Qu, Diyang Li, Xiaohan Zhao, Bin Gu

The SPL regime involves a self-paced regularizer and a gradually increasing age parameter, which plays a key role in SPL but where to optimally terminate this process is still non-trivial to determine.

Computational Efficiency

An Accelerated Doubly Stochastic Gradient Method with Faster Explicit Model Identification

no code implementations11 Aug 2022 Runxue Bao, Bin Gu, Heng Huang

To address this challenge, we propose a novel accelerated doubly stochastic gradient descent (ADSGD) method for sparsity regularized loss minimization problems, which can reduce the number of block iterations by eliminating inactive coefficients during the optimization process and eventually achieve faster explicit model identification and improve the algorithm efficiency.

Dimensionality Reduction

Balanced Self-Paced Learning for AUC Maximization

no code implementations8 Jul 2022 Bin Gu, Chenkang Zhang, Huan Xiong, Heng Huang

Self-paced learning is an effective method for handling noisy data.

On the Convergence of Distributed Stochastic Bilevel Optimization Algorithms over a Network

no code implementations30 Jun 2022 Hongchang Gao, Bin Gu, My T. Thai

Bilevel optimization has been applied to a wide variety of machine learning models, and numerous stochastic bilevel optimization algorithms have been developed in recent years.

BIG-bench Machine Learning Bilevel Optimization +1

On the Intrinsic Structures of Spiking Neural Networks

no code implementations21 Jun 2022 Shao-Qun Zhang, Jia-Yi Chen, Jin-Hui Wu, Gao Zhang, Huan Xiong, Bin Gu, Zhi-Hua Zhou

Initially, we unveil two pivotal components of intrinsic structures: the integration operation and firing-reset mechanism, by elucidating their influence on the expressivity of SNNs.

Learning to Control under Time-Varying Environment

no code implementations6 Jun 2022 Yuzhen Han, Ruben Solozabal, Jing Dong, Xingyu Zhou, Martin Takac, Bin Gu

To the best of our knowledge, our study establishes the first model-based online algorithm with regret guarantees under LTV dynamical systems.

Desirable Companion for Vertical Federated Learning: New Zeroth-Order Gradient Based Algorithm

no code implementations19 Mar 2022 Qingsong Zhang, Bin Gu, Zhiyuan Dang, Cheng Deng, Heng Huang

Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO.

Vertical Federated Learning

Multi-Level Contrastive Learning for Cross-Lingual Alignment

no code implementations26 Feb 2022 Beiduo Chen, Wu Guo, Bin Gu, Quan Liu, Yongchao Wang

Cross-language pre-trained models such as multilingual BERT (mBERT) have achieved significant performance in various cross-lingual downstream NLP tasks.

Contrastive Learning Cross-Lingual Transfer +1

A Fully Single Loop Algorithm for Bilevel Optimization without Hessian Inverse

no code implementations9 Dec 2021 Junyi Li, Bin Gu, Heng Huang

Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm.

Bilevel Optimization

Efficient Semi-Supervised Adversarial Training without Guessing Labels

no code implementations29 Sep 2021 Huimin Wu, Heng Huang, Bin Gu

To adapt to semi-supervised learning problems, they need to estimate labels for unlabeled data in advance, which inevitably degenerates the performance of the learned model due to the bias on the estimation of labels for unlabeled data.

Accelerated Gradient-Free Method for Heavily Constrained Nonconvex Optimization

no code implementations29 Sep 2021 Wanli Shi, Hongchang Gao, Bin Gu

In this paper, to solve the nonconvex problem with a large number of white/black-box constraints, we proposed a doubly stochastic zeroth-order gradient method (DSZOG).

Perturbation Diversity Certificates Robust Generalisation

no code implementations29 Sep 2021 Zhuang Qian, Shufei Zhang, Kaizhu Huang, Qiufeng Wang, Bin Gu, Huan Xiong, Xinping Yi

It is possibly due to the fact that the conventional adversarial training methods generate adversarial perturbations usually in a supervised way, so that the adversarial samples are highly biased towards the decision boundary, resulting in an inhomogeneous data distribution.

Efficient Bi-level Optimization for Non-smooth Optimization

no code implementations29 Sep 2021 Wanli Shi, Heng Huang, Bin Gu

Then, we transform the smoothed bi-level optimization to an unconstrained penalty problem by replacing the smoothed sub-problem with its first-order necessary conditions.

Open-Ended Question Answering

AsySQN: Faster Vertical Federated Learning Algorithms with Better Computation Resource Utilization

no code implementations26 Sep 2021 Qingsong Zhang, Bin Gu, Cheng Deng, Songxiang Gu, Liefeng Bo, Jian Pei, Heng Huang

To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i. e. AsySQN-SGD, -SVRG and -SAGA, are proposed.

Privacy Preserving Vertical Federated Learning

An Accelerated Variance-Reduced Conditional Gradient Sliding Algorithm for First-order and Zeroth-order Optimization

no code implementations18 Sep 2021 Xiyuan Wei, Bin Gu, Heng Huang

The conditional gradient algorithm (also known as the Frank-Wolfe algorithm) has recently regained popularity in the machine learning community due to its projection-free property to solve constrained problems.

Fast and Scalable Adversarial Training of Kernel SVM via Doubly Stochastic Gradients

1 code implementation21 Jul 2021 Huimin Wu, Zhengmian Hu, Bin Gu

Although a wide range of researches have been done in recent years to improve the adversarial robustness of learning models, but most of them are limited to deep neural networks (DNNs) and the work for kernel SVM is still vacant.

Adversarial Robustness

Topic Classification on Spoken Documents Using Deep Acoustic and Linguistic Features

no code implementations16 Jun 2021 Tan Liu, Wu Guo, Bin Gu

In this paper, instead of using the ASR transcripts, the fusion of deep acoustic and linguistic features is used for topic classification on spoken documents.

Automatic Speech Recognition Automatic Speech Recognition (ASR) +3

Learning Sampling Policy for Faster Derivative Free Optimization

no code implementations9 Apr 2021 Zhou Zhai, Bin Gu, Heng Huang

To explore this problem, in this paper, we propose a new reinforcement learning based ZO algorithm (ZO-RL) with learning the sampling policy for generating the perturbations in ZO optimization instead of using random sampling.

reinforcement-learning Reinforcement Learning (RL)

Bidirectional Multiscale Feature Aggregation for Speaker Verification

no code implementations1 Apr 2021 Jiajun Qi, Wu Guo, Bin Gu

In this paper, we propose a novel bidirectional multiscale feature aggregation (BMFA) network with attentional fusion modules for text-independent speaker verification.

Text-Independent Speaker Verification

Improved Meta-Learning Training for Speaker Verification

no code implementations29 Mar 2021 Yafeng Chen, Wu Guo, Bin Gu

By combining these two methods, we can observe further improvements on these two databases.

Data Augmentation Meta-Learning +3

Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating

no code implementations1 Mar 2021 Qingsong Zhang, Bin Gu, Cheng Deng, Heng Huang

Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage.

Vertical Federated Learning

Optimizing Large-Scale Hyperparameters via Automated Learning Algorithm

1 code implementation17 Feb 2021 Bin Gu, Guodong Liu, yanfu Zhang, Xiang Geng, Heng Huang

Modern machine learning algorithms usually involve tuning multiple (from one to thousands) hyperparameters which play a pivotal role in terms of model generalizability.

Hyperparameter Optimization

Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee

no code implementations1 Sep 2020 Junyi Li, Bin Gu, Heng Huang

In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation.

Representation Learning

Privacy-Preserving Asynchronous Federated Learning Algorithms for Multi-Party Vertically Collaborative Learning

no code implementations14 Aug 2020 Bin Gu, An Xu, Zhouyuan Huo, Cheng Deng, Heng Huang

To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.

Federated Learning Privacy Preserving

Federated Doubly Stochastic Kernel Learning for Vertically Partitioned Data

no code implementations14 Aug 2020 Bin Gu, Zhiyuan Dang, Xiang Li, Heng Huang

In this paper, we focus on nonlinear learning with kernels, and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data.

BIG-bench Machine Learning Federated Learning

Fast OSCAR and OWL Regression via Safe Screening Rules

1 code implementation29 Jun 2020 Runxue Bao, Bin Gu, Heng Huang

Moreover, we prove that the algorithms with our screening rule are guaranteed to have identical results with the original algorithms.

regression Sparse Learning

Faster On-Device Training Using New Federated Momentum Algorithm

no code implementations6 Feb 2020 Zhouyuan Huo, Qian Yang, Bin Gu, Lawrence Carin. Heng Huang

Mobile crowdsensing has gained significant attention in recent years and has become a critical paradigm for emerging Internet of Things applications.

Federated Learning

Large Batch Training Does Not Need Warmup

no code implementations4 Feb 2020 Zhouyuan Huo, Bin Gu, Heng Huang

Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications.

Quadruply Stochastic Gradient Method for Large Scale Nonlinear Semi-Supervised Ordinal Regression AUC Optimization

no code implementations24 Dec 2019 Wanli Shi, Bin Gu, Xinag Li, Heng Huang

Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled.

regression

Safe Sample Screening for Robust Support Vector Machine

no code implementations24 Dec 2019 Zhou Zhai, Bin Gu, Xiang Li, Heng Huang

To address this challenge, in this paper, we propose two safe sample screening rules for RSVM based on the framework of concave-convex procedure (CCCP).

Quadruply Stochastic Gradients for Large Scale Nonlinear Semi-Supervised AUC Optimization

no code implementations29 Jul 2019 Wanli Shi, Bin Gu, Xiang Li, Xiang Geng, Heng Huang

To address this problem, in this paper, we propose a novel scalable quadruply stochastic gradient algorithm (QSG-S2AUC) for nonlinear semi-supervised AUC optimization.

Stochastic Optimization

Scalable Semi-Supervised SVM via Triply Stochastic Gradients

no code implementations26 Jul 2019 Xiang Geng, Bin Gu, Xiang Li, Wanli Shi, Guansheng Zheng, Heng Huang

Specifically, to handle two types of data instances involved in S$^3$VM, TSGS$^3$VM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient.

Faster Gradient-Free Proximal Stochastic Methods for Nonconvex Nonsmooth Optimization

no code implementations16 Feb 2019 Feihu Huang, Bin Gu, Zhouyuan Huo, Songcan Chen, Heng Huang

Proximal gradient method has been playing an important role to solve many machine learning tasks, especially for the nonsmooth problems.

BIG-bench Machine Learning

Training Neural Networks Using Features Replay

no code implementations NeurIPS 2018 Zhouyuan Huo, Bin Gu, Heng Huang

Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network.

Faster Derivative-Free Stochastic Algorithm for Shared Memory Machines

no code implementations ICML 2018 Bin Gu, Zhouyuan Huo, Cheng Deng, Heng Huang

Asynchronous parallel stochastic gradient optimization has been playing a pivotal role to solve large-scale machine learning problems in big data applications.

Ensemble Learning

Decoupled Parallel Backpropagation with Convergence Guarantee

3 code implementations ICML 2018 Zhouyuan Huo, Bin Gu, Qian Yang, Heng Huang

The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully leveraging the computing resources.

Accelerated Method for Stochastic Composition Optimization with Nonsmooth Regularization

no code implementations10 Nov 2017 Zhouyuan Huo, Bin Gu, Ji Liu, Heng Huang

To the best of our knowledge, our method admits the fastest convergence rate for stochastic composition optimization: for strongly convex composition problem, our algorithm is proved to admit linear convergence; for general composition problem, our algorithm significantly improves the state-of-the-art convergence rate from $O(T^{-1/2})$ to $O((n_1+n_2)^{{2}/{3}}T^{-1})$.

Management reinforcement-learning +1

Inexact Proximal Gradient Methods for Non-convex and Non-smooth Optimization

no code implementations18 Dec 2016 Bin Gu, De Wang, Zhouyuan Huo, Heng Huang

The theoretical results show that our inexact proximal gradient algorithms can have the same convergence rates as the ones of exact proximal gradient algorithms in the non-convex setting.

BIG-bench Machine Learning

Zeroth-order Asynchronous Doubly Stochastic Algorithm with Variance Reduction

no code implementations5 Dec 2016 Bin Gu, Zhouyuan Huo, Heng Huang

The convergence rate of existing asynchronous doubly stochastic zeroth order algorithms is $O(\frac{1}{\sqrt{T}})$ (also for the sequential stochastic zeroth-order optimization algorithms).

Asynchronous Stochastic Block Coordinate Descent with Variance Reduction

no code implementations29 Oct 2016 Bin Gu, Zhouyuan Huo, Heng Huang

In this paper, we focus on a composite objective function consisting of a smooth convex function $f$ and a block separable convex function, which widely exists in machine learning and computer vision.

Stochastic Optimization

Decoupled Asynchronous Proximal Stochastic Gradient Descent with Variance Reduction

no code implementations22 Sep 2016 Zhouyuan Huo, Bin Gu, Heng Huang

In this paper, we propose a faster method, decoupled asynchronous proximal stochastic variance reduced gradient descent method (DAP-SVRG).

Cannot find the paper you are looking for? You can Submit a new open access paper.