no code implementations • ICML 2020 • Runxue Bao, Bin Gu, Heng Huang
Ordered Weight $L_{1}$-Norms (OWL) is a new family of regularizers for high-dimensional sparse regression.
no code implementations • COLING (TextGraphs) 2020 • Zhenqi Zhao, Yuchen Guo, Dingxian Wang, Yufan Huang, Xiangnan He, Bin Gu
Entity Resolution (ER) identifies records that refer to the same real-world entity.
no code implementations • 27 Mar 2024 • Xiaofeng Wu, Velibor Bojkovic, Bin Gu, Kun Suo, Kai Zou
Spiking Neural Networks (SNNs) offer a promising avenue for energy-efficient computing compared with Artificial Neural Networks (ANNs), closely mirroring biological neural processes.
1 code implementation • 20 Feb 2024 • Loka Li, Ignavier Ng, Gongxu Luo, Biwei Huang, Guangyi Chen, Tongliang Liu, Bin Gu, Kun Zhang
This discrepancy has motivated the development of federated causal discovery (FCD) approaches.
no code implementations • 2 Feb 2024 • Hilal AlQuabeh, William de Vazelhes, Bin Gu
Recently, an OGD algorithm emerged, employing gradient computation involving prior and most recent examples, a step that effectively reduces algorithmic complexity to $O(T)$, with $T$ being the number of received examples.
no code implementations • 12 Jan 2024 • Jia Li, Ge Li, YunFei Zhao, Yongmin Li, Zhi Jin, Hao Zhu, Huanyu Liu, Kaibo Liu, Lecheng Wang, Zheng Fang, Lanshen Wang, Jiazheng Ding, Xuanming Zhang, Yihong Dong, Yuqi Zhu, Bin Gu, Mengfei Yang
Compared to previous benchmarks, DevEval aligns to practical projects in multiple dimensions, e. g., real program distributions, sufficient dependencies, and enough-scale project contexts.
1 code implementation • 19 Dec 2023 • William de Vazelhes, Bhaskar Mukhoty, Xiao-Tong Yuan, Bin Gu
However, most of those iterative methods are based on the $\ell_1$ norm which requires restrictive applicability conditions and could fail in many cases.
no code implementations • 15 Dec 2023 • Nan Yin, Mengzhu Wang, Zhenghan Chen, Giulia De Masi, Bin Gu, Huan Xiong
Current work often uses SNNs instead of Recurrent Neural Networks (RNNs) by using binary features instead of continuous ones for efficient training, which would overlooks graph structure information and leads to the loss of details during propagation.
no code implementations • 12 Dec 2023 • Yang Xu, Yongqiang Yao, Yufan Huang, MengNan Qi, Maoquan Wang, Bin Gu, Neel Sundaresan
Instruction tuning, a specialized technique to enhance large language model (LLM) performance via instruction datasets, relies heavily on the quality of employed data.
no code implementations • 22 Oct 2023 • MengNan Qi, Yufan Huang, Maoquan Wang, Yongqiang Yao, Zihan Liu, Bin Gu, Colin Clement, Neel Sundaresan
In this paper we introduce a new metrics for programming language translation and these metrics address these basic syntax errors.
no code implementations • 17 Oct 2023 • Yufan Huang, MengNan Qi, Yongqiang Yao, Maoquan Wang, Bin Gu, Colin Clement, Neel Sundaresan
Distilled code serves as a translation pivot for any programming language, leading by construction to parallel corpora which scale to all available source code by simply applying the distillation compiler.
1 code implementation • 10 Oct 2023 • Hilal AlQuabeh, Bhaskar Mukhoty, Bin Gu
Specifically, we establish a clear connection between the variance of online gradients and the regret, and construct online gradients using the most recent stratified samples with a limited buffer of size of $s$ representing all past data, which have a complexity of $O(sT)$ and employs $O(\sqrt{T}\log{T})$ random Fourier features for kernel approximation.
no code implementations • 12 Sep 2023 • Jiang Liu, Han Su, Yunjun Bai, Bin Gu, Bai Xue, Mengfei Yang, Naijun Zhan
Controller synthesis, including reset controller, feedback controller, and switching logic controller, provides an essential mechanism to guarantee the correctness and reliability of hybrid systems in a correct-by-construction manner.
no code implementations • 12 Sep 2023 • Han Su, Jiyu Zhu, Shenghua Feng, Yunjun Bai, Bin Gu, Jiang Liu, Mengfei Yang, Naijun Zhan
A reset controller plays a crucial role in designing hybrid systems.
no code implementations • 28 Jun 2023 • Ganyu Wang, Qingsong Zhang, Li Xiang, Boyu Wang, Bin Gu, Charles Ling
Meanwhile, the upstream model (server) is updated with first-order optimization (FOO) locally, which significantly improves the convergence rate, making it feasible to train the large models without compromising privacy and security.
no code implementations • 9 Jun 2023 • Shaoan Xie, Biwei Huang, Bin Gu, Tongliang Liu, Kun Zhang
Traditional counterfactual inference, under Pearls' counterfactual framework, typically depends on having access to or estimating a structural causal model.
1 code implementation • 28 Feb 2023 • Zhijie Shen, Wu Guo, Bin Gu
In this paper, we propose a language-universal adapter learning framework based on a pre-trained model for end-to-end multilingual automatic speech recognition (ASR).
Automatic Speech Recognition Automatic Speech Recognition (ASR) +2
no code implementations • 20 Feb 2023 • Jun Chen, Hong Chen, Xue Jiang, Bin Gu, Weifu Li, Tieliang Gong, Feng Zheng
Triplet learning, i. e. learning from triplet data, has attracted much attention in computer vision tasks with an extremely large number of categories, e. g., face recognition and person re-identification.
no code implementations • 20 Feb 2023 • Jiahuan Wang, Jun Chen, Hong Chen, Bin Gu, Weifu Li, Xin Tang
Recently, some mixture algorithms of pointwise and pairwise learning (PPL) have been formulated by employing the hybrid error metric of "pointwise loss + pairwise loss" and have shown empirical effectiveness on feature selection, ranking and recommendation tasks.
no code implementations • 2 Feb 2023 • Bhaskar Mukhoty, Velibor Bojkovic, William de Vazelhes, Giulia De Masi, Huan Xiong, Bin Gu
To circumvent the problem surrogate method uses a differentiable approximation of the Heaviside in the backward pass, while the forward pass uses the Heaviside as the spiking function.
no code implementations • 19 Nov 2022 • Chenkang Zhang, Lei Luo, Bin Gu
To efficiently solve this nonconvex quadratic problem, we propose a doubly stochastic projection coordinate gradient algorithm.
no code implementations • 11 Oct 2022 • William de Vazelhes, Hualin Zhang, Huimin Wu, Xiao-Tong Yuan, Bin Gu
To solve this puzzle, in this paper, we focus on the $\ell_0$ constrained black-box stochastic optimization problems, and propose a new stochastic zeroth-order gradient hard-thresholding (SZOHT) algorithm with a general ZO gradient estimator powered by a novel random support sampling.
no code implementations • 4 Oct 2022 • Hualin Zhang, Huan Xiong, Bin Gu
We consider escaping saddle points of nonconvex problems where only the function evaluations can be accessed.
no code implementations • 15 Sep 2022 • Xingyu Qu, Diyang Li, Xiaohan Zhao, Bin Gu
The SPL regime involves a self-paced regularizer and a gradually increasing age parameter, which plays a key role in SPL but where to optimally terminate this process is still non-trivial to determine.
no code implementations • 11 Aug 2022 • Runxue Bao, Bin Gu, Heng Huang
To address this challenge, we propose a novel accelerated doubly stochastic gradient descent (ADSGD) method for sparsity regularized loss minimization problems, which can reduce the number of block iterations by eliminating inactive coefficients during the optimization process and eventually achieve faster explicit model identification and improve the algorithm efficiency.
no code implementations • 8 Jul 2022 • Bin Gu, Chenkang Zhang, Huan Xiong, Heng Huang
Self-paced learning is an effective method for handling noisy data.
no code implementations • 30 Jun 2022 • Hongchang Gao, Bin Gu, My T. Thai
Bilevel optimization has been applied to a wide variety of machine learning models, and numerous stochastic bilevel optimization algorithms have been developed in recent years.
no code implementations • 21 Jun 2022 • Shao-Qun Zhang, Jia-Yi Chen, Jin-Hui Wu, Gao Zhang, Huan Xiong, Bin Gu, Zhi-Hua Zhou
Initially, we unveil two pivotal components of intrinsic structures: the integration operation and firing-reset mechanism, by elucidating their influence on the expressivity of SNNs.
no code implementations • 6 Jun 2022 • Yuzhen Han, Ruben Solozabal, Jing Dong, Xingyu Zhou, Martin Takac, Bin Gu
To the best of our knowledge, our study establishes the first model-based online algorithm with regret guarantees under LTV dynamical systems.
no code implementations • 19 Mar 2022 • Qingsong Zhang, Bin Gu, Zhiyuan Dang, Cheng Deng, Heng Huang
Based on that, we propose a novel and practical VFL framework with black-box models, which is inseparably interconnected to the promising properties of ZOO.
no code implementations • 26 Feb 2022 • Beiduo Chen, Wu Guo, Bin Gu, Quan Liu, Yongchao Wang
Cross-language pre-trained models such as multilingual BERT (mBERT) have achieved significant performance in various cross-lingual downstream NLP tasks.
no code implementations • 9 Dec 2021 • Junyi Li, Bin Gu, Heng Huang
Combining our new formulation with the alternative update of the inner and outer variables, we propose an efficient fully single loop algorithm.
no code implementations • 29 Sep 2021 • Huimin Wu, Heng Huang, Bin Gu
To adapt to semi-supervised learning problems, they need to estimate labels for unlabeled data in advance, which inevitably degenerates the performance of the learned model due to the bias on the estimation of labels for unlabeled data.
no code implementations • 29 Sep 2021 • Wanli Shi, Hongchang Gao, Bin Gu
In this paper, to solve the nonconvex problem with a large number of white/black-box constraints, we proposed a doubly stochastic zeroth-order gradient method (DSZOG).
no code implementations • 29 Sep 2021 • Zhuang Qian, Shufei Zhang, Kaizhu Huang, Qiufeng Wang, Bin Gu, Huan Xiong, Xinping Yi
It is possibly due to the fact that the conventional adversarial training methods generate adversarial perturbations usually in a supervised way, so that the adversarial samples are highly biased towards the decision boundary, resulting in an inhomogeneous data distribution.
no code implementations • 29 Sep 2021 • Wanli Shi, Heng Huang, Bin Gu
Then, we transform the smoothed bi-level optimization to an unconstrained penalty problem by replacing the smoothed sub-problem with its first-order necessary conditions.
no code implementations • 26 Sep 2021 • Qingsong Zhang, Bin Gu, Cheng Deng, Songxiang Gu, Liefeng Bo, Jian Pei, Heng Huang
To address the challenges of communication and computation resource utilization, we propose an asynchronous stochastic quasi-Newton (AsySQN) framework for VFL, under which three algorithms, i. e. AsySQN-SGD, -SVRG and -SAGA, are proposed.
no code implementations • 18 Sep 2021 • Xiyuan Wei, Bin Gu, Heng Huang
The conditional gradient algorithm (also known as the Frank-Wolfe algorithm) has recently regained popularity in the machine learning community due to its projection-free property to solve constrained problems.
1 code implementation • 21 Jul 2021 • Huimin Wu, Zhengmian Hu, Bin Gu
Although a wide range of researches have been done in recent years to improve the adversarial robustness of learning models, but most of them are limited to deep neural networks (DNNs) and the work for kernel SVM is still vacant.
no code implementations • 16 Jun 2021 • Tan Liu, Wu Guo, Bin Gu
In this paper, instead of using the ASR transcripts, the fusion of deep acoustic and linguistic features is used for topic classification on spoken documents.
Automatic Speech Recognition Automatic Speech Recognition (ASR) +3
no code implementations • 9 Apr 2021 • Zhou Zhai, Bin Gu, Heng Huang
To explore this problem, in this paper, we propose a new reinforcement learning based ZO algorithm (ZO-RL) with learning the sampling policy for generating the perturbations in ZO optimization instead of using random sampling.
no code implementations • 1 Apr 2021 • Jiajun Qi, Wu Guo, Bin Gu
In this paper, we propose a novel bidirectional multiscale feature aggregation (BMFA) network with attentional fusion modules for text-independent speaker verification.
no code implementations • 29 Mar 2021 • Yafeng Chen, Wu Guo, Bin Gu
By combining these two methods, we can observe further improvements on these two databases.
no code implementations • 1 Mar 2021 • Qingsong Zhang, Bin Gu, Cheng Deng, Heng Huang
Vertical federated learning (VFL) attracts increasing attention due to the emerging demands of multi-party collaborative modeling and concerns of privacy leakage.
1 code implementation • 17 Feb 2021 • Bin Gu, Guodong Liu, yanfu Zhang, Xiang Geng, Heng Huang
Modern machine learning algorithms usually involve tuning multiple (from one to thousands) hyperparameters which play a pivotal role in terms of model generalizability.
no code implementations • 1 Sep 2020 • Junyi Li, Bin Gu, Heng Huang
In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation.
no code implementations • 14 Aug 2020 • Bin Gu, An Xu, Zhouyuan Huo, Cheng Deng, Heng Huang
To the best of our knowledge, AFSGD-VP and its SVRG and SAGA variants are the first asynchronous federated learning algorithms for vertically partitioned data.
no code implementations • 14 Aug 2020 • Bin Gu, Zhiyuan Dang, Xiang Li, Heng Huang
In this paper, we focus on nonlinear learning with kernels, and propose a federated doubly stochastic kernel learning (FDSKL) algorithm for vertically partitioned data.
1 code implementation • 29 Jun 2020 • Runxue Bao, Bin Gu, Heng Huang
Moreover, we prove that the algorithms with our screening rule are guaranteed to have identical results with the original algorithms.
no code implementations • 6 Feb 2020 • Zhouyuan Huo, Qian Yang, Bin Gu, Lawrence Carin. Heng Huang
Mobile crowdsensing has gained significant attention in recent years and has become a critical paradigm for emerging Internet of Things applications.
no code implementations • 4 Feb 2020 • Zhouyuan Huo, Bin Gu, Heng Huang
Training deep neural networks using a large batch size has shown promising results and benefits many real-world applications.
no code implementations • 24 Dec 2019 • Wanli Shi, Bin Gu, Xinag Li, Heng Huang
Semi-supervised ordinal regression (S$^2$OR) problems are ubiquitous in real-world applications, where only a few ordered instances are labeled and massive instances remain unlabeled.
no code implementations • 24 Dec 2019 • Zhou Zhai, Bin Gu, Xiang Li, Heng Huang
To address this challenge, in this paper, we propose two safe sample screening rules for RSVM based on the framework of concave-convex procedure (CCCP).
no code implementations • 29 Jul 2019 • Wanli Shi, Bin Gu, Xiang Li, Xiang Geng, Heng Huang
To address this problem, in this paper, we propose a novel scalable quadruply stochastic gradient algorithm (QSG-S2AUC) for nonlinear semi-supervised AUC optimization.
no code implementations • 26 Jul 2019 • Xiang Geng, Bin Gu, Xiang Li, Wanli Shi, Guansheng Zheng, Heng Huang
Specifically, to handle two types of data instances involved in S$^3$VM, TSGS$^3$VM samples a labeled instance and an unlabeled instance as well with the random features in each iteration to compute a triply stochastic gradient.
no code implementations • 16 Feb 2019 • Feihu Huang, Bin Gu, Zhouyuan Huo, Songcan Chen, Heng Huang
Proximal gradient method has been playing an important role to solve many machine learning tasks, especially for the nonsmooth problems.
no code implementations • NeurIPS 2018 • Zhouyuan Huo, Bin Gu, Heng Huang
Training a neural network using backpropagation algorithm requires passing error gradients sequentially through the network.
no code implementations • ICML 2018 • Bin Gu, Zhouyuan Huo, Cheng Deng, Heng Huang
Asynchronous parallel stochastic gradient optimization has been playing a pivotal role to solve large-scale machine learning problems in big data applications.
3 code implementations • ICML 2018 • Zhouyuan Huo, Bin Gu, Qian Yang, Heng Huang
The backward locking in backpropagation algorithm constrains us from updating network layers in parallel and fully leveraging the computing resources.
no code implementations • 10 Nov 2017 • Zhouyuan Huo, Bin Gu, Ji Liu, Heng Huang
To the best of our knowledge, our method admits the fastest convergence rate for stochastic composition optimization: for strongly convex composition problem, our algorithm is proved to admit linear convergence; for general composition problem, our algorithm significantly improves the state-of-the-art convergence rate from $O(T^{-1/2})$ to $O((n_1+n_2)^{{2}/{3}}T^{-1})$.
no code implementations • 18 Dec 2016 • Bin Gu, De Wang, Zhouyuan Huo, Heng Huang
The theoretical results show that our inexact proximal gradient algorithms can have the same convergence rates as the ones of exact proximal gradient algorithms in the non-convex setting.
no code implementations • 5 Dec 2016 • Bin Gu, Zhouyuan Huo, Heng Huang
The convergence rate of existing asynchronous doubly stochastic zeroth order algorithms is $O(\frac{1}{\sqrt{T}})$ (also for the sequential stochastic zeroth-order optimization algorithms).
no code implementations • 29 Oct 2016 • Bin Gu, Zhouyuan Huo, Heng Huang
In this paper, we focus on a composite objective function consisting of a smooth convex function $f$ and a block separable convex function, which widely exists in machine learning and computer vision.
no code implementations • 22 Sep 2016 • Zhouyuan Huo, Bin Gu, Heng Huang
In this paper, we propose a faster method, decoupled asynchronous proximal stochastic variance reduced gradient descent method (DAP-SVRG).