no code implementations • ECCV 2020 • Zhihang Yuan, Bingzhe Wu, Guangyu Sun, Zheng Liang, Shiwan Zhao, Weichen Bi
To this end, based on a given CNN model, we first generate a CNN architecture space in which each architecture is a multi-stage CNN generated from the given model using some predefined transformations.
no code implementations • 2 May 2024 • Matias Mendieta, Guangyu Sun, Chen Chen
Federated learning (FL) enables multiple clients to train models collectively while preserving data privacy.
no code implementations • 18 Apr 2024 • Guangyu Sun, Matias Mendieta, Aritra Dutta, Xin Li, Chen Chen
Multi-modal transformers mark significant progress in different domains, but siloed high-quality data hinders their further improvement.
2 code implementations • 26 Feb 2024 • Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, Yan Yan, Beidi Chen, Guangyu Sun, Kurt Keutzer
Our survey stands out from traditional literature reviews by not only summarizing the current state of research but also by introducing a framework based on roofline model for systematic analysis of LLM inference techniques.
1 code implementation • 10 Dec 2023 • Zhihang Yuan, Yuzhang Shang, Yue Song, Qiang Wu, Yan Yan, Guangyu Sun
Based on the success of the low-rank decomposition of projection matrices in the self-attention module, we further introduce ASVD to compress the KV cache.
1 code implementation • ICCV 2023 • Guangyu Sun, Matias Mendieta, Jun Luo, Shandong Wu, Chen Chen
Personalized Federated Learning (PFL) represents a promising solution for decentralized learning in heterogeneous data environments.
1 code implementation • 3 Apr 2023 • Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, Bingzhe Wu
In this paper, we identify that the challenge in quantizing activations in LLMs arises from varying ranges across channels, rather than solely the presence of outliers.
no code implementations • 23 Mar 2023 • Zhihang Yuan, Jiawei Liu, Jiaxiang Wu, Dawei Yang, Qiang Wu, Guangyu Sun, Wenyu Liu, Xinggang Wang, Bingzhe Wu
Post-training quantization (PTQ) is a popular method for compressing deep neural networks (DNNs) without modifying their original architecture or training procedures.
2 code implementations • 12 Oct 2022 • Yizeng Han, Zhihang Yuan, Yifan Pu, Chenhao Xue, Shiji Song, Guangyu Sun, Gao Huang
The latency prediction model can efficiently estimate the inference latency of dynamic networks by simultaneously considering algorithms, scheduling strategies, and hardware properties.
no code implementations • 4 Oct 2022 • Guangyu Sun, Umar Khalid, Matias Mendieta, Taojiannan Yang, Chen Chen
Recently, the use of small pre-trained models has been shown effective in federated learning optimization and improving convergence.
no code implementations • 20 May 2022 • Bingzhe Wu, Jintang Li, Junchi Yu, Yatao Bian, Hengtong Zhang, Chaochao Chen, Chengbin Hou, Guoji Fu, Liang Chen, Tingyang Xu, Yu Rong, Xiaolin Zheng, Junzhou Huang, Ran He, Baoyuan Wu, Guangyu Sun, Peng Cui, Zibin Zheng, Zhe Liu, Peilin Zhao
Deep graph learning has achieved remarkable progresses in both business and scientific areas ranging from finance and e-commerce, to drug and advanced material discovery.
no code implementations • 12 Dec 2021 • Guangyu Sun, Zhang Liu, Lianggong Wen, Jing Shi, Chenliang Xu
Video anomaly detection aims to identify abnormal events that occurred in videos.
1 code implementation • 24 Nov 2021 • Zhihang Yuan, Chenhao Xue, Yiqi Chen, Qiang Wu, Guangyu Sun
We observe the distributions of activation values after softmax and GELU functions are quite different from the Gaussian distribution.
no code implementations • 1 Nov 2021 • Zhe Zhou, Cong Li, Xuechao Wei, Xiaoyang Wang, Guangyu Sun
However, to realize efficient GNN training is challenging, especially on large graphs.
no code implementations • 18 Oct 2021 • Zhe Zhou, Junlin Liu, Zhenyu Gu, Guangyu Sun
To enable such an algorithm with lower latency and better energy efficiency, we also propose an Energon co-processor architecture.
no code implementations • 15 Oct 2021 • Zhihang Yuan, Yiqi Chen, Chenhao Xue, Chenguang Zhang, Qiankun Wang, Guangyu Sun
Network quantization is a powerful technique to compress convolutional neural networks.
1 code implementation • 26 Aug 2021 • Xin Liu, Mingyu Yan, Shuhan Song, Zhengyang Lv, WenMing Li, Guangyu Sun, Xiaochun Ye, Dongrui Fan
Extensive experiments show that our method is universal to mainstream sampling algorithms and helps significantly reduce the training time, especially in large-scale graphs.
no code implementations • 13 Apr 2021 • Zhe Zhou, Bizhao Shi, Zhe Zhang, Yijin Guan, Guangyu Sun, Guojie Luo
At the hardware design level, we propose a pipelined CirCore architecture, which supports efficient block-circulant matrices computation.
no code implementations • 19 Sep 2020 • Zhihang Yuan, Xin Liu, Bingzhe Wu, Guangyu Sun
The inference of a input sample can exit from early stage if the prediction of the stage is confident enough.
no code implementations • 16 Nov 2019 • Zhihang Yuan, Bingzhe Wu, Zheng Liang, Shiwan Zhao, Weichen Bi, Guangyu Sun
Recently, dynamic inference has emerged as a promising way to reduce the computational cost of deep convolutional neural network (CNN).
no code implementations • 5 Oct 2019 • Bingzhe Wu, Chaochao Chen, Shiwan Zhao, Cen Chen, Yuan YAO, Guangyu Sun, Li Wang, Xiaolu Zhang, Jun Zhou
Based on this framework, we demonstrate that SGLD can prevent the information leakage of the training dataset to a certain extent.
no code implementations • NeurIPS 2019 • Bingzhe Wu, Shiwan Zhao, Chaochao Chen, Haoyang Xu, Li Wang, Xiaolu Zhang, Guangyu Sun, Jun Zhou
In this paper, we aim to understand the generalization properties of generative adversarial networks (GANs) from a new perspective of privacy protection.
1 code implementation • 26 Jul 2019 • Ming Liu, Dongpeng Liu, Guangyu Sun, Yi Zhao, Duolin Wang, Fangxing Liu, Xiang Fang, Qing He, Dong Xu
Detecting inaccurate smart meters and targeting them for replacement can save significant resources.
no code implementations • 3 Jun 2019 • Peichen Xie, Bingzhe Wu, Guangyu Sun
Specifically, we use homomorphic encryption to protect a client's raw data and use Bayesian neural networks to protect the DNN weights in a cloud server.
no code implementations • CVPR 2019 • Bingzhe Wu, Shiwan Zhao, Guangyu Sun, Xiaolu Zhang, Zhong Su, Caihong Zeng, Zhihong Liu
(2) privacy leakage: the model trained using a conventional method may involuntarily reveal the private information of the patients in the training dataset.
no code implementations • IEEE 2019 • Yun Ju, Guangyu Sun, Quanhe Chen, Min Zhang, Huixian Zhu, Mujeeb Ur Rehman
In this paper, a new forecasting model based on a convolution neural network and LightGBM is constructed.
no code implementations • 30 Jun 2018 • Bingzhe Wu, Xiaolu Zhang, Shiwan Zhao, Lingxi Xie, Caihong Zeng, Zhihong Liu, Guangyu Sun
Given an input image from a specified stain, several generators are first applied to estimate its appearances in other staining methods, and a classifier follows to combine visual cues from different stains for prediction (whether it is pathological, or which type of pathology it has).
no code implementations • 16 Dec 2017 • Bingzhe Wu, Haodong Duan, Zhichao Liu, Guangyu Sun
In this paper, we build a super resolution perceptual generative adversarial network (SRPGAN) framework for SISR tasks.