Search Results for author: Jinyang Guo

Found 23 papers, 9 papers with code

TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models

no code implementations21 Dec 2024 Haocheng Huang, Jiaxin Chen, Jinyang Guo, Ruiyi Zhan, Yunhong Wang

However, most of them fail to tackle with the large variations in the distribution of activations across distinct channels and timesteps, as well as the inconsistent of input between quantization and inference on diffusion models, thus leaving much room for improvement.

Quantization Video Generation

PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models

1 code implementation10 Dec 2024 Zining Wnag, Jinyang Guo, Ruihao Gong, Yang Yong, Aishan Liu, Yushi Huang, Jiaheng Liu, Xianglong Liu

Our PTSBench can provide (1) new observations for a better understanding of the PTS algorithms, (2) in-depth and comprehensive evaluations for the sparsification ability of models, and (3) a well-structured and easy-integrate open-source framework.

BiDM: Pushing the Limit of Quantization for Diffusion Models

1 code implementation8 Dec 2024 Xingyu Zheng, Xianglong Liu, Yichen Bian, Xudong Ma, Yulun Zhang, Jiakai Wang, Jinyang Guo, Haotong Qin

Diffusion models (DMs) have been significantly developed and widely used in various applications due to their excellent generative qualities.

Binarization Image Generation +2

A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithms

no code implementations25 Sep 2024 Ruihao Gong, Yifu Ding, Zining Wang, Chengtao Lv, Xingyu Zheng, Jinyang Du, Haotong Qin, Jinyang Guo, Michele Magno, Xianglong Liu

Large language models (LLMs) have achieved remarkable advancements in natural language processing, showcasing exceptional performance across various tasks.

Quantization

DDK: Distilling Domain Knowledge for Efficient Large Language Models

no code implementations23 Jul 2024 Jiaheng Liu, Chenchen Zhang, Jinyang Guo, Yuanxing Zhang, Haoran Que, Ken Deng, Zhiqi Bai, Jie Liu, Ge Zhang, Jiakai Wang, Yanan Wu, Congnan Liu, Wenbo Su, Jiamang Wang, Lin Qu, Bo Zheng

Despite the advanced intelligence abilities of large language models (LLMs) in various applications, they still face significant computational and storage demands.

Knowledge Distillation

QVD: Post-training Quantization for Video Diffusion Models

no code implementations16 Jul 2024 Shilong Tian, Hong Chen, Chengtao Lv, Yu Liu, Jinyang Guo, Xianglong Liu, Shengxi Li, Hao Yang, Tao Xie

Furthermore, we investigate significant inter-channel disparities and asymmetries in the activation of video diffusion models, resulting in low coverage of quantization levels by individual channels and increasing the challenge of quantization.

Computational Efficiency Quantization

Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes

1 code implementation9 May 2024 Ruihao Gong, Yang Yong, Zining Wang, Jinyang Guo, Xiuying Wei, Yuqing Ma, Xianglong Liu

Previous methods for finding sparsity rates mainly focus on the training-aware scenario, which usually fails to converge stably under the PTS setting with limited data and much less training cost.

PTQ4SAM: Post-Training Quantization for Segment Anything

1 code implementation CVPR 2024 Chengtao Lv, Hong Chen, Jinyang Guo, Yifu Ding, Xianglong Liu

We analyze its characteristics from both per-tensor and per-channel perspectives, and propose a Bimodal Integration strategy, which utilizes a mathematically equivalent sign operation to transform the bimodal distribution into a relatively easy-quantized normal distribution offline.

Instance Segmentation object-detection +4

BinaryDM: Accurate Weight Binarization for Efficient Diffusion Models

1 code implementation8 Apr 2024 Xingyu Zheng, Xianglong Liu, Haotong Qin, Xudong Ma, Mingyuan Zhang, Haojie Hao, Jiakai Wang, Zixiang Zhao, Jinyang Guo, Michele Magno

From the optimization perspective, a Low-rank Representation Mimicking (LRM) is applied to assist the optimization of binarized DMs.

Binarization Quantization

DB-LLM: Accurate Dual-Binarization for Efficient LLMs

no code implementations19 Feb 2024 Hong Chen, Chengtao Lv, Liang Ding, Haotong Qin, Xiabin Zhou, Yifu Ding, Xuebo Liu, Min Zhang, Jinyang Guo, Xianglong Liu, DaCheng Tao

Large language models (LLMs) have significantly advanced the field of natural language processing, while the expensive memory and computation consumption impede their practical deployment.

Binarization Computational Efficiency +1

Reg-PTQ: Regression-specialized Post-training Quantization for Fully Quantized Object Detector

no code implementations CVPR 2024 Yifu Ding, Weilun Feng, Chuyan Chen, Jinyang Guo, Xianglong Liu

However they suffer from severe performance degradation when performing full quantization due to overlooking the unique characteristics of regression tasks in object detection.

Object object-detection +3

LTA-PCS: Learnable Task-Agnostic Point Cloud Sampling

no code implementations CVPR 2024 Jiaheng Liu, Jianhao Li, Kaisiyuan Wang, Hongcheng Guo, Jian Yang, Junran Peng, Ke Xu, Xianglong Liu, Jinyang Guo

Existing task-agnostic point cloud sampling strategy (e. g. FPS) does not consider semantic information of point clouds causing degraded performance on downstream tasks.

RobustMQ: Benchmarking Robustness of Quantized Models

no code implementations4 Aug 2023 Yisong Xiao, Aishan Liu, Tianyuan Zhang, Haotong Qin, Jinyang Guo, Xianglong Liu

Quantization has emerged as an essential technique for deploying deep neural networks (DNNs) on devices with limited resources.

Adversarial Robustness Benchmarking +1

Coarse-to-fine Deep Video Coding with Hyperprior-guided Mode Prediction

no code implementations CVPR 2022 Zhihao Hu, Guo Lu, Jinyang Guo, Shan Liu, Wei Jiang, Dong Xu

The previous deep video compression approaches only use the single scale motion compensation strategy and rarely adopt the mode prediction technique from the traditional standards like H. 264/H. 265 for both motion and residual compression.

Motion Compensation Motion Estimation +2

CoupleFace: Relation Matters for Face Recognition Distillation

no code implementations12 Apr 2022 Jiaheng Liu, Haoyu Qin, Yichao Wu, Jinyang Guo, Ding Liang, Ke Xu

In this work, we observe that mutual relation knowledge between samples is also important to improve the discriminative ability of the learned representation of the student model, and propose an effective face recognition distillation method called CoupleFace by additionally introducing the Mutual Relation Distillation (MRD) into existing distillation framework.

Face Recognition Knowledge Distillation +1

Unsupervised Learning of Accurate Siamese Tracking

1 code implementation CVPR 2022 Qiuhong Shen, Lei Qiao, Jinyang Guo, Peixia Li, Xin Li, Bo Li, Weitao Feng, Weihao Gan, Wei Wu, Wanli Ouyang

As unlimited self-supervision signals can be obtained by tracking a video along a cycle in time, we investigate evolving a Siamese tracker by tracking videos forward-backward.

Visual Object Tracking

CBANet: Towards Complexity and Bitrate Adaptive Deep Image Compression using a Single Network

no code implementations26 May 2021 Jinyang Guo, Dong Xu, Guo Lu

Furthermore, to achieve variable bitrate decoding with one single decoder, we propose a bitrate adaptive module to project the representation from a base bitrate to the expected representation at a target bitrate for transmission.

Decoder Image Compression

Multi-Dimensional Pruning: A Unified Framework for Model Compression

no code implementations CVPR 2020 Jinyang Guo, Wanli Ouyang, Dong Xu

Specifically, in order to reduce the redundancy along the spatial/spatial-temporal dimension, we downsample the input tensor of a convolutional layer, in which the scaling factor for the downsampling operation is adaptively selected by our approach.

Model Compression

Channel Pruning Guided by Classification Loss and Feature Importance

no code implementations15 Mar 2020 Jinyang Guo, Wanli Ouyang, Dong Xu

To this end, we propose a new strategy to suppress the influence of unimportant features (i. e., the features will be removed at the next pruning stage).

Classification Feature Importance +1

Cannot find the paper you are looking for? You can Submit a new open access paper.