Search Results for author: Minyi Guo

Found 24 papers, 10 papers with code

SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequences

no code implementations29 Jun 2022 Guan Shen, Jieru Zhao, Quan Chen, Jingwen Leng, Chao Li, Minyi Guo

However, the quadratic complexity of self-attention w. r. t the sequence length incurs heavy computational and memory burdens, especially for tasks with long sequences.

Transkimmer: Transformer Learns to Layer-wise Skim

no code implementations ACL 2022 Yue Guan, Zhengyi Li, Jingwen Leng, Zhouhan Lin, Minyi Guo

To address the above limitations, we propose the Transkimmer architecture, which learns to identify hidden state tokens that are not required by each layer.

Computer Vision Natural Language Processing

SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation

1 code implementation ICLR 2022 Cong Guo, Yuxian Qiu, Jingwen Leng, Xiaotian Gao, Chen Zhang, Yunxin Liu, Fan Yang, Yuhao Zhu, Minyi Guo

This paper proposes an on-the-fly DFQ framework with sub-second quantization time, called SQuant, which can quantize networks on inference-only devices with low computation and memory requirements.

Data Free Quantization

Block-Skim: Efficient Question Answering for Transformer

1 code implementation16 Dec 2021 Yue Guan, Zhengyi Li, Jingwen Leng, Zhouhan Lin, Minyi Guo, Yuhao Zhu

We further prune the hidden states corresponding to the unnecessary positions early in lower layers, achieving significant inference-time speedup.

Natural Language Processing Question Answering

Dubhe: Towards Data Unbiasedness with Homomorphic Encryption in Federated Learning Client Selection

no code implementations8 Sep 2021 Shulai Zhang, Zirui Li, Quan Chen, Wenli Zheng, Jingwen Leng, Minyi Guo

Federated learning (FL) is a distributed machine learning paradigm that allows clients to collaboratively train a model over their own local data.

Federated Learning

TempNet: Online Semantic Segmentation on Large-Scale Point Cloud Series

no code implementations ICCV 2021 Yunsong Zhou, Hongzi Zhu, Chunqin Li, Tiankai Cui, Shan Chang, Minyi Guo

In this paper, we propose a light-weight semantic segmentation framework for large-scale point cloud series, called TempNet, which can improve both the accuracy and the stability of existing semantic segmentation models by combining a novel frame aggregation scheme.

Autonomous Driving Point Cloud Segmentation +2

Block Skim Transformer for Efficient Question Answering

no code implementations1 Jan 2021 Yue Guan, Jingwen Leng, Yuhao Zhu, Minyi Guo

Following this idea, we proposed Block Skim Transformer (BST) to improve and accelerate the processing of transformer QA models.

Language Modelling Model Compression +2

How Far Does BERT Look At: Distance-based Clustering and Analysis of BERT's Attention

no code implementations COLING 2020 Yue Guan, Jingwen Leng, Chao Li, Quan Chen, Minyi Guo

Recent research on the multi-head attention mechanism, especially that in pre-trained models such as BERT, has shown us heuristics and clues in analyzing various aspects of the mechanism.

How Far Does BERT Look At:Distance-based Clustering and Analysis of BERT$'$s Attention

no code implementations2 Nov 2020 Yue Guan, Jingwen Leng, Chao Li, Quan Chen, Minyi Guo

Recent research on the multi-head attention mechanism, especially that in pre-trained models such as BERT, has shown us heuristics and clues in analyzing various aspects of the mechanism.

Architectural Implications of Graph Neural Networks

no code implementations2 Sep 2020 Zhihui Zhang, Jingwen Leng, Lingxiao Ma, Youshan Miao, Chao Li, Minyi Guo

Graph neural networks (GNN) represent an emerging line of deep learning models that operate on graph structures.

Balancing Efficiency and Flexibility for DNN Acceleration via Temporal GPU-Systolic Array Integration

no code implementations18 Feb 2020 Cong Guo, Yangjie Zhou, Jingwen Leng, Yuhao Zhu, Zidong Du, Quan Chen, Chao Li, Bin Yao, Minyi Guo

We propose Simultaneous Multi-mode Architecture (SMA), a novel architecture design and execution model that offers general-purpose programmability on DNN accelerators in order to accelerate end-to-end applications.

Adversarial Defense Through Network Profiling Based Path Extraction

no code implementations CVPR 2019 Yuxian Qiu, Jingwen Leng, Cong Guo, Quan Chen, Chao Li, Minyi Guo, Yuhao Zhu

Recently, researchers have started decomposing deep neural network models according to their semantics or functions.

Adversarial Defense

Position-Aware Convolutional Networks for Traffic Prediction

no code implementations12 Apr 2019 Shiheng Ma, Jingcai Guo, Song Guo, Minyi Guo

Our approach employs the inception backbone network to capture rich features of traffic distribution on the whole area.

Traffic Prediction

Knowledge Graph Convolutional Networks for Recommender Systems

8 code implementations18 Mar 2019 Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, Minyi Guo

To alleviate sparsity and cold start problem of collaborative filtering based recommender systems, researchers and engineers usually collect attributes of users and items, and design delicate algorithms to exploit these additional information.

Click-Through Rate Prediction Collaborative Filtering +2

Multi-Task Feature Learning for Knowledge Graph Enhanced Recommendation

4 code implementations23 Jan 2019 Hongwei Wang, Fuzheng Zhang, Miao Zhao, Wenjie Li, Xing Xie, Minyi Guo

Collaborative filtering often suffers from sparsity and cold start problems in real recommendation scenarios, therefore, researchers and engineers usually use side information to address the issues and improve the performance of recommender systems.

Collaborative Filtering Knowledge Graph Embedding +4

Effective Path: Know the Unknowns of Neural Network

no code implementations27 Sep 2018 Yuxian Qiu, Jingwen Leng, Yuhao Zhu, Quan Chen, Chao Li, Minyi Guo

Despite their enormous success, there is still no solid understanding of deep neural network’s working mechanism.

RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems

10 code implementations9 Mar 2018 Hongwei Wang, Fuzheng Zhang, Jialin Wang, Miao Zhao, Wenjie Li, Xing Xie, Minyi Guo

To address the sparsity and cold start problem of collaborative filtering, researchers usually make use of side information, such as social networks or item attributes, to improve recommendation performance.

Click-Through Rate Prediction Collaborative Filtering +2

Personalized Exposure Control Using Adaptive Metering and Reinforcement Learning

no code implementations6 Mar 2018 Huan Yang, Baoyuan Wang, Noranart Vesdapunt, Minyi Guo, Sing Bing Kang

We propose a reinforcement learning approach for real-time exposure control of a mobile camera that is personalizable.


DKN: Deep Knowledge-Aware Network for News Recommendation

4 code implementations25 Jan 2018 Hongwei Wang, Fuzheng Zhang, Xing Xie, Minyi Guo

To solve the above problems, in this paper, we propose a deep knowledge-aware network (DKN) that incorporates knowledge graph representation into news recommendation.

Click-Through Rate Prediction Common Sense Reasoning +2

Joint Topic-Semantic-aware Social Recommendation for Online Voting

1 code implementation3 Dec 2017 Hongwei Wang, Jia Wang, Miao Zhao, Jiannong Cao, Minyi Guo

JTS-MF model calculates similarity among users and votings by combining their TEWE representation and structural information of social networks, and preserves this topic-semantic-social similarity during matrix factorization.

SHINE: Signed Heterogeneous Information Network Embedding for Sentiment Link Prediction

1 code implementation3 Dec 2017 Hongwei Wang, Fuzheng Zhang, Min Hou, Xing Xie, Minyi Guo, Qi Liu

First, due to the lack of explicit sentiment links in mainstream social networks, we establish a labeled heterogeneous sentiment dataset which consists of users' sentiment relation, social relation and profile knowledge by entity-level sentiment extraction method.

Link Prediction Network Embedding +1

Unsupervised Extraction of Video Highlights Via Robust Recurrent Auto-encoders

no code implementations ICCV 2015 Huan Yang, Baoyuan Wang, Stephen Lin, David Wipf, Minyi Guo, Baining Guo

With the growing popularity of short-form video sharing platforms such as \em{Instagram} and \em{Vine}, there has been an increasing need for techniques that automatically extract highlights from video.

Cannot find the paper you are looking for? You can Submit a new open access paper.