Search Results for author: Zhiru Zhang

Found 26 papers, 10 papers with code

FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search

no code implementations7 Aug 2023 Jordan Dotzel, Gang Wu, Andrew Li, Muhammad Umar, Yun Ni, Mohamed S. Abdelfattah, Zhiru Zhang, Liqun Cheng, Martin G. Dixon, Norman P. Jouppi, Quoc V. Le, Sheng Li

With the proposed integer quantization search, we increase the accuracy of ResNet-18 on ImageNet by 1. 31% points and ResNet-50 by 0. 90% points with equivalent model cost over previous methods.


Decoupled Model Schedule for Deep Learning Training

no code implementations16 Feb 2023 Hongzheng Chen, Cody Hao Yu, Shuai Zheng, Zhen Zhang, Zhiru Zhang, Yida Wang

Specifically, the schedule works on a PyTorch model and uses a set of schedule primitives to convert the model for common model training optimizations such as high-performance kernels, effective 3D parallelism, and efficient activation checkpointing.


Binarized Neural Machine Translation

no code implementations9 Feb 2023 Yichi Zhang, Ankush Garg, Yuan Cao, Łukasz Lew, Behrooz Ghorbani, Zhiru Zhang, Orhan Firat

In this work, we propose a novel binarization technique for Transformers applied to machine translation (BMT), the first of its kind.

Binarization Machine Translation +2

Analysis and Optimization of GNN-Based Recommender Systems on Persistent Memory

no code implementations25 Jul 2022 Yuwei Hu, Jiajie Li, Zhongming Yu, Zhiru Zhang

To understand whether persistent memory is a good fit for GNNRecSys training, we perform an in-depth characterization of GNNRecSys workloads and a comprehensive analysis of their performance on a persistent memory device, namely, Intel Optane.

Link Prediction Recommendation Systems

Structured Pruning is All You Need for Pruning CNNs at Initialization

no code implementations4 Mar 2022 Yaohui Cai, Weizhe Hua, Hongzheng Chen, G. Edward Suh, Christopher De Sa, Zhiru Zhang

In addition, since PreCropping compresses CNNs at initialization, the computational and memory costs of CNNs are reduced for both training and inference on commodity hardware.

Model Compression

Understanding Hyperdimensional Computing for Parallel Single-Pass Learning

1 code implementation10 Feb 2022 Tao Yu, Yichi Zhang, Zhiru Zhang, Christopher De Sa

Using representation theory, we characterize which similarity matrices can be "expressed" by finite group VSA hypervectors, and we show how these VSAs can be constructed.

GARNET: Reduced-Rank Topology Learning for Robust and Scalable Graph Neural Networks

1 code implementation30 Jan 2022 Chenhui Deng, Xiuyu Li, Zhuo Feng, Zhiru Zhang

Graph neural networks (GNNs) have been increasingly deployed in various applications that involve learning on non-Euclidean data.

Adversarial Robustness

PokeBNN: A Binary Pursuit of Lightweight Accuracy

1 code implementation CVPR 2022 Yichi Zhang, Zhiru Zhang, Lukasz Lew

In order to enable joint optimization of the cost together with accuracy, we define arithmetic computation effort (ACE), a hardware- and energy-inspired cost metric for quantized and binarized networks.


GARNET: A Spectral Approach to Robust and Scalable Graph Neural Networks

no code implementations29 Sep 2021 Chenhui Deng, Xiuyu Li, Zhuo Feng, Zhiru Zhang

In this paper, we propose GARNET, a scalable spectral method to boost the adversarial robustness of GNN models for both homophilic and heterophilic graphs.

Adversarial Robustness Graph Embedding

Dense Pruning of Pointwise Convolutions in the Frequency Domain

no code implementations16 Sep 2021 Mark Buckler, Neil Adit, Yuwei Hu, Zhiru Zhang, Adrian Sampson

Our key insights are that 1) pointwise convolutions commute with frequency transformation and thus can be computed in the frequency domain without modification, 2) each channel within a given layer has a different level of sensitivity to frequency domain pruning, and 3) each channel's sensitivity to frequency pruning is approximately monotonic with respect to frequency.

Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design

no code implementations25 Mar 2021 Cong Hao, Jordan Dotzel, JinJun Xiong, Luca Benini, Zhiru Zhang, Deming Chen

Artificial intelligence (AI) technologies have dramatically advanced in recent years, resulting in revolutionary changes in people's lives.

Benchmarking Edge-computing

SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation

1 code implementation7 Feb 2021 Wuxinlin Cheng, Chenhui Deng, Zhiqiang Zhao, Yaohui Cai, Zhiru Zhang, Zhuo Feng

A black-box spectral method is introduced for evaluating the adversarial robustness of a given machine learning (ML) model.

Adversarial Robustness Graph Embedding

GuardNN: Secure Accelerator Architecture for Privacy-Preserving Deep Learning

no code implementations26 Aug 2020 Weizhe Hua, Muhammad Umar, Zhiru Zhang, G. Edward Suh

This paper proposes GuardNN, a secure DNN accelerator that provides hardware-based protection for user data and model parameters even in an untrusted environment.

Privacy Preserving Privacy Preserving Deep Learning

FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems

no code implementations26 Aug 2020 Yuwei Hu, Zihao Ye, Minjie Wang, Jiali Yu, Da Zheng, Mu Li, Zheng Zhang, Zhiru Zhang, Yida Wang

FeatGraph provides a flexible programming interface to express diverse GNN models by composing coarse-grained sparse templates with fine-grained user-defined functions (UDFs) on each vertex/edge.

MGX: Near-Zero Overhead Memory Protection for Data-Intensive Accelerators

no code implementations20 Apr 2020 Weizhe Hua, Muhammad Umar, Zhiru Zhang, G. Edward Suh

This paper introduces MGX, a near-zero overhead memory protection scheme for hardware accelerators.

Precision Gating: Improving Neural Network Efficiency with Dynamic Dual-Precision Activations

1 code implementation ICLR 2020 Yichi Zhang, Ritchie Zhao, Weizhe Hua, Nayun Xu, G. Edward Suh, Zhiru Zhang

The proposed approach is applicable to a variety of DNN architectures and significantly reduces the computational cost of DNN execution with almost no accuracy loss.


OverQ: Opportunistic Outlier Quantization for Neural Network Accelerators

no code implementations13 Oct 2019 Ritchie Zhao, Jordan Dotzel, Zhanqiu Hu, Preslav Ivanov, Christopher De Sa, Zhiru Zhang

Specialized hardware for handling activation outliers can enable low-precision neural networks, but at the cost of nontrivial area overhead.


GraphZoom: A multi-level spectral approach for accurate and scalable graph embedding

1 code implementation ICLR 2020 Chenhui Deng, Zhiqiang Zhao, Yongyu Wang, Zhiru Zhang, Zhuo Feng

GraphZoom first performs graph fusion to generate a new graph that effectively encodes the topology of the original graph and the node attribute information.

Graph Embedding

Painting on Placement: Forecasting Routing Congestion using Conditional Generative Adversarial Nets

no code implementations15 Apr 2019 Cunxi Yu, Zhiru Zhang

Physical design process commonly consumes hours to days for large designs, and routing is known as the most critical step.

Colorization Translation

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

3 code implementations28 Jan 2019 Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Christopher De Sa, Zhiru Zhang

The majority of existing literature focuses on training quantized DNNs, while this work examines the less-studied topic of quantizing a floating-point model without (re)training.

Language Modelling Neural Network Compression +1

Building Efficient Deep Neural Networks with Unitary Group Convolutions

no code implementations CVPR 2019 Ritchie Zhao, Yuwei Hu, Jordan Dotzel, Christopher De Sa, Zhiru Zhang

UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i. e. ShuffleNet) and block-circulant networks (i. e. CirCNN), and provide unifying insights that lead to a deeper understanding of each technique.

Channel Gating Neural Networks

1 code implementation NeurIPS 2019 Weizhe Hua, Yuan Zhou, Christopher De Sa, Zhiru Zhang, G. Edward Suh

Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2. 6$\times$ without accuracy drop on ImageNet.

Knowledge Distillation Network Pruning

Binarized Convolutional Neural Networks with Separable Filters for Efficient Hardware Acceleration

no code implementations15 Jul 2017 Jeng-Hau Lin, Tianwei Xing, Ritchie Zhao, Zhiru Zhang, Mani Srivastava, Zhuowen Tu, Rajesh K. Gupta

State-of-the-art convolutional neural networks are enormously costly in both compute and memory, demanding massively parallel GPUs for execution.

Cannot find the paper you are looking for? You can Submit a new open access paper.