Search Results for author:
Found 35 papers, 15 papers with code
Binarized Convolutional Neural Networks with Separable Filters for Efficient Hardware Acceleration
State-of-the-art convolutional neural networks are enormously costly in both compute and memory, demanding massively parallel GPUs for execution.
Channel Gating Neural Networks
Combining our method with knowledge distillation reduces the compute cost of ResNet-18 by 2. 6$\times$ without accuracy drop on ImageNet.
Building Efficient Deep Neural Networks with Unitary Group Convolutions
UGConvs generalize two disparate ideas in CNN architecture, channel shuffling (i. e. ShuffleNet) and block-circulant networks (i. e. CirCNN), and provide unifying insights that lead to a deeper understanding of each technique.
Improving Neural Network Quantization without Retraining using Outlier Channel Splitting
The majority of existing literature focuses on training quantized DNNs, while this work examines the less-studied topic of quantizing a floating-point model without (re)training.
Painting on Placement: Forecasting Routing Congestion using Conditional Generative Adversarial Nets
Physical design process commonly consumes hours to days for large designs, and routing is known as the most critical step.
GraphZoom: A multi-level spectral approach for accurate and scalable graph embedding
GraphZoom first performs graph fusion to generate a new graph that effectively encodes the topology of the original graph and the node attribute information.
OverQ: Opportunistic Outlier Quantization for Neural Network Accelerators
Specialized hardware for handling activation outliers can enable low-precision neural networks, but at the cost of nontrivial area overhead.
Precision Gating: Improving Neural Network Efficiency with Dynamic Dual-Precision Activations
The proposed approach is applicable to a variety of DNN architectures and significantly reduces the computational cost of DNN execution with almost no accuracy loss.
MGX: Near-Zero Overhead Memory Protection for Data-Intensive Accelerators
This paper introduces MGX, a near-zero overhead memory protection scheme for hardware accelerators.
GuardNN: Secure Accelerator Architecture for Privacy-Preserving Deep Learning
This paper proposes GuardNN, a secure DNN accelerator that provides hardware-based protection for user data and model parameters even in an untrusted environment.
FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems
FeatGraph provides a flexible programming interface to express diverse GNN models by composing coarse-grained sparse templates with fine-grained user-defined functions (UDFs) on each vertex/edge.
Logic Synthesis Meets Machine Learning: Trading Exactness for Generalization
If the function is incompletely-specified, the implementation has to be true only on the care set.
FracBNN: Accurate and FPGA-Efficient Binary Neural Networks with Fractional Activations
We design an efficient FPGA-based accelerator for our novel BNN model that supports the fractional activations.
SPADE: A Spectral Method for Black-Box Adversarial Robustness Evaluation
A black-box spectral method is introduced for evaluating the adversarial robustness of a given machine learning (ML) model.
Enabling Design Methodologies and Future Trends for Edge AI: Specialization and Co-design
Artificial intelligence (AI) technologies have dramatically advanced in recent years, resulting in revolutionary changes in people's lives.
Dense Pruning of Pointwise Convolutions in the Frequency Domain
Our key insights are that 1) pointwise convolutions commute with frequency transformation and thus can be computed in the frequency domain without modification, 2) each channel within a given layer has a different level of sensitivity to frequency domain pruning, and 3) each channel's sensitivity to frequency pruning is approximately monotonic with respect to frequency.
GARNET: A Spectral Approach to Robust and Scalable Graph Neural Networks
In this paper, we propose GARNET, a scalable spectral method to boost the adversarial robustness of GNN models for both homophilic and heterophilic graphs.
BulletTrain: Accelerating Robust Neural Network Training via Boundary Example Mining
Neural network robustness has become a central topic in machine learning in recent years.
PokeBNN: A Binary Pursuit of Lightweight Accuracy
In order to enable joint optimization of the cost together with accuracy, we define arithmetic computation effort (ACE), a hardware- and energy-inspired cost metric for quantized and binarized networks.
Ranked #1 on
Binarization
on ImageNet
GARNET: Reduced-Rank Topology Learning for Robust and Scalable Graph Neural Networks
Graph neural networks (GNNs) have been increasingly deployed in various applications that involve learning on non-Euclidean data.
Understanding Hyperdimensional Computing for Parallel Single-Pass Learning
Using representation theory, we characterize which similarity matrices can be "expressed" by finite group VSA hypervectors, and we show how these VSAs can be constructed.
Structured Pruning is All You Need for Pruning CNNs at Initialization
In addition, since PreCropping compresses CNNs at initialization, the computational and memory costs of CNNs are reduced for both training and inference on commodity hardware.
Analysis and Optimization of GNN-Based Recommender Systems on Persistent Memory
To understand whether persistent memory is a good fit for GNNRecSys training, we perform an in-depth characterization of GNNRecSys workloads and a comprehensive analysis of their performance on a persistent memory device, namely, Intel Optane.
Binarized Neural Machine Translation
In this work, we propose a novel binarization technique for Transformers applied to machine translation (BMT), the first of its kind.
Slapo: A Schedule Language for Progressive Optimization of Large Deep Learning Model Training
Specifically, Slapo works on a PyTorch model and uses a set of schedule primitives to convert the model for common model training optimizations such as high-performance kernels, effective 3D parallelism, and efficient activation checkpointing.
FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search
With the proposed integer quantization search, we increase the accuracy of ResNet-18 on ImageNet by 1. 31% points and ResNet-50 by 0. 90% points with equivalent model cost over previous methods.
Understanding the Potential of FPGA-Based Spatial Acceleration for Large Language Model Inference
Experimental results demonstrate our approach can achieve up to 13. 4x speedup when compared to previous FPGA-based accelerators for the BERT model.
Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs
Quantization is a crucial technique for deploying deep learning models on resource-constrained devices, such as embedded FPGAs.
SAGMAN: Stability Analysis of Graph Neural Networks on the Manifolds
Modern graph neural networks (GNNs) can be sensitive to changes in the input graph structure and node features, potentially resulting in unpredictable behavior and degraded performance.
Exploring the Limits of Semantic Image Compression at Micro-bits per Pixel
Traditional methods, such as JPEG, perform image compression by operating on structural information, such as pixel values or frequency content.
Polynormer: Polynomial-Expressive Graph Transformer in Linear Time
To enable the base model permutation equivariant, we integrate it with graph topology and node features separately, resulting in local and global equivariant attention models.
Ranked #1 on
Node Classification
on pokec
Less is More: Hop-Wise Graph Attention for Scalable and Generalizable Learning on Circuits
In this work we propose HOGA, a novel attention-based model for learning circuit representations in a scalable and generalizable manner.
UniSparse: An Intermediate Language for General Sparse Format Customization
The ongoing trend of hardware specialization has led to a growing use of custom data formats when processing sparse workloads, which are typically memory-bound.
Allo: A Programming Model for Composable Accelerator Design
For the GPT2 model, the inference latency of the Allo generated accelerator is 1. 7x faster than the NVIDIA A100 GPU with 5. 4x higher energy efficiency, demonstrating the capability of Allo to handle large-scale designs.
Radial Networks: Dynamic Layer Routing for High-Performance Large Language Models
In this work, we explore the practicality of layer sparsity by profiling residual connections and establish the relationship between model depth and layer sparsity.
