Search Results for author:
Found 101 papers, 60 papers with code
DataMix: Efficient Privacy-Preserving Edge-Cloud Inference
Third, our solution is extit{efficient} on the edge since the majority of the workload is delegated to the cloud, and our mixing and de-mixing processes introduce very few extra computations.
Condition-Aware Neural Network for Controlled Image Generation
In parallel to prior conditional control methods, CAN controls the image generation process by dynamically manipulating the weight of the neural network.
Tiny Machine Learning: Progress and Futures
By squeezing deep learning models into billions of IoT devices and microcontrollers (MCUs), we expand the scope of AI applications and enable ubiquitous intelligence.
DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models
To overcome this dilemma, we observe the high similarity between the input from adjacent diffusion steps and propose displaced patch parallelism, which takes advantage of the sequential nature of the diffusion process by reusing the pre-computed feature maps from the previous timestep to provide context for the current step.
BitDelta: Your Fine-Tune May Only Be Worth One Bit
Large Language Models (LLMs) are typically trained in two phases: pre-training on large internet-scale datasets, and fine-tuning for downstream tasks.
InfLLM: Unveiling the Intrinsic Capacity of LLMs for Understanding Extremely Long Sequences with Training-Free Memory
To alleviate these issues, existing efforts employ sliding attention windows and discard distant tokens to achieve the processing of extremely long sequences.
EfficientViT-SAM: Accelerated Segment Anything Model Without Performance Loss
For the training, we begin with the knowledge distillation from the SAM-ViT-H image encoder to EfficientViT.
QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits
To address these two pain points, we propose QuantumSEA, an in-time sparse exploration for noise-adaptive quantum circuits, aiming to achieve two key objectives: (1) implicit circuits capacity during training - by dynamically exploring the circuit's sparse connectivity and sticking a fixed small number of quantum gates throughout the training which satisfies the coherence time and enjoy light noises, enabling feasible executions on real quantum devices; (2) noise robustness - by jointly optimizing the topology and parameters of quantum circuits under real device noise models.
VILA: On Pre-training for Visual Language Models
Visual language models (VLMs) rapidly progressed with the recent success of large language models.
Ranked #21 on
Visual Question Answering
on MM-Vet
DGR: Tackling Drifted and Correlated Noise in Quantum Error Correction via Decoding Graph Re-weighting
By counting the occurrences of edges and edge pairs in decoded matchings, we can statistically estimate the up-to-date probabilities of each edge and the correlations between them.
RobustState: Boosting Fidelity of Quantum State Preparation via Noise-Aware Variational Training
Arbitrary state preparation algorithms can be broadly categorized into arithmetic decomposition (AD) and variational quantum state preparation (VQSP).
Transformer-QEC: Quantum Error Correction Code Decoding with Transferable Transformers
Quantum Error Correction (QEC) mitigates this by employing redundancy, distributing quantum information across multiple data qubits and utilizing syndrome qubits to monitor their states for errors.
Machine learning's own Industrial Revolution
Machine learning is expected to enable the next Industrial Revolution.
PockEngine: Sparse and Efficient Fine-tuning in a Pocket
On-device learning and efficient fine-tuning enable continuous and privacy-preserving customization (e. g., locally fine-tuning large language models on personalized data).
TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
On top of this, we design the Sparse Autotuner, which extends the design space of existing sparse convolution libraries and searches for the best dataflow configurations for training and inference workloads.
Efficient Streaming Language Models with Attention Sinks
In this paper, we first demonstrate that the emergence of attention sink is due to the strong attention scores towards initial tokens as a "sink" even if they are not semantically important.
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
For example, training on the context length of 8192 needs 16x computational costs in self-attention layers as that of 2048.
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Large language models (LLMs) have shown excellent performance on various tasks, but the astronomical model size raises the hardware barrier for serving (memory size) and slows down token generation (memory bandwidth).
FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
FastComposer proposes delayed subject conditioning in the denoising step to maintain both identity and editability in subject-driven image generation.
Ranked #7 on
Diffusion Personalization Tuning Free
on AgeDB
SparseViT: Revisiting Activation Sparsity for Efficient High-Resolution Vision Transformer
High-resolution images enable neural networks to learn richer visual representations.
Offsite-Tuning: Transfer Learning without Full Model
In this paper, we propose Offsite-Tuning, a privacy-preserving and efficient transfer learning framework that can adapt billion-parameter foundation models to downstream data without access to the full model.
FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer
Transformer, as an alternative to CNN, has been proven effective in many modalities (e. g., texts and images).
EfficientViT: Lightweight Multi-Scale Attention for High-Resolution Dense Prediction
Without performance loss on Cityscapes, our EfficientViT provides up to 8. 8x and 3. 8x GPU latency reduction over SegFormer and SegNeXt, respectively.
SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models
We propose SmoothQuant, a training-free, accuracy-preserving, and general-purpose post-training quantization (PTQ) solution to enable 8-bit weight, 8-bit activation (W8A8) quantization for LLMs.
Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models
With about $1\%$-area edits, SIGE accelerates DDPM by $3. 0\times$ on NVIDIA RTX 3090 and $4. 6\times$ on Apple M1 Pro GPU, Stable Diffusion by $7. 2\times$ on 3090, and GauGAN by $5. 6\times$ on 3090 and $5. 2\times$ on M1 Pro GPU.
QuEst: Graph Transformer for Quantum Circuit Reliability Estimation
Specifically, the TorchQuantum library also supports using data-driven ML models to solve problems in quantum system research, such as predicting the impact of quantum noise on circuit fidelity and improving the quantum circuit compilation efficiency.
NAPA: Intermediate-level Variational Native-pulse Ansatz for Variational Quantum Algorithms
Therefore, we propose NAPA, a native-pulse ansatz generator framework for VQAs.
RobustAnalog: Fast Variation-Aware Analog Circuit Design Via Multi-task RL
To address the challenge, we present RobustAnalog, a robust circuit design framework that involves the variation information in the optimization process.
On-Device Training Under 256KB Memory
To reduce the memory footprint, we propose Sparse Update to skip the gradient computation of less important layers and sub-tensors.
MME-CRS: Multi-Metric Evaluation Based on Correlation Re-Scaling for Evaluating Open-Domain Dialogue
Firstly, we build an evaluation metric composed of 5 groups of parallel sub-metrics called Multi-Metric Evaluation (MME) to evaluate the quality of dialogue comprehensively.
EfficientViT: Multi-Scale Linear Attention for High-Resolution Dense Prediction
Without performance loss on Cityscapes, our EfficientViT provides up to 13. 9$\times$ and 6. 2$\times$ GPU latency reduction over SegFormer and SegNeXt, respectively.
Ranked #23 on
Semantic Segmentation
on Cityscapes val
BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
Multi-sensor fusion is essential for an accurate and reliable autonomous driving system.
Ranked #4 on
3D Object Detection
on nuScenes
Lite Pose: Efficient Architecture Design for 2D Human Pose Estimation
Inspired by this finding, we design LitePose, an efficient single-branch architecture for pose estimation, and introduce two simple approaches to enhance the capacity of LitePose, including Fusion Deconv Head and Large Kernel Convs.
Ranked #5 on
Multi-Person Pose Estimation
on MS COCO
(Validation AP metric)
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech recognition.
PVNAS: 3D Neural Architecture Search with Point-Voxel Convolution
3D neural networks are widely used in real-world applications (e. g., AR/VR headsets, self-driving cars).
TorchSparse: Efficient Point Cloud Inference Engine
TorchSparse directly optimizes the two bottlenecks of sparse convolution: irregular computation and data movement.
QOC: Quantum On-Chip Training with Parameter Shift and Gradient Pruning
Nevertheless, we find that due to the significant quantum errors (noises) on real machines, gradients obtained from naive parameter shift have low fidelity and thus degrading the training accuracy.
AET-SGD: Asynchronous Event-triggered Stochastic Gradient Descent
In this paper, we propose an Asynchronous Event-triggered Stochastic Gradient Descent (SGD) framework, called AET-SGD, to i) reduce the communication cost among the compute nodes, and ii) mitigate the impact of the delay.
Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning
Federated Learning is an emerging direction in distributed machine learning that en-ables jointly training a model without sharing the data.
Memory-efficient Patch-based Inference for Tiny Deep Learning
We further propose receptive field redistribution to shift the receptive field and FLOPs to the later stage and reduce the computation overhead.
VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles
Simulation has the potential to transform the development of robust algorithms for mobile agents deployed in safety-critical scenarios.
MCUNetV2: Memory-Efficient Patch-based Inference for Tiny Deep Learning
We further propose network redistribution to shift the receptive field and FLOPs to the later stage and reduce the computation overhead.
QuantumNAT: Quantum Noise-Aware Training with Noise Injection, Quantization and Normalization
Furthermore, to improve the robustness against noise, we propose noise injection to the training process by inserting quantum error gates to PQC according to realistic noise models of quantum hardware.
Network Augmentation for Tiny Deep Learning
We introduce Network Augmentation (NetAug), a new training method for improving the performance of tiny neural networks.
RoDesigner: Variation-Aware Optimization for Robust Analog Design with Multi-Task RL
Specifically, circuit optimizations under different variations are considered as a set of tasks.
Towards Efficient On-Chip Training of Quantum Neural Networks
The results demonstrate that our on-chip training achieves over 90% and 60% accuracy for 2-class and 4-class image classification tasks.
TSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Device
Secondly, TSM has high efficiency; it achieves a high frame rate of 74fps and 29fps for online video recognition on Jetson Nano and Galaxy Note8.
LocTex: Learning Data-Efficient Visual Representations from Localized Textual Supervision
Computer vision tasks such as object detection and semantic/instance segmentation rely on the painstaking annotation of large training datasets.
QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuits
Extensively evaluated with 12 QML and VQE benchmarks on 14 quantum computers, QuantumNAS significantly outperforms baselines.
NAAS: Neural Accelerator Architecture Search
Data-driven, automatic design space exploration of neural accelerator architecture is desirable for specialization and productivity.
Efficient and Robust LiDAR-Based End-to-End Navigation
On the other hand, increasing the robustness of these systems is also critical; however, even estimating the model's uncertainty is very challenging due to the cost of sampling-based methods.
PatchNet -- Short-range Template Matching for Efficient Video Processing
Object recognition is a fundamental problem in many video processing tasks, accurately locating seen objects at low computation cost paves the way for on-device video recognition.
Anycost GANs for Interactive Image Synthesis and Editing
Generative adversarial networks (GANs) have enabled photorealistic image synthesis and editing.
SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruning
Inspired by the high redundancy of human languages, we propose the novel cascade token pruning to prune away unimportant tokens in the sentence.
IOS: Inter-Operator Scheduler for CNN Acceleration
To accelerate CNN inference, existing deep learning frameworks focus on optimizing intra-operator parallelization.
Hardware-Centric AutoML for Mixed-Precision Quantization
Compared with conventional methods, our framework is fully automated and can specialize the quantization policy for different neural network architectures and hardware architectures.
Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution
Self-driving cars need to understand 3D scenes efficiently and accurately in order to drive safely.
Ranked #1 on
Robust 3D Semantic Segmentation
on SemanticKITTI-C
TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
Furthermore, combined with feature extractor adaptation, TinyTL provides 7. 3-12. 9x memory saving without sacrificing accuracy compared to fine-tuning the full Inception-V3.
MCUNet: Tiny Deep Learning on IoT Devices
Machine learning on tiny IoT devices based on microcontroller units (MCU) is appealing but challenging: the memory of microcontrollers is 2-3 orders of magnitude smaller even than mobile phones.
Differentiable Augmentation for Data-Efficient GAN Training
Furthermore, with only 20% training data, we can match the top performance on CIFAR-10 and CIFAR-100.
Ranked #1 on
Image Generation
on CIFAR-10 (20% data)
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
However, training this quantization-aware accuracy predictor requires collecting a large number of quantized <model, accuracy> pairs, which involves quantization-aware finetuning and thus is highly time-consuming.
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
To enable low-latency inference on resource-constrained hardware platforms, we propose to design Hardware-Aware Transformers (HAT) with neural architecture search.
Ranked #21 on
Machine Translation
on WMT2014 English-French
MicroNet for Efficient Language Modeling
In this paper, we provide the winning solution to the NeurIPS 2019 MicroNet Challenge in the language modeling track.
Once for All: Train One Network and Specialize it for Efficient Deployment
Most of the traditional approaches either manually design or use neural architecture search (NAS) to find a specialized neural network and train it from scratch for each case, which is computationally expensive and unscalable.
GCN-RL Circuit Designer: Transferable Transistor Sizing with Graph Neural Networks and Reinforcement Learning
Our transferable optimization method makes transistor sizing and design porting more effective and efficient.
Lite Transformer with Long-Short Range Attention
For language modeling, Lite Transformer achieves 1. 8 lower perplexity than the transformer at around 500M MACs.
Ranked #36 on
Machine Translation
on WMT2014 English-French
GAN Compression: Efficient Architectures for Interactive Conditional GANs
Directly applying existing compression methods yields poor performance due to the difficulty of GAN training and the differences in generator architectures.
SpArch: Efficient Architecture for Sparse Matrix Multiplication
We then propose a condensed matrix representation that reduces the number of partial matrices by three orders of magnitude and thus reduces DRAM access by 5. 4x.
Hardware Architecture Distributed, Parallel, and Cluster Computing
Park: An Open Platform for Learning-Augmented Computer Systems
We present Park, a platform for researchers to experiment with Reinforcement Learning (RL) for computer systems.
Training Kinetics in 15 Minutes: Large-scale Distributed Training on Videos
With such hardware-aware model design, we are able to scale up the training on Summit supercomputer and reduce the training time on Kinetics dataset from 49 hours 55 minutes to 14 minutes 13 seconds, achieving a top-1 accuracy of 74. 0%, which is 1. 6x and 2. 9x faster than previous 3D video models with higher accuracy.
Distributed Training Across the World
Traditional synchronous distributed training is performed inside a cluster, since it requires high bandwidth and low latency network (e. g. 25Gb Ethernet or Infini-band).
Once-for-All: Train One Network and Specialize it for Efficient Deployment
On diverse edge devices, OFA consistently outperforms state-of-the-art (SOTA) NAS methods (up to 4. 0% ImageNet top1 accuracy improvement over MobileNetV3, or same accuracy but 1. 5x faster than MobileNetV3, 2. 6x faster than EfficientNet w. r. t measured latency) while reducing many orders of magnitude GPU hours and $CO_2$ emission.
Ranked #76 on
Neural Architecture Search
on ImageNet
Point-Voxel CNN for Efficient 3D Deep Learning
The computation cost and memory footprints of the voxel-based models grow cubically with the input resolution, making it memory-prohibitive to scale up the resolution.
Ranked #1 on
3D Object Detection
on KITTI Pedestrian Hard val
Deep Leakage from Gradients
Exchanging gradients is a widely used method in modern multi-node machine learning system (e. g., distributed training, collaborative learning).
Design Automation for Efficient Deep Learning Computing
Moreover, we shorten the design cycle by 200x than previous work, so that we can afford to design specialized neural network models for different hardware platforms.
Defensive Quantization: When Efficiency Meets Robustness
This paper aims to raise people's awareness about the security of the quantized models, and we designed a novel quantization methodology to jointly optimize the efficiency and robustness of deep learning models.
MLSys: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoption.
Learning to Design Circuits
We propose Learning to Design Circuits (L2DC) to leverage reinforcement learning that learns to efficiently generate new circuits data and to optimize circuits.
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
We address the high memory consumption issue of differentiable NAS and reduce the computational cost (GPU hours and GPU memory) to the same level of regular training while still allowing a large candidate set.
Ranked #6 on
Neural Architecture Search
on CIFAR-10 Image Classification
(using extra training data)
HAQ: Hardware-Aware Automated Quantization with Mixed Precision
Compared with conventional methods, our framework is fully automated and can specialize the quantization policy for different neural network architectures and hardware architectures.
TSM: Temporal Shift Module for Efficient Video Understanding
The explosive growth in video streaming gives rise to challenges on performing video understanding at high accuracy and low computation cost.
Ranked #2 on
3D Action Recognition
on Assembly101
Adaptive Mixture of Low-Rank Factorizations for Compact Neural Modeling
Modern deep neural networks have a large amount of weights, which make them difficult to deploy on computation constrained devices such as mobile phones.
Path-Level Network Transformation for Efficient Architecture Search
We introduce a new function-preserving transformation for efficient neural architecture search.
Fast inference of deep neural networks in FPGAs for particle physics
For our example jet substructure model, we fit well within the available resources of modern FPGAs with a latency on the scale of 100 ns.
Efficient Sparse-Winograd Convolutional Neural Networks
First, we move the ReLU operation into the Winograd domain to increase the sparsity of the transformed activations.
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets.
Deep Gradient Compression Reduce the Communication Bandwidth For distributed Traning
Large-scale distributed training requires significant communication bandwidth for gradient exchange that limits the scalability of multi-node training, and requires expensive high-bandwidth network infrastructure.
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
The situation gets even worse with distributed training on mobile devices (federated learning), which suffers from higher latency, lower throughput, and intermittent poor connections.
Deep Generative Adversarial Networks for Compressed Sensing Automates MRI
A multilayer convolutional neural network is then jointly trained based on diagnostic quality images to discriminate the projection quality.
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
Since memory reference is more than two orders of magnitude more expensive than arithmetic operations, the regularity of sparse structure leads to more efficient hardware design.
Classification of Neurological Gait Disorders Using Multi-task Feature Learning
An advanced machine learning method, multi-task feature learning (MTFL), is used to jointly train classification models of a subject's gait in three classes, post-stroke, PD and healthy gait.
Trained Ternary Quantization
To solve this problem, we propose Trained Ternary Quantization (TTQ), a method that can reduce the precision of weights in neural networks to ternary values.
ESE: Efficient Speech Recognition Engine with Sparse LSTM on FPGA
Evaluated on the LSTM for speech recognition benchmark, ESE is 43x and 3x faster than Core i7 5930k CPU and Pascal Titan X GPU implementations.
DSD: Dense-Sparse-Dense Training for Deep Neural Networks
We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance.
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
(2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car.
Ranked #1 on
Image Classification
on ImageNet-P
Generate Image Descriptions based on Deep RNN and Memory Cells for Images Features
Generating natural language descriptions for images is a challenging task.
EIE: Efficient Inference Engine on Compressed Deep Neural Network
EIE has a processing power of 102GOPS/s working directly on a compressed network, corresponding to 3TOPS/s on an uncompressed network, and processes FC layers of AlexNet at 1. 88x10^4 frames/sec with a power dissipation of only 600mW.
Learning both Weights and Connections for Efficient Neural Network
On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9×, from 61 million to 6. 7 million, without incurring accuracy loss.
Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding
To address this limitation, we introduce "deep compression", a three stage pipeline: pruning, trained quantization and Huffman coding, that work together to reduce the storage requirement of neural networks by 35x to 49x without affecting their accuracy.
Learning both Weights and Connections for Efficient Neural Networks
On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9x, from 61 million to 6. 7 million, without incurring accuracy loss.
Robust Face Recognition using Local Illumination Normalization and Discriminant Feature Point Selection
Face recognition systems must be robust to the variation of various factors such as facial expression, illumination, head pose and aging.
