Search Results for author:
Found 60 papers, 17 papers with code
Hello Edge: Keyword Spotting on Microcontrollers
We train various neural network architectures for keyword spotting published in literature to compare their accuracy and memory/compute requirements.
Ranked #13 on
Keyword Spotting
on Google Speech Commands
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning
Motivated by this, we target to build a unified interface for completing many vision-language tasks including image description, visual question answering, and visual grounding, among others.
Ranked #10 on
Visual Question Answering
on BenchLMM
EfficientSAM: Leveraged Masked Image Pretraining for Efficient Segment Anything
On segment anything task such as zero-shot instance segmentation, our EfficientSAMs with SAMI-pretrained lightweight image encoders perform favorably with a significant gain (e. g., ~4 AP on COCO/LVIS) over other fast SAM models.
Ranked #3 on
Zero-Shot Instance Segmentation
on LVIS v1.0 val
CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication.
Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation
Therefore, we propose HRViT, which enhances ViTs to learn semantically-rich and spatially-precise multi-scale representations by integrating high-resolution multi-branch architectures with ViTs.
Ranked #24 on
Semantic Segmentation
on Cityscapes val
AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling
Our discovered model family, AttentiveNAS models, achieves top-1 accuracy from 77. 3% to 80. 7% on ImageNet, and outperforms SOTA models, including BigNAS and Once-for-All networks.
Ranked #21 on
Neural Architecture Search
on ImageNet
AlphaNet: Improved Training of Supernets with Alpha-Divergence
Weight-sharing NAS builds a supernet that assembles all the architectures as its sub-networks and jointly trains the supernet with the sub-networks.
Ranked #12 on
Neural Architecture Search
on ImageNet
Federated Learning with Non-IID Data
Experiments show that accuracy can be increased by 30% for the CIFAR-10 dataset with only 5% globally shared data.
Vision Transformers with Patch Diversification
To alleviate this problem, in this work, we introduce novel loss functions in vision transformer training to explicitly encourage diversity across patch representations for more discriminative feature extraction.
Ranked #19 on
Semantic Segmentation
on Cityscapes val
NASViT: Neural Architecture Search for Efficient Vision Transformers with Gradient Conflict aware Supernet Training
In this work, we observe that the poor performance is due to a gradient conflict issue: the gradients of different sub-networks conflict with that of the supernet more severely in ViTs than CNNs, which leads to early saturation in training and inferior convergence.
Ranked #7 on
Neural Architecture Search
on ImageNet
DepthShrinker: A New Compression Paradigm Towards Boosting Real-Hardware Efficiency of Compact Neural Networks
Efficient deep neural network (DNN) models equipped with compact operators (e. g., depthwise convolutions) have shown great potential in reducing DNNs' theoretical complexity (e. g., the total number of weights/operations) while maintaining a decent model accuracy.
CPT: Efficient Deep Neural Network Training via Cyclic Precision
In this paper, we attempt to explore low-precision training from a new perspective as inspired by recent findings in understanding DNN training: we conjecture that DNNs' precision might have a similar effect as the learning rate during DNN training, and advocate dynamic precision along the training trajectory for further boosting the time/energy efficiency of DNN training.
Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Search
The key idea is to derive a smooth, differentiable approximation to the otherwise non-smooth, non-convex search space.
Fast Point Cloud Generation with Straight Flows
We perform evaluations on multiple 3D tasks and find that our PSF performs comparably to the standard diffusion model, outperforming other efficient 3D point cloud generation methods.
KeepAugment: A Simple Information-Preserving Data Augmentation Approach
Data augmentation (DA) is an essential technique for training state-of-the-art deep learning systems.
Energy-Aware Neural Architecture Optimization with Fast Splitting Steepest Descent
Designing energy-efficient networks is of critical importance for enabling state-of-the-art deep learning in mobile and edge settings where the computation and energy budgets are highly limited.
Revisiting Sample Size Determination in Natural Language Understanding
Knowing exactly how many data points need to be labeled to achieve a certain model performance is a hugely beneficial step towards reducing the overall budgets for annotation.
Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networks
Compared to Stripes, BitFusion provides 2. 6x speedup and 3. 9x energy reduction at 45 nm node when BitFusion area and frequency are set to those of Stripes.
Not All Ops Are Created Equal!
Efficient and compact neural network models are essential for enabling the deployment on mobile and embedded devices.
PrivyNet: A Flexible Framework for Privacy-Preserving Deep Neural Network Training
Massive data exist among user local platforms that usually cannot support deep neural network (DNN) training due to computation and storage resource constraints.
Deep Convolutional Neural Network Inference with Floating-point Weights and Fixed-point Activations
To alleviate these problems to some extent, prior research utilize low precision fixed-point numbers to represent the CNN weights and activations.
Heterogeneous Dataflow Accelerators for Multi-DNN Workloads
The results suggest that HDA is an alternative class of Pareto-optimal accelerators to RDA with strength in energy, which can be a better choice than RDAs depending on the use cases.
Distributed, Parallel, and Cluster Computing
Co-Exploration of Neural Architectures and Heterogeneous ASIC Accelerator Designs Targeting Multiple Tasks
Neural Architecture Search (NAS) has demonstrated its power on various AI accelerating platforms such as Field Programmable Gate Arrays (FPGAs) and Graphic Processing Units (GPUs).
Improving Efficiency in Neural Network Accelerator Using Operands Hamming Distance optimization
Neural network accelerator is a key enabler for the on-device AI inference, for which energy efficiency is an important metric.
NASGEM: Neural Architecture Search via Graph Embedding Method
To preserve graph correlation information in encoding, we propose NASGEM which stands for Neural Architecture Search via Graph Embedding Method.
One Weight Bitwidth to Rule Them All
Weight quantization for deep ConvNets has shown promising results for applications such as image classification and semantic segmentation and is especially important for applications where memory storage is limited.
DNA: Differentiable Network-Accelerator Co-Search
Powerful yet complex deep neural networks (DNNs) have fueled a booming demand for efficient DNN solutions to bring DNN-powered intelligence into numerous applications.
Can Temporal Information Help with Contrastive Self-Supervised Learning?
To this end, we present Temporal-aware Contrastive self-supervised learningTaCo, as a general paradigm to enhance video CSL.
ScaleNAS: One-Shot Learning of Scale-Aware Representations for Visual Recognition
We use ScaleNAS to create high-resolution models for two different tasks, ScaleNet-P for human pose estimation and ScaleNet-S for semantic segmentation.
Ranked #5 on
Multi-Person Pose Estimation
on COCO test-dev
Memory-efficient Speech Recognition on Smart Devices
Recurrent transducer models have emerged as a promising solution for speech recognition on the current and next generation smart devices.
Feature-Align Network with Knowledge Distillation for Efficient Denoising
(2) A novel Feature Matching Loss that allows knowledge distillation from large denoising networks in the form of a perceptual content loss.
Collaborative Training of Acoustic Encoders for Speech Recognition
On-device speech recognition requires training models of different sizes for deploying on devices with various computational budgets.
Noisy Training Improves E2E ASR for the Edge
We apply noisy training to improve both dense and sparse state-of-the-art Emformer models and observe consistent WER reduction.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Contrastive Quant: Quantization Makes Stronger Contrastive Learning
Contrastive learning, which learns visual representations by enforcing feature consistency under different augmented views, has emerged as one of the most effective unsupervised learning methods.
Omni-sparsity DNN: Fast Sparsity Optimization for On-Device Streaming E2E ASR via Supernet
From wearables to powerful smart devices, modern automatic speech recognition (ASR) models run on a variety of edge devices with different computational budgets.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Low-Rank+Sparse Tensor Compression for Neural Networks
Low-rank tensor compression has been proposed as a promising approach to reduce the memory and compute requirements of neural networks for their deployment on edge devices.
On the Pareto Efficiency of Quantized CNN
Weight Quantization for deep convolutional neural networks (CNNs) has shown promising results in compressing and accelerating CNN-powered applications such as semantic segmentation, gesture recognition, and scene understanding.
Streaming parallel transducer beam search with fast-slow cascaded encoders
Streaming ASR with strict latency constraints is required in many speech recognition applications.
LiCo-Net: Linearized Convolution Network for Hardware-efficient Keyword Spotting
This paper proposes a hardware-efficient architecture, Linearized Convolution Network (LiCo-Net) for keyword spotting.
XRBench: An Extended Reality (XR) Machine Learning Benchmark Suite for the Metaverse
We hope that our work will stimulate research and lead to the development of a new generation of ML systems for XR use cases.
DREAM: A Dynamic Scheduler for Dynamic Real-time Multi-model ML Workloads
Such dynamic behaviors introduce new challenges to the system software in an ML system since the overall system load is not completely predictable, unlike traditional ML workloads.
PathFusion: Path-consistent Lidar-Camera Deep Feature Fusion
Fusing 3D LiDAR features with 2D camera features is a promising technique for enhancing the accuracy of 3D detection, thanks to their complementary physical properties.
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
Several post-training quantization methods have been applied to large language models (LLMs), and have been shown to perform well down to 8-bits.
Mixture-of-Supernets: Improving Weight-Sharing Supernet Training with Architecture-Routed Mixture-of-Experts
In addition, the proposed method achieves the SOTA performance in NAS for building fast machine translation models, yielding better latency-BLEU tradeoff compared to HAT, state-of-the-art NAS for MT.
TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models
Results demonstrate that our TODM Supernet either matches or surpasses the performance of manually tuned models by up to a relative of 3% better in word error rate (WER), while efficiently keeping the cost of training many models at a small constant.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+2
Folding Attention: Memory and Power Optimization for On-Device Transformer-based Streaming Speech Recognition
Instead, the bottleneck lies in the linear projection layers of multi-head attention and feedforward networks, constituting a substantial portion of the model size and contributing significantly to computation, memory, and power usage.
Enhance audio generation controllability through representation similarity regularization
This paper presents an innovative approach to enhance control over audio generation by emphasizing the alignment between audio and text representations during model training.
Stack-and-Delay: a new codebook pattern for music generation
In language modeling based music generation, a generated waveform is represented by a sequence of hierarchical token stacks that can be decoded either in an auto-regressive manner or in parallel, depending on the codebook patterns.
FoleyGen: Visually-Guided Audio Generation
A prevalent problem in V2A generation is the misalignment of generated audio with the visible actions in the video.
Exploring Speech Enhancement for Low-resource Speech Synthesis
High-quality and intelligible speech is essential to text-to-speech (TTS) model training, however, obtaining high-quality data for low-resource languages is challenging and expensive.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+3
On The Open Prompt Challenge In Conditional Audio Generation
Text-to-audio generation (TTA) produces audio from a text description, learning from pairs of audio samples and hand-annotated text.
In-Context Prompt Editing For Conditional Audio Generation
We show that the framework enhanced the audio quality across the set of collected user prompts, which were edited with reference to the training captions as exemplars.
SqueezeSAM: User friendly mobile interactive segmentation
Finally, when a user clicks on an object, they typically expect all related pieces of the object to be segmented.
SteinDreamer: Variance Reduction for Text-to-3D Score Distillation via Stein Identity
In this paper, we reveal that the gradient estimation in score distillation is inherent to high variance.
Taming Mode Collapse in Score Distillation for Text-to-3D Generation
In this paper, we reveal that the existing score distillation-based text-to-3D generation frameworks degenerate to maximal likelihood seeking on each view independently and thus suffer from the mode collapse problem, manifesting as the Janus artifact in practice.
MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction
MVDiffusion++ achieves superior flexibility and scalability with two surprisingly simple ideas: 1) A ``pose-free architecture'' where standard self-attention among 2D latent features learns 3D consistency across an arbitrary number of conditional and generation views without explicitly using camera pose information; and 2) A ``view dropout strategy'' that discards a substantial number of output views during training, which reduces the training-time memory footprint and enables dense and high-resolution view synthesis at test time.
Not All Weights Are Created Equal: Enhancing Energy Efficiency in On-Device Streaming Speech Recognition
This study delves into how weight parameters in speech recognition models influence the overall power consumption of these models.
MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases
The resultant models, denoted as MobileLLM-LS, demonstrate a further accuracy enhancement of 0. 7%/0. 8% than MobileLLM 125M/350M.
CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians
The field of 3D reconstruction from images has rapidly evolved in the past few years, first with the introduction of Neural Radiance Field (NeRF) and more recently with 3D Gaussian Splatting (3DGS).
