Search Results for author:
Found 19 papers, 9 papers with code
DataMix: Efficient Privacy-Preserving Edge-Cloud Inference
Third, our solution is extit{efficient} on the edge since the majority of the workload is delegated to the cloud, and our mixing and de-mixing processes introduce very few extra computations.
VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation
VILA-U is a Unified foundation model that integrates Video, Image, Language understanding and generation.
LongVILA: Scaling Long-Context Visual Language Models for Long Videos
We introduce the long-context Multi-Modal Sequence Parallelism (MM-SP) system that efficiently parallelizes long video training and inference, enabling 2M context length training on 256 GPUs without any gradient checkpointing.
Wolf: Captioning Everything with a World Summarization Framework
Finally, we establish a benchmark for video captioning and introduce a leaderboard, aiming to accelerate advancements in video understanding, captioning, and data alignment.
$VILA^2$: VILA Augmented VILA
In this work, we introduce a novel approach that includes a self-augment step and a specialist-augment step to iteratively improve data quality and model performance.
Ranked #36 on
Visual Question Answering
on MM-Vet
Tiny Machine Learning: Progress and Futures
By squeezing deep learning models into billions of IoT devices and microcontrollers (MCUs), we expand the scope of AI applications and enable ubiquitous intelligence.
PockEngine: Sparse and Efficient Fine-tuning in a Pocket
On-device learning and efficient fine-tuning enable continuous and privacy-preserving customization (e. g., locally fine-tuning large language models on personalized data).
On-Device Training Under 256KB Memory
To reduce the memory footprint, we propose Sparse Update to skip the gradient computation of less important layers and sub-tensors.
Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Deep neural networks (DNNs) have achieved unprecedented success in the field of artificial intelligence (AI), including computer vision, natural language processing and speech recognition.
Delayed Gradient Averaging: Tolerate the Communication Latency for Federated Learning
Federated Learning is an emerging direction in distributed machine learning that en-ables jointly training a model without sharing the data.
IOS: Inter-Operator Scheduler for CNN Acceleration
To accelerate CNN inference, existing deep learning frameworks focus on optimizing intra-operator parallelization.
TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning
Furthermore, combined with feature extractor adaptation, TinyTL provides 7. 3-12. 9x memory saving without sacrificing accuracy compared to fine-tuning the full Inception-V3.
HAT: Hardware-Aware Transformers for Efficient Natural Language Processing
To enable low-latency inference on resource-constrained hardware platforms, we propose to design Hardware-Aware Transformers (HAT) with neural architecture search.
Ranked #21 on
Machine Translation
on WMT2014 English-French
Distributed Training Across the World
Traditional synchronous distributed training is performed inside a cluster, since it requires high bandwidth and low latency network (e. g. 25Gb Ethernet or Infini-band).
Deep Leakage from Gradients
Exchanging gradients is a widely used method in modern multi-node machine learning system (e. g., distributed training, collaborative learning).
Design Automation for Efficient Deep Learning Computing
Moreover, we shorten the design cycle by 200x than previous work, so that we can afford to design specialized neural network models for different hardware platforms.
ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware
We address the high memory consumption issue of differentiable NAS and reduce the computational cost (GPU hours and GPU memory) to the same level of regular training while still allowing a large candidate set.
Ranked #6 on
Neural Architecture Search
on CIFAR-10 Image Classification
(using extra training data)
Sparsely Aggregated Convolutional Networks
We explore a key architectural aspect of deep convolutional neural networks: the pattern of internal skip connections used to aggregate outputs of earlier layers for consumption by deeper layers.
Learning to Forecast Videos of Human Activity with Multi-granularity Models and Adaptive Rendering
An architecture combining a hierarchical temporal model for predicting human poses and encoder-decoder convolutional neural networks for rendering target appearances is proposed.
