Search Results for author:
Found 9 papers, 1 papers with code
Higher Layers Need More LoRA Experts
In this paper, we introduce a novel parameter-efficient MoE method, \textit{\textbf{M}oE-L\textbf{o}RA with \textbf{L}ayer-wise Expert \textbf{A}llocation (MoLA)} for Transformer-based models, where each model layer has the flexibility to employ a varying number of LoRA experts.
Evaluation and Enhancement of Semantic Grounding in Large Vision-Language Models
Large Vision-Language Models (LVLMs) offer remarkable benefits for a variety of vision-language tasks.
Tackling Vision Language Tasks Through Learning Inner Monologues
Visual language tasks require AI models to comprehend and reason with both visual and textual content.
Air Pollution Hotspot Detection and Source Feature Analysis using Cross-domain Urban Data
In this work, we explore the use of mobile sensing data (i. e., air quality sensors installed on vehicles) to detect pollution hotspots.
Squeezing SGD Parallelization Performance in Distributed Training Using Delayed Averaging
Mini-batch Stochastic Gradient Descent (SGD) requires workers to halt forward/backward propagations, to wait for gradients synchronized among all workers before the next batch of tasks.
DaSGD: Squeezing SGD Parallelization Performance in Distributed Training Using Delayed Averaging
The state-of-the-art deep learning algorithms rely on distributed training systems to tackle the increasing sizes of models and training data sets.
Learn Electronic Health Records by Fully Decentralized Federated Learning
Federated learning opens a number of research opportunities due to its high communication efficiency in distributed training problems within a star network.
Applying High-Resolution Visible Imagery to Satellite Melt Pond Fraction Retrieval: A Neural Network Approach
It is important to retrieve accurate melt pond fraction (MPF) from satellite data for Arctic research.
