Search Results for author:
Found 25 papers, 12 papers with code
ELMS: Elasticized Large Language Models On Mobile Devices
To address this issue, we introduce ELMS, an on-device LLM service designed to provide elasticity in both the model and prompt dimensions of an LLMaaS.
Empowering 1000 tokens/second on-device LLM prefilling with mllm-NPU
On-device large language models (LLMs) are catalyzing novel mobile applications such as UI task automation and personalized email auto-reply, without giving away users' private data.
RAGCache: Efficient Knowledge Caching for Retrieval-Augmented Generation
We implement RAGCache and evaluate it on vLLM, a state-of-the-art LLM inference system and Faiss, a state-of-the-art vector database.
LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism
The context window of large language models (LLMs) is rapidly increasing, leading to a huge variance in resource usage between different requests as well as between different phases of the same request.
Anatomizing Deep Learning Inference in Web Browsers
The gap on mobile CPU and mobile GPU is 15. 8 times and 7. 8 times, respectively.
A Survey of Resource-efficient LLM and Multimodal Foundation Models
Large foundation models, including large language models (LLMs), vision transformers (ViTs), diffusion, and LLM-based multimodal models, are revolutionizing the entire machine learning lifecycle, from training to deployment.
LLMCad: Fast and Scalable On-device Large Language Model Inference
Currently, the execution of these generative tasks heavily depends on Large Language Models (LLMs).
Bias Behind the Wheel: Fairness Analysis of Autonomous Driving Systems
This paper analyzes fairness in automated pedestrian detection, a crucial but under-explored issue in autonomous driving systems.
Fast Distributed Inference Serving for Large Language Models
Large language models (LLMs) power a new generation of interactive AI applications exemplified by ChatGPT.
Benchmarking of DL Libraries and Models on Mobile Devices
Deploying deep learning (DL) on mobile devices has been a notable trend in recent years.
An Empirical Study on Deployment Faults of Deep Learning Based Mobile Applications
In contrast, faults related to the deployment of DL models on mobile devices (named as deployment faults of mobile DL apps) have not been well studied.
Hierarchical Federated Learning through LAN-WAN Orchestration
Our proposed FL can accelerate the learning process and reduce the monetary cost with frequent local aggregation in the same LAN and infrequent global aggregation on a cloud across WAN.
Exploring the Generalizability of Spatio-Temporal Traffic Prediction: Meta-Modeling and an Analytic Framework
The Spatio-Temporal Traffic Prediction (STTP) problem is a classical problem with plenty of prior research efforts that benefit from traditional statistical learning and recent deep learning approaches.
Characterizing Impacts of Heterogeneity in Federated Learning upon Large-Scale Smartphone Data
Based on the data and the platform, we conduct extensive experiments to compare the performance of state-of-the-art FL algorithms under heterogeneity-aware and heterogeneity-unaware settings.
Understanding Challenges in Deploying Deep Learning Based Software: An Empirical Study
Deep learning (DL) becomes increasingly pervasive, being used in a wide range of software applications.
Software Engineering
Federated Neural Architecture Search
To preserve user privacy while enabling mobile intelligence, techniques have been proposed to train deep neural networks on decentralized data.
Approximate Query Service on Autonomous IoT Cameras
Elf is a runtime for an energy-constrained camera to continuously summarize video scenes as approximate object counts.
Databases
SEntiMoji: An Emoji-Powered Learning Approach for Sentiment Analysis in Software Engineering
However, commonly used out-of-the-box sentiment analysis tools cannot obtain reliable results on SE tasks and the misunderstanding of technical jargon is demonstrated to be the main reason.
Moving Deep Learning into Web Browser: How Far Can We Go?
Recently, several JavaScript-based deep learning frameworks have emerged, making it possible to perform deep learning tasks directly in browsers.
Software Engineering
A First Look at Emoji Usage on GitHub: An Empirical Study
We find that emojis are used by a considerable proportion of GitHub users.
Computers and Society Software Engineering
A First Look at Deep Learning Apps on Smartphones
We are in the dawn of deep learning explosion for smartphones.
Emoji-Powered Representation Learning for Cross-Lingual Sentiment Classification
To tackle this problem, cross-lingual sentiment classification approaches aim to transfer knowledge learned from one language that has abundant labeled examples (i. e., the source language, usually English) to another language with fewer labels (i. e., the target language).
Listening to Chaotic Whispers: A Deep Learning Framework for News-oriented Stock Trend Prediction
Stock trend prediction plays a critical role in seeking maximized profit from stock investment.
Ranked #16 on
Stock Market Prediction
on Astock
DeepCache: Principled Cache for Mobile Deep Vision
We present DeepCache, a principled cache design for deep learning inference in continuous mobile vision.
DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning
Due to their on-body and ubiquitous nature, wearables can generate a wide range of unique sensor data creating countless opportunities for deep learning tasks.
