Search Results for author:
Found 5 papers, 1 papers with code
CSR:Achieving 1 Bit Key-Value Cache via Sparse Representation
The emergence of long-context text applications utilizing large language models (LLMs) has presented significant scalability challenges, particularly in memory footprint.
GreenFlow: A Computation Allocation Framework for Building Environmentally Sound Recommendation System
of a cascade RS, when a user triggers a request, we define two actions that determine the computation: (1) the trained instances of models with different computational complexity; and (2) the number of items to be inferred in the stage.
Fast Chain-of-Thought: A Glance of Future from Parallel Decoding Leads to Answers Faster
In this work, we propose FastCoT, a model-agnostic framework based on parallel decoding without any further training of an auxiliary model or modification to the LLM itself.
On the Opportunities of Green Computing: A Survey
To tackle the challenges of computing resources and environmental impact of AI, Green Computing has become a hot research topic.
Dynamic DNN Decomposition for Lossless Synergistic Inference
The algorithm can partially adjust the partitions at run time according to processing time and network conditions.
