Search Results for author:
Found 5 papers, 2 papers with code
MBQ: Modality-Balanced Quantization for Large Vision-Language Models
Therefore, treating tokens from different modalities equally, as in existing PTQ methods, may over-emphasize the insensitive modalities, leading to significant accuracy loss.
A Survey on Efficient Inference for Large Language Models
This paper presents a comprehensive survey of the existing literature on efficient LLM inference.
FlashDecoding++: Faster Large Language Model Inference on GPUs
A single and static dataflow may lead to a 50. 25% performance loss for GEMMs of different shapes in LLM inference.
TorchSparse++: Efficient Training and Inference Framework for Sparse Convolution on GPUs
On top of this, we design the Sparse Autotuner, which extends the design space of existing sparse convolution libraries and searches for the best dataflow configurations for training and inference workloads.
Ada3D : Exploiting the Spatial Redundancy with Adaptive Inference for Efficient 3D Object Detection
One reason for this high resource consumption is the presence of a large number of redundant background points in Lidar point clouds, resulting in spatial redundancy in both 3D voxel and dense BEV map representations.
