Search Results for author:
Found 9 papers, 4 papers with code
CommVQ: Commutative Vector Quantization for KV Cache Compression
We first introduce additive quantization with a lightweight encoder and codebook to compress the KV cache, which can be decoded via simple matrix multiplication.
The Super Weight in Large Language Models
For weight quantization, we similarly find that by preserving the super weight and clipping other weight outliers, round-to-nearest quantization can scale to much larger block sizes than previously considered.
Apple Intelligence Foundation Language Models
We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute.
Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers
SPAN operates by regularizing attention masks from separate transformer heads to follow various priors over semantic regions.
HyperionSolarNet: Solar Panel Detection from Aerial Images
The energy sector is the single largest contributor to climate change and many efforts are focused on reducing dependence on carbon-emitting power plants and moving to renewable energy sources, such as solar power.
Multi-source Few-shot Domain Adaptation
Multi-source Domain Adaptation (MDA) aims to transfer predictive models from multiple, fully-labeled source domains to an unlabeled target domain.
Self-supervised Contrastive Learning for Irrigation Detection in Satellite Imagery
Climate change has caused reductions in river runoffs and aquifer recharge resulting in an increasingly unsustainable crop water demand from reduced freshwater availability.
Multi-source Domain Adaptation in the Deep Learning Era: A Systematic Survey
Therefore, transferring the learned knowledge from a separate, labeled source domain to an unlabeled or sparsely labeled target domain becomes an appealing alternative.
Scaling the Indian Buffet Process via Submodular Maximization
Inference for latent feature models is inherently difficult as the inference space grows exponentially with the size of the input data and number of latent features.
