Search Results for author:
Found 12 papers, 5 papers with code
Sandwiched Compression: Repurposing Standard Codecs with Neural Network Wrappers
Through a variety of examples, we apply the sandwich architecture to sources with different numbers of channels, higher resolution, higher dynamic range, and perceptual distortion measures.
Scaling Laws for Downstream Task Performance of Large Language Models
With sufficient alignment, both downstream cross-entropy and BLEU score improve monotonically with more pretraining data.
Adaptive Compression in Federated Learning via Side Information
The high communication cost of sending model updates from the clients to the server is a significant bottleneck for scalable federated learning (FL).
Sandwiched Video Compression: Efficiently Extending the Reach of Standard Codecs with Neural Wrappers
We propose differentiable approximations to key video codec components and demonstrate that, in addition to providing meaningful compression improvements over the standard codec, the neural codes of the sandwich lead to significantly better rate-distortion performance in two important scenarios. When transporting high-resolution video via low-resolution HEVC, the sandwich system obtains 6. 5 dB improvements over standard HEVC.
Sparse Random Networks for Communication-Efficient Federated Learning
At the end of the training, the final model is a sparse network with random weights -- or a subnetwork inside the dense random network.
Lossy Compression of Noisy Data for Private and Data-Efficient Learning
In this sense, the utility of the data for learning is essentially maintained, while reducing storage and privacy leakage by quantifiable amounts.
LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks
We consider the attributes of a point cloud as samples of a vector-valued volumetric function at discrete positions.
Neural 3D Scene Compression via Model Compression
In this work, we take a different approach and compress a functional representation of 3D scenes.
An Information-Theoretic Justification for Model Pruning
We study the neural network (NN) compression problem, viewing the tension between the compression ratio and NN performance through the lens of rate-distortion theory.
Neural Network Compression for Noisy Storage Devices
Compression and efficient storage of neural network (NN) parameters is critical for applications that run on resource-constrained devices.
Noisy Neural Network Compression for Analog Storage Devices
Efficient compression and storage of neural network (NN) parameters is critical for resource-constrained, downstream machine learning applications.
rTop-k: A Statistical Estimation Approach to Distributed SGD
The statistically optimal communication scheme arising from the analysis of this model leads to a new sparsification technique for SGD, which concatenates random-k and top-k, considered separately in the prior literature.
