Search Results for author:
Found 10 papers, 3 papers with code
Wisdom of Committee: Distilling from Foundation Model to Specialized Application Model
Further, to accommodate the dissimilarity among the teachers in the committee, we introduce DiverseDistill, which allows the student to understand the expertise of each teacher and extract task knowledge.
Heterogeneous federated collaborative filtering using FAIR: Federated Averaging in Random Subspaces
To include data from all devices in federated learning, we must enable collective training of embedding tables on devices with heterogeneous memory capacities.
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time
We show that contextual sparsity exists, that it can be accurately predicted, and that we can exploit it to speed up LLM inference in wall-clock time without compromising LLM's quality or in-context learning ability.
Learning Multimodal Data Augmentation in Feature Space
The ability to jointly learn from multiple modalities, such as text, audio, and visual data, is a defining feature of intelligent systems.
SAR-Net: Shape Alignment and Recovery Network for Category-level 6D Object Pose and Size Estimation
The concatenation of the observed point cloud and symmetric one reconstructs a coarse object shape, thus facilitating object center (3D translation) and 3D size estimation.
Efficient Inference via Universal LSH Kernel
Large machine learning models achieve unprecedented performance on various tasks and have evolved as the go-to technique.
MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training
Recent advances by practitioners in the deep learning community have breathed new life into Locality Sensitive Hashing (LSH), using it to reduce memory and time bottlenecks in neural network (NN) training.
Neighbor Oblivious Learning (NObLe) for Device Localization and Tracking
In this paper, we argue that the state-of-the-art-systems are significantly worse in terms of accuracy because they are incapable of utilizing these essential structural information.
Conditional Automated Channel Pruning for Deep Neural Networks
To address this issue, given a target compression rate for the whole model, one can search for the optimal compression rate for each layer.
Climbing the WOL: Training for Cheaper Inference
Efficient inference for wide output layers (WOLs) is an essential yet challenging task in large scale machine learning.
