Search Results for author:
Found 25 papers, 9 papers with code
MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Training LLMs at this scale brings unprecedented challenges to training efficiency and stability.
Object-Driven One-Shot Fine-tuning of Text-to-Image Diffusion with Prototypical Embedding
Our proposed method aims to address the challenges of generalizability and fidelity in an object-driven way, using only a single input image and the object-specific regions of interest.
LEMON: Lossless model expansion
Our empirical results demonstrate that LEMON reduces computational costs by 56. 7% for Vision Transformers and 33. 2% for BERT when compared to training from scratch.
Baechi: Fast Device Placement of Machine Learning Graphs
While these result in model placements that train fast on data (i. e., low step times), learning-based model-parallelism is time-consuming, taking many hours or days to create a placement plan of operators on devices.
ZenoPS: A Distributed Learning System Integrating Communication Efficiency and Security
Distributed machine learning is primarily motivated by the promise of increased computation power for accelerating training and mitigating privacy concerns.
Learning Shape Priors by Pairwise Comparison for Robust Semantic Segmentation
A cosine similarity based attention module is proposed to fuse the information from both encoders, to utilize both types of prior information encoded by the template-encoder and model the inter-subject similarity for each foreground class.
RECIST-Net: Lesion detection via grouping keypoints on RECIST-based annotation
Universal lesion detection in computed tomography (CT) images is an important yet challenging task due to the large variations in lesion type, size, shape, and appearance.
Compressed Communication for Distributed Training: Adaptive Methods and System
Recently, there has been a growing interest in using gradient compression to reduce the communication overhead of the distributed training.
NAS-Navigator: Visual Steering for Explainable One-Shot Deep Neural Network Synthesis
To deal with the problem, we formulate the task of neural network architecture optimization as a graph space exploration, based on the one-shot architecture search technique.
SMAP: A Joint Dimensionality Reduction Scheme for Secure Multi-Party Visualization
We build a prototype system based on our method, SMAP, to support the organization, computation, and exploration of secure joint embedding.
CSER: Communication-efficient SGD with Error Reset
The scalability of Distributed Stochastic Gradient Descent (SGD) is today limited by communication bottlenecks.
Local AdaAlter: Communication-Efficient Stochastic Gradient Descent with Adaptive Learning Rates
When scaling distributed training, the communication overhead is often the bottleneck.
Zeno++: Robust Fully Asynchronous SGD
We propose Zeno++, a new robust asynchronous Stochastic Gradient Descent~(SGD) procedure which tolerates Byzantine failures of the workers.
SLSGD: Secure and Efficient Distributed On-device Machine Learning
We consider distributed on-device learning with limited communication and security requirements.
Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation
Recently, new defense techniques have been developed to tolerate Byzantine failures for distributed machine learning.
Asynchronous Federated Optimization
Federated learning enables training on a massive number of edge devices.
Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance
We present Zeno, a technique to make distributed machine learning, particularly Stochastic Gradient Descent (SGD), tolerant to an arbitrary number of faulty workers.
Phocas: dimensional Byzantine-resilient stochastic gradient descent
We propose a novel robust aggregation rule for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model.
Generalized Byzantine-tolerant SGD
We propose three new robust aggregation rules for distributed synchronous Stochastic Gradient Descent~(SGD) under a general Byzantine failure model.
Attack RMSE Leaderboard: An Introduction and Case Study
In this manuscript, we briefly introduce several tricks to climb the leaderboards which use RMSE for evaluation without exploiting any training data.
Faster Distributed Synchronous SGD with Weak Synchronization
Distributed training of deep learning is widely conducted with large neural networks and large datasets.
Wishart Mechanism for Differentially Private Principal Components Analysis
We propose a new input perturbation mechanism for publishing a covariance matrix to achieve $(\epsilon, 0)$-differential privacy.
A New Relaxation Approach to Normalized Hypergraph Cut
Normalized graph cut (NGC) has become a popular research topic due to its wide applications in a large variety of areas like machine learning and very large scale integration (VLSI) circuit design.
A Scalable and Extensible Framework for Superposition-Structured Models
In many learning tasks, structural models usually lead to better interpretability and higher generalization performance.
Distributed Power-law Graph Computing: Theoretical and Empirical Analysis
We theoretically prove that DBH can achieve lower communication cost than existing methods and can simultaneously guarantee good workload balance.
