Search Results for author:
Found 14 papers, 5 papers with code
Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models
We categorize methods based on their optimization focus: computational, memory, energy, financial, and network resources and their applicability across various stages of an LLM's lifecycle, including architecture design, pretraining, finetuning, and system design.
Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction
It utilizes an offline memory to cache historical information (e. g., node embedding) as an affordable approximation of the exact value and achieves high concurrency.
Distributed Graph Neural Network Training with Periodic Stale Representation Synchronization
Traditional sampling-based methods accelerate GNN training by dropping edges and nodes, which impairs the graph integrity and model performance.
LOF: Structure-Aware Line Tracking based on Optical Flow
Lines provide the significantly richer geometric structural information about the environment than points, so lines are widely used in recent Visual Odometry (VO) works.
Asynchronous Federated Learning for Sensor Data with Concept Drift
We propose a novel approach, FedConD, to detect and deal with the concept drift on local devices and minimize the effect on the performance of models in asynchronous FL.
Method Towards CVPR 2021 SimLocMatch Challenge
This report describes Megvii-3D team's approach towards SimLocMatch Challenge @ CVPR 2021 Image Matching Workshop.
Method Towards CVPR 2021 Image Matching Challenge
This report describes Megvii-3D team's approach towards CVPR 2021 Image Matching Workshop.
Towards Quantized Model Parallelism for Graph-Augmented MLPs Based on Gradient-Free ADMM Framework
Theoretical convergence to a (quantized) stationary point of the pdADMM-G algorithm and the pdADMM-G-Q algorithm is provided with a sublinear convergence rate $o(1/k)$, where $k$ is the number of iterations.
pdADMM: parallel deep learning Alternating Direction Method of Multipliers
In this paper, we propose a novel parallel deep learning ADMM framework (pdADMM) to achieve layer parallelism: parameters in each layer of neural networks can be updated independently in parallel.
FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
By bridging the synchronous and asynchronous training through tiering, FedAT minimizes the straggler effect with improved convergence speed and test accuracy.
Tunable Subnetwork Splitting for Model-parallelism of Neural Network Training
In this paper, we analyze the reason and propose to achieve a compelling trade-off between parallelism and accuracy by a reformulation called Tunable Subnetwork Splitting Method (TSSM), which can tune the decomposition granularity of deep neural networks.
TiFL: A Tier-based Federated Learning System
To this end, we propose TiFL, a Tier-based Federated Learning System, which divides clients into tiers based on their training performance and selects clients from the same tier in each training round to mitigate the straggler problem caused by heterogeneity in resource and data quantity.
Federated Multi-task Hierarchical Attention Model for Sensor Analytics
The attention mechanism of the proposed model seeks to extract feature representations from the input and learn a shared representation focused on time dimensions across multiple sensors.
Characterizing Co-located Datacenter Workloads: An Alibaba Case Study
Warehouse-scale cloud datacenters co-locate workloads with different and often complementary characteristics for improved resource utilization.
Distributed, Parallel, and Cluster Computing
