Search Results for author:
Found 15 papers, 8 papers with code
Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models
We categorize methods based on their optimization focus: computational, memory, energy, financial, and network resources and their applicability across various stages of an LLM's lifecycle, including architecture design, pretraining, finetuning, and system design.
Staleness-Alleviated Distributed GNN Training via Online Dynamic-Embedding Prediction
It utilizes an offline memory to cache historical information (e. g., node embedding) as an affordable approximation of the exact value and achieves high concurrency.
Distributed Graph Neural Network Training with Periodic Stale Representation Synchronization
Traditional sampling-based methods accelerate GNN training by dropping edges and nodes, which impairs the graph integrity and model performance.
Towards cost-effective and resource-aware aggregation at Edge for Federated Learning
Federated Learning (FL) is a machine learning approach that addresses privacy and data transfer costs by computing data at the source.
Asynchronous Federated Learning for Sensor Data with Concept Drift
We propose a novel approach, FedConD, to detect and deal with the concept drift on local devices and minimize the effect on the performance of models in asynchronous FL.
Towards Quantized Model Parallelism for Graph-Augmented MLPs Based on Gradient-Free ADMM Framework
Theoretical convergence to a (quantized) stationary point of the pdADMM-G algorithm and the pdADMM-G-Q algorithm is provided with a sublinear convergence rate $o(1/k)$, where $k$ is the number of iterations.
Representation Iterative Fusion based on Heterogeneous Graph Neural Network for Joint Entity and Relation Extraction
Joint entity and relation extraction is an essential task in information extraction, which aims to extract all relational triples from unstructured text.
Ranked #2 on
Relation Extraction
on SemEval-2010 Task-8
pdADMM: parallel deep learning Alternating Direction Method of Multipliers
In this paper, we propose a novel parallel deep learning ADMM framework (pdADMM) to achieve layer parallelism: parameters in each layer of neural networks can be updated independently in parallel.
Wukong: A Scalable and Locality-Enhanced Framework for Serverless Parallel Computing
Serverless computing is increasingly being used for parallel computing, which have traditionally been implemented as stateful applications.
Distributed, Parallel, and Cluster Computing
FedAT: A High-Performance and Communication-Efficient Federated Learning System with Asynchronous Tiers
By bridging the synchronous and asynchronous training through tiering, FedAT minimizes the straggler effect with improved convergence speed and test accuracy.
Tunable Subnetwork Splitting for Model-parallelism of Neural Network Training
In this paper, we analyze the reason and propose to achieve a compelling trade-off between parallelism and accuracy by a reformulation called Tunable Subnetwork Splitting Method (TSSM), which can tune the decomposition granularity of deep neural networks.
TiFL: A Tier-based Federated Learning System
To this end, we propose TiFL, a Tier-based Federated Learning System, which divides clients into tiers based on their training performance and selects clients from the same tier in each training round to mitigate the straggler problem caused by heterogeneity in resource and data quantity.
In Search of a Fast and Efficient Serverless DAG Engine
The auto-scaling property of serverless computing platforms accommodates short tasks and bursty workloads, while the pay-per-use billing model of serverless computing providers keeps the cost of short tasks low.
Distributed, Parallel, and Cluster Computing
Super-Resolved Image Perceptual Quality Improvement via Multi-Feature Discriminators
However, the GAN-based SR methods only use image discriminator to distinguish SR images and high-resolution (HR) images.
Characterizing Co-located Datacenter Workloads: An Alibaba Case Study
Warehouse-scale cloud datacenters co-locate workloads with different and often complementary characteristics for improved resource utilization.
Distributed, Parallel, and Cluster Computing
