Search Results for author:
Found 36 papers, 9 papers with code
FCNCP: A Coupled Nonnegative CANDECOMP/PARAFAC Decomposition Based on Federated Learning
In view of this, this study proposes to study and develop a series of efficient non-negative coupled tensor decomposition algorithm frameworks based on federated learning called FCNCP for the EEG data arranged on different servers.
Tao: Re-Thinking DL-based Microarchitecture Simulation
Microarchitecture simulators are indispensable tools for microarchitecture designers to validate, estimate, and optimize new hardware that meets specific design requirements.
Zero-Space Cost Fault Tolerance for Transformer-based Language Models on ReRAM
Resistive Random Access Memory (ReRAM) has emerged as a promising platform for deep neural networks (DNNs) due to its support for parallel in-situ matrix-vector multiplication.
MALCOM-PSGD: Inexact Proximal Stochastic Gradient Descent for Communication-Efficient Decentralized Machine Learning
Recent research indicates that frequent model communication stands as a major bottleneck to the efficiency of decentralized machine learning (ML), particularly for large-scale and over-parameterized neural networks (NNs).
EmojiLM: Modeling the New Emoji Language
With the rapid development of the internet, online social media welcomes people with different backgrounds through its diverse content.
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies
In the upcoming decade, deep learning may revolutionize the natural sciences, enhancing our capacity to model and predict natural occurrences.
Tango: rethinking quantization for graph neural network training on GPUs
Graph Neural Networks (GNNs) are becoming increasingly popular due to their superior performance in critical graph-related tasks.
Blind Graph Matching Using Graph Signals
Our analysis shows that the blind matching outcome converges to the result obtained with known graph topologies when the signal sampling size is large and the signal noise is small.
Differentially Private Over-the-Air Federated Learning Over MIMO Fading Channels
Consequently, relying solely on communication noise, as done in the multiple-input single-output system, cannot meet high privacy requirements, and a device-side privacy-preserving mechanism is necessary for optimal DP design.
Inference via robust optimal transportation: theory and methods
Fourth, we use $W^{(\lambda)}$ to define minimum distance estimators, we provide their statistical guarantees and we illustrate how to apply the derived concentration inequalities for a data driven selection of $\lambda$.
Motif-based Graph Representation Learning with Application to Chemical Molecules
MCM builds a motif vocabulary in an unsupervised way and deploys a novel motif convolution operation to extract the local structural context of individual nodes, which is then used to learn higher-level node representations via multilayer perceptron and/or message passing in graph neural networks.
A Length Adaptive Algorithm-Hardware Co-design of Transformer on FPGA Through Sparse Attention and Dynamic Pipelining
Particularly, we develop a hardware-friendly sparse attention operator and a length-aware hardware resource scheduling algorithm.
CFLIT: Coexisting Federated Learning and Information Transfer
We propose a coexisting federated learning and information transfer (CFLIT) communication framework, where the FL and IT devices share the wireless spectrum in an OFDM system.
Detecting Gender Bias in Transformer-based Models: A Case Study on BERT
In this paper, we propose a novel gender bias detection method by utilizing attention map for transformer-based models.
Sparse Progressive Distillation: Resolving Overfitting under Pretrain-and-Finetune Paradigm
Conventional wisdom in pruning Transformer-based language models is that pruning reduces the model expressiveness and thus is more likely to underfit rather than overfit.
Dr. Top-k: Delegate-Centric Top-k on GPUs
Recent top-$k$ computation efforts explore the possibility of revising various sorting algorithms to answer top-$k$ queries on GPUs.
Reconfigurable Intelligent Surface Empowered Over-the-Air Federated Edge Learning
Federated edge learning (FEEL) has emerged as a revolutionary paradigm to develop AI services at the edge of 6G wireless networks as it supports collaborative model training at a massive number of mobile devices.
Binary Complex Neural Network Acceleration on FPGA
Deep complex networks (DCN), in contrast, can learn from complex data, but have high computational costs; therefore, they cannot satisfy the instant decision-making requirements of many deployable systems dealing with short observations or short signal bursts.
Angel's Girl for Blind Painters: an Efficient Painting Navigation System Validated by Multimodal Evaluation Approach
This work demonstrates that it is practicable for the blind people to feel the world through the brush in their hands.
Relay-Assisted Cooperative Federated Learning
Then, we analyze the model aggregation error in a single-relay case and show that our relay-assisted scheme achieves a smaller error than the one without relays provided that the relay transmit power and the relay channel gains are sufficiently large.
FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Accelerator
With weights stored in the ReRAM crossbar cells as conductance, when the input vector is applied to word lines, the matrix-vector multiplication results can be generated as the current in bit lines.
SimNet: Accurate and High-Performance Computer Architecture Simulation using Deep Learning
While discrete-event simulators are essential tools for architecture research, design, and development, their practicality is limited by an extremely long time-to-solution for realistic applications under investigation.
BIG-bench Machine Learning
Vocal Bursts Intensity Prediction
TAG: Gradient Attack on Transformer-based Language Models
In this paper, as the first attempt, we formulate the gradient attack problem on the Transformer-based language models and propose a gradient attack algorithm, TAG, to reconstruct the local training data.
Federated Learning
Cryptography and Security
CSIT-Free Model Aggregation for Federated Edge Learning via Reconfigurable Intelligent Surface
We study over-the-air model aggregation in federated edge learning (FEEL) systems, where channel state information at the transmitters (CSIT) is assumed to be unavailable.
Conversational Query Rewriting with Self-supervised Learning
Conversational Query Rewriting (CQR) aims to simplify the multi-turn dialogue modeling into a single-turn problem by explicitly rewriting the conversational query into a self-contained utterance.
Semi-Blind Cascaded Channel Estimation for Reconfigurable Intelligent Surface Aided Massive MIMO
In a RIS-aided MIMO system, the acquisition of channel state information (CSI) is important for achieving passive beamforming gains of the RIS, but is also challenging due to the cascaded property of the transmitter-RIS-receiver channel and the lack of signal processing capability of the passive RIS elements.
Bayesian Inference
Information Theory
Information Theory
Reconfigurable Intelligent Surface Enabled Federated Learning: A Unified Communication-Learning Design Approach
However, due to the heterogeneity of communication capacities among edge devices, over-the-air FL suffers from the straggler issue in which the device with the weakest channel acts as a bottleneck of the model aggregation performance.
C-SAW: A Framework for Graph Sampling and Random Walk on GPUs
In this paper, we propose, to the best of our knowledge, the first GPU-based framework for graph sampling/random walk.
Graph Sampling
Distributed, Parallel, and Cluster Computing
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning
Pre-trained large-scale language models have increasingly demonstrated high accuracy on many natural language processing (NLP) tasks.
SAPAG: A Self-Adaptive Privacy Attack From Gradients
Distributed learning such as federated learning or collaborative learning enables model training on decentralized data from users and only collects local gradients, where data is processed close to its sources for data privacy.
Against Membership Inference Attack: Pruning is All You Need
The large model size, high computational operations, and vulnerability against membership inference attack (MIA) have impeded deep learning or deep neural networks (DNNs) popularity, especially on mobile devices.
EZLDA: Efficient and Scalable LDA on GPUs
LDA is a statistical approach for topic modeling with a wide range of applications.
FTRANS: Energy-Efficient Acceleration of Transformers using FPGA
In natural language processing (NLP), the "Transformer" architecture was proposed as the first transduction model replying entirely on self-attention mechanisms without using sequence-aligned recurrent neural networks (RNNs) or convolution, and it achieved significant improvements for sequence to sequence tasks.
Information Freshness-Aware Task Offloading in Air-Ground Integrated Edge Computing Systems
Using the proposed deep RL scheme, each MU in the system is able to make decisions without a priori statistical knowledge of dynamics.
Reconfigurable-Intelligent-Surface Empowered Wireless Communications: Challenges and Opportunities
Reconfigurable intelligent surfaces (RISs) are regarded as a promising emerging hardware technology to improve the spectrum and energy efficiency of wireless networks by artificially reconfiguring the propagation environment of electromagnetic waves.
Information Theory Signal Processing Information Theory
Construction of all-in-focus images assisted by depth sensing
A graph-based segmentation algorithm is used to segment the depth map from the depth sensor, and the segmented regions are used to guide a focus algorithm to locate in-focus image blocks from among multi-focus source images to construct the reference all-in-focus image.
