Search Results for author:
Found 18 papers, 3 papers with code
Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models
The problem of pre-training data detection for large language models (LLMs) has received growing attention due to its implications in critical issues like copyright violation and test data contamination.
Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents
This process allows local devices to train smaller surrogate models while enabling the training of a larger global model on the server, effectively minimizing resource utilization.
DACBERT: Leveraging Dependency Agreement for Cost-Efficient Bert Pretraining
Building on the cost-efficient pretraining advancements brought about by Crammed BERT, we enhance its performance and interpretability further by introducing a novel pretrained model Dependency Agreement Crammed BERT (DACBERT) and its two-stage pretraining framework - Dependency Agreement Pretraining.
Towards Building the Federated GPT: Federated Instruction Tuning
This repository offers a foundational framework for exploring federated fine-tuning of LLMs using heterogeneous instructions across diverse categories.
Rethinking Normalization Methods in Federated Learning
We also show that layer normalization is a better choice in FL which can mitigate the external covariate shift and improve the performance of the global model.
Join-Chain Network: A Logical Reasoning View of the Multi-head Attention in Transformer
Developing neural architectures that are capable of logical reasoning has become increasingly important for a wide range of applications (e. g., natural language processing).
Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction
Based on this measure, we also design a computation-efficient client sampling strategy, such that the actively selected clients will generate a more class-balanced grouped dataset with theoretical guarantees.
FADE: Enabling Federated Adversarial Training on Heterogeneous Resource-Constrained Edge Devices
However, the high demand for memory capacity and computing power makes large-scale federated adversarial training infeasible on resource-constrained edge devices.
GAN-based Medical Image Small Region Forgery Detection via a Two-Stage Cascade Framework
Our method can classify whether a CT image has been tampered and locate the tampered position.
Towards Fair Federated Learning with Zero-Shot Data Augmentation
Federated learning has emerged as an important distributed learning paradigm, where a server aggregates a global model from many client-trained models while having no access to the client data.
Safe Distributional Reinforcement Learning
Safety in reinforcement learning (RL) is a key property in both training and execution in many domains such as autonomous driving or finance.
Differentiable Logic Machines
As a step in this direction, we propose a novel neural-logic architecture, called differentiable logic machine (DLM), that can solve both inductive logic programming (ILP) and reinforcement learning (RL) problems, where the solution can be interpreted as a first-order logic program.
FLOP: Federated Learning on Medical Datasets using Partial Networks
While different data-driven deep learning models have been developed to mitigate the diagnosis of COVID-19, the data itself is still scarce due to patient privacy concerns.
Cyclical Stochastic Gradient MCMC for Bayesian Deep Learning
The posteriors over neural network weights are high dimensional and multimodal.
Self-Adversarially Learned Bayesian Sampling
Scalable Bayesian sampling is playing an important role in modern machine learning, especially in the fast-developed unsupervised-(deep)-learning models.
Variance Reduction in Stochastic Particle-Optimization Sampling
Stochastic particle-optimization sampling (SPOS) is a recently-developed scalable Bayesian sampling framework that unifies stochastic gradient MCMC (SG-MCMC) and Stein variational gradient descent (SVGD) algorithms based on Wasserstein gradient flows.
Towards More Theoretically-Grounded Particle Optimization Sampling for Deep Learning
With such theoretical guarantees, SPOS can be safely and effectively applied on both Bayesian DL and deep RL tasks.
Stochastic Particle-Optimization Sampling and the Non-Asymptotic Convergence Theory
Particle-optimization-based sampling (POS) is a recently developed effective sampling technique that interactively updates a set of particles.
