Search Results for author:
Found 26 papers, 11 papers with code
Keyframe-oriented Vision Token Pruning: Enhancing Efficiency of Large Vision Language Models on Long-Form Video Processing
In this work, we propose KVTP (Keyframe-oriented Vision Token Pruning), a novel framework that overcomes the drawbacks of token pruning and keyframe selection.
Proactive Privacy Amnesia for Large Language Models: Safeguarding PII with Negligible Impact on Model Utility
With the rise of large language models (LLMs), increasing research has recognized their risk of leaking personally identifiable information (PII) under malicious attacks.
EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluation
This allows us to comprehensively evaluate the effectiveness of image-text alignment metrics for T2I models.
SpeechPrune: Context-aware Token Pruning for Speech Information Retrieval
We introduce Speech Information Retrieval (SIR), a new long-context task for Speech Large Language Models (Speech LLMs), and present SPIRAL, a 1, 012-sample benchmark testing models' ability to extract critical details from approximately 90-second spoken inputs.
LoBAM: LoRA-Based Backdoor Attack on Model Merging
In practice where resources are limited and the attacker can only employ techniques like Low-Rank Adaptation (LoRA) to produce the malicious model, it remains unclear whether the attack can still work and pose threats.
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
The rapid development of large language models (LLMs) has significantly transformed the field of artificial intelligence, demonstrating remarkable capabilities in natural language processing and moving towards multi-modal functionality.
Latent Feature and Attention Dual Erasure Attack against Multi-View Diffusion Models for 3D Assets Protection
This paper is the first to address the intellectual property infringement issue arising from MVDMs.
Knowledge Graph Tuning: Real-time Large Language Model Personalization based on Human Feedback
To enhance user experience, real-time model personalization is essential, allowing LLMs to adapt user-specific knowledge based on user feedback during human-LLM interactions.
Min-K%++: Improved Baseline for Detecting Pre-Training Data from Large Language Models
The problem of pre-training data detection for large language models (LLMs) has received growing attention due to its implications in critical issues like copyright violation and test data contamination.
Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latents
This process allows local devices to train smaller surrogate models while enabling the training of a larger global model on the server, effectively minimizing resource utilization.
SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Models
Specifically, SiDA-MoE attains a remarkable speedup in MoE inference with up to $3. 93\times$ throughput increasing, up to $72\%$ latency reduction, and up to $80\%$ GPU memory saving with down to $1\%$ performance drop.
FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models
However, applying FL to finetune PLMs is hampered by challenges, including restricted model parameter access, high computational requirements, and communication overheads.
FedLALR: Client-Specific Adaptive Learning Rates Achieve Linear Speedup for Non-IID Data
Federated learning is an emerging distributed machine learning method, enables a large number of clients to train a model without exchanging their local data.
PrivaScissors: Enhance the Privacy of Collaborative Inference through the Lens of Mutual Information
Edge-cloud collaborative inference empowers resource-limited IoT devices to support deep learning applications without disclosing their raw data to the cloud server, thus preserving privacy.
Communication-Efficient Vertical Federated Learning with Limited Overlapping Samples
We propose a practical vertical federated learning (VFL) framework called \textbf{one-shot VFL} that can solve the communication bottleneck and the problem of limited overlapping samples simultaneously based on semi-supervised learning.
Robust and IP-Protecting Vertical Federated Learning against Unexpected Quitting of Parties
In this paper, we propose \textbf{Party-wise Dropout} to improve the VFL model's robustness against the unexpected exit of passive parties and a defense method called \textbf{DIMIP} to protect the active party's IP in the deployment phase.
Boundary-to-Solution Mapping for Groundwater Flows in a Toth Basin
In this paper, the authors propose a new approach to solving the groundwater flow equation in the Toth basin of arbitrary top and bottom topographies using deep learning.
AdaSAM: Boosting Sharpness-Aware Minimization with Adaptive Learning Rate and Momentum for Training Deep Neural Networks
Integrating SAM with adaptive learning rate and momentum acceleration, dubbed AdaSAM, has already been explored empirically to train large-scale deep neural networks without theoretical guarantee due to the triple difficulties in analyzing the coupled perturbation step, adaptive learning rate and momentum step.
More Generalized and Personalized Unsupervised Representation Learning In A Distributed System
In this work, we propose a novel method, FedStyle, to learn a more generalized global model by infusing local style information with local content information for contrastive learning, and to learn more personalized local models by inducing local style information for downstream tasks.
Rethinking Normalization Methods in Federated Learning
We also show that layer normalization is a better choice in FL which can mitigate the external covariate shift and improve the performance of the global model.
Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction
Based on this measure, we also design a computation-efficient client sampling strategy, such that the actively selected clients will generate a more class-balanced grouped dataset with theoretical guarantees.
FL-WBC: Enhancing Robustness against Model Poisoning Attacks in Federated Learning from a Client Perspective
Furthermore, we derive a certified robustness guarantee against model poisoning attacks and a convergence guarantee to FedAvg after applying our FL-WBC.
Soteria: Provable Defense Against Privacy Leakage in Federated Learning From Representation Perspective
The key idea of our defense is learning to perturb data representation such that the quality of the reconstructed data is severely degraded, while FL performance is maintained.
FedCor: Correlation-Based Active Client Selection Strategy for Heterogeneous Federated Learning
In this work, we propose FedCor -- an FL framework built on a correlation-based client selection strategy, to boost the convergence rate of FL.
Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective
In this work, we show our key observation that the data representation leakage from gradients is the essential cause of privacy leakage in FL.
LotteryFL: Personalized and Communication-Efficient Federated Learning with Lottery Ticket Hypothesis on Non-IID Datasets
Rather than learning a shared global model in classic federated learning, each client learns a personalized model via LotteryFL; the communication cost can be significantly reduced due to the compact size of lottery networks.
