Search Results for author:
Found 12 papers, 0 papers with code
Learn from Downstream and Be Yourself in Multimodal Large Language Model Fine-Tuning
To balance the trade-off between generalization and specialization, we propose measuring the parameter importance for both pre-trained and fine-tuning distributions, based on frozen pre-trained weight magnitude and accumulated fine-tuning gradient values.
Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspace
In this paper, we first investigate the vulnerabilities of existing model merging methods to backdoor attacks, identifying two critical challenges: backdoor succession and backdoor transfer.
Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering
The centroid of each cluster serves as a representative MSU, enabling the assembly of a merged LoRA with an adjusted rank of $k$.
Improving Group Connectivity for Generalization of Federated Deep Learning
Bridging the gap between LMC and FL, in this paper, we leverage fixed anchor models to empirically and theoretically study the transitivity property of connectivity from two models (LMC) to a group of models (model fusion in FL).
Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Models
Catastrophic forgetting emerges as a critical challenge when fine-tuning multi-modal large language models (MLLMs), where improving performance on unseen tasks often leads to a significant performance drop on the original tasks.
RESMatch: Referring Expression Segmentation in a Semi-Supervised Manner
This pioneering work lays the groundwork for future research in semi-supervised learning for referring expression segmentation.
Understanding Prompt Tuning for V-L Models Through the Lens of Neural Collapse
It is found that NC optimality of text-to-image representations shows a positive correlation with downstream generalizability, which is more severe under class imbalance settings.
Quantitatively Measuring and Contrastively Exploring Heterogeneity for Domain Generalization
With contrastive learning, we propose a learning potential-guided metric for domain heterogeneity by promoting learning variant features.
Generalized Universal Domain Adaptation with Generative Flow Networks
We introduce a new problem in unsupervised domain adaptation, termed as Generalized Universal Domain Adaptation (GUDA), which aims to achieve precise prediction of all target labels including unknown categories.
Universal Domain Adaptation via Compressive Attention Matching
To address this issue, we propose a Universal Attention Matching (UniAM) framework by exploiting the self-attention mechanism in vision transformer to capture the crucial object information.
Ranked #1 on
Universal Domain Adaptation
on Office-Home
Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matching
In the literature, centralized clustered FL algorithms require the assumption of the number of clusters and hence are not effective enough to explore the latent relationships among clients.
Ensemble Federated Adversarial Training with Non-IID data
Despite federated learning endows distributed clients with a cooperative training mode under the premise of protecting data privacy and security, the clients are still vulnerable when encountering adversarial samples due to the lack of robustness.
