Search Results for author:
Found 17 papers, 7 papers with code
FedFN: Feature Normalization for Alleviating Data Heterogeneity Problem in Federated Learning
Additionally, we observe that as data heterogeneity increases, the gap between higher feature norms for observed classes, obtained from local models, and feature norms of unobserved classes widens, in contrast to the behavior of classifier weight norms.
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval
Given a query image, we harness the power of CLIP's cross-modal representations to retrieve relevant textual information from an external image-text pair dataset.
FedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning
FedSOL is designed to identify gradients of local objectives that are inherently orthogonal to directions affecting the proximal objective.
Synergy with Translation Artifacts for Training and Inference in Multilingual Tasks
Translation has played a crucial role in improving the performance on multilingual tasks: (1) to generate the target language data from the source language data for training and (2) to generate the source language data from the target language data for inference.
Demystifying the Base and Novel Performances for Few-shot Class-incremental Learning
Few-shot class-incremental learning (FSCIL) has addressed challenging real-world scenarios where unseen novel classes continually arrive with few samples.
How to Fine-tune Models with Few Samples: Update, Data Augmentation, and Test-time Augmentation
Next, we show that data augmentation cannot guarantee few-shot performance improvement and investigate the effectiveness of data augmentation based on the intensity of augmentation.
ReFine: Re-randomization before Fine-tuning for Cross-domain Few-shot Learning
Cross-domain few-shot learning (CD-FSL), where there are few target samples under extreme differences between source and target domains, has recently attracted huge attention.
Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty
This data enables self-supervised pre-training on the target domain, in addition to supervised pre-training on the source domain.
FedBABU: Toward Enhanced Representation for Federated Image Classification
Based on this observation, we propose a novel federated learning algorithm, coined FedBABU, which only updates the body of the model during federated training (i. e., the head is randomly initialized and never updated), and the head is fine-tuned for personalization during the evaluation process.
FedBABU: Towards Enhanced Representation for Federated Image Classification
Based on this observation, we propose a novel federated learning algorithm, coined FedBABU, which only updates the body of the model during federated training (i. e., the head is randomly initialized and never updated), and the head is fine-tuned for personalization during the evaluation process.
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation
From this observation, we consider an intuitive KD loss function, the mean squared error (MSE) between the logit vectors, so that the student model can directly learn the logit of the teacher model.
Understanding Knowledge Distillation
To verify this conjecture, we test an extreme logit learning model, where the KD is implemented with Mean Squared Error (MSE) between the student's logit and the teacher's logit.
Accurate and Fast Federated Learning via IID and Communication-Aware Grouping
Federated learning has emerged as a new paradigm of collaborative machine learning; however, it has also faced several challenges such as non-independent and identically distributed(IID) data and high communication cost.
TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture
Federated learning has emerged as a new paradigm of collaborative machine learning; however, many prior studies have used global aggregation along a star topology without much consideration of the communication scalability or the diurnal property relied on clients' local time variety.
BOIL: Towards Representation Change for Few-shot Learning
It has recently been hypothesized that representation reuse, which makes little change in efficient representations, is the dominant factor in the performance of the meta-initialized model through MAML in contrast to representation change, which causes a significant change in representations.
SIPA: A Simple Framework for Efficient Networks
With the success of deep learning in various fields and the advent of numerous Internet of Things (IoT) devices, it is essential to lighten models suitable for low-power devices.
Spectrogram-channels u-net: a source separation model viewing each channel as the spectrogram of each source
The proposed model can be used for not only singing voice separation but also multi-instrument separation by changing only the number of output channels.
