Search Results for author:
Found 13 papers, 8 papers with code
DistiLLM: Towards Streamlined Distillation for Large Language Models
Knowledge distillation (KD) is widely used for compressing a teacher model to a smaller student model, reducing its inference cost and memory footprint while preserving model capabilities.
STaR: Distilling Speech Temporal Relation for Lightweight Speech Self-Supervised Learning Models
Albeit great performance of Transformer-based speech selfsupervised learning (SSL) models, their large parameter size and computational cost make them unfavorable to utilize.
DiffBlender: Scalable and Composable Multimodal Text-to-Image Diffusion Models
In this study, we aim to extend the capabilities of diffusion-based text-to-image (T2I) generation models by incorporating diverse modalities beyond textual description, such as sketch, box, color palette, and style embedding, within a single model.
Patch-Mix Contrastive Learning with Audio Spectrogram Transformer on Respiratory Sound Classification
Respiratory sound contains crucial information for the early diagnosis of fatal lung diseases.
Ranked #1 on
Audio Classification
on ICBHI Respiratory Sound Database
(using extra training data)
Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation
Transformer-based speech self-supervised learning (SSL) models, such as HuBERT, show surprising performance in various speech processing tasks.
Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning
Fortunately, the recent self-supervised learning (SSL) is a promising approach to pretrain a model without annotations, serving as an effective initialization for any downstream tasks.
How to Fine-tune Models with Few Samples: Update, Data Augmentation, and Test-time Augmentation
Next, we show that data augmentation cannot guarantee few-shot performance improvement and investigate the effectiveness of data augmentation based on the intensity of augmentation.
ReFine: Re-randomization before Fine-tuning for Cross-domain Few-shot Learning
Cross-domain few-shot learning (CD-FSL), where there are few target samples under extreme differences between source and target domains, has recently attracted huge attention.
Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficulty
This data enables self-supervised pre-training on the target domain, in addition to supervised pre-training on the source domain.
Self-Contrastive Learning
This paper proposes a novel contrastive learning framework, called Self-Contrastive (SelfCon) Learning, that self-contrasts within multiple outputs from the different levels of a multi-exit network.
Self-Contrastive Learning: Single-viewed Supervised Contrastive Framework using Sub-network
To this end, we propose Self-Contrastive (SelfCon) learning, which self-contrasts within multiple outputs from the different levels of a single network.
Task Calibration for Distributional Uncertainty in Few-Shot Classification
As numerous meta-learning algorithms improve performance when solving few-shot classification problems for practical applications, accurate prediction of uncertainty, though challenging, has been considered essential.
MixCo: Mix-up Contrastive Learning for Visual Representation
Contrastive learning has shown remarkable results in recent self-supervised approaches for visual representation.
