Search Results for author:
Found 20 papers, 11 papers with code
Towards Fast Inference: Exploring and Improving Blockwise Parallel Drafts
Despite the remarkable strides made by autoregressive language models, their potential is often hampered by the slow inference speeds inherent in sequential token generation.
Semantic Layering in Room Segmentation via LLMs
In this paper, we introduce Semantic Layering in Room Segmentation via LLMs (SeLRoS), an advanced method for semantic room segmentation by integrating Large Language Models (LLMs) with traditional 2D map-based segmentation.
Non-linear Fusion in Federated Learning: A Hypernetwork Approach to Federated Domain Generalization
We propose an innovative federated algorithm, termed hFedF for hypernetwork-based Federated Fusion, designed to bridge the performance gap between generalization and personalization, capable of addressing various degrees of domain shift.
Revisiting Early-Learning Regularization When Federated Learning Meets Noisy Labels
In the evolving landscape of federated learning (FL), addressing label noise presents unique challenges due to the decentralized and diverse nature of data collection across clients.
Leveraging Normalization Layer in Adapters With Progressive Learning and Adaptive Distillation for Cross-Domain Few-Shot Learning
Second, to address the pitfalls of noisy statistics, we deploy two strategies: a progressive training of the two adapters and an adaptive distillation technique derived from features determined by the model solely with the adapter devoid of a normalization layer.
Instructive Decoding: Instruction-Tuned Large Language Models are Self-Refiner from Noisy Instructions
Notably, utilizing 'opposite' as the noisy instruction in ID, which exhibits the maximum divergence from the original instruction, consistently produces the most significant performance gains across multiple models and tasks.
Navigating Data Heterogeneity in Federated Learning A Semi-Supervised Federated Object Detection
Federated Learning (FL) has emerged as a potent framework for training models across distributed data sources while maintaining data privacy.
Region-Conditioned Orthogonal 3D U-Net for Weather4Cast Competition
The Weather4Cast competition (hosted by NeurIPS 2022) required competitors to predict super-resolution rain movies in various regions of Europe when low-resolution satellite contexts covering wider regions are given.
Benchmark Dataset for Precipitation Forecasting by Post-Processing the Numerical Weather Prediction
Historically, this challenge has been tackled using numerical weather prediction (NWP) models, grounded on physics-based simulations.
Revisiting Architecture-aware Knowledge Distillation: Smaller Models and Faster Search
Knowledge Distillation (KD) has recently emerged as a popular method for compressing neural networks.
Revisiting Orthogonality Regularization: A Study for Convolutional Neural Networks in Image Classification
Recent research in deep Convolutional Neural Networks(CNN) faces the challenges of vanishing/exploding gradient issues, training instability, and feature redundancy.
Supernet Training for Federated Image Classification under System Heterogeneity
The approach is inspired by observing that averaging parameters during model aggregation for FL is similar to weight-sharing in supernet training.
A Survey of Supernet Optimization and its Applications: Spatial and Temporal Optimization for Neural Architecture Search
The survey analyses supernet optimization methods based on their approaches to spatial and temporal optimization.
Mold into a Graph: Efficient Bayesian Optimization over Mixed-Spaces
Real-world optimization problems are generally not just black-box problems, but also involve mixed types of inputs in which discrete and continuous variables coexist.
Comparing Kullback-Leibler Divergence and Mean Squared Error Loss in Knowledge Distillation
From this observation, we consider an intuitive KD loss function, the mean squared error (MSE) between the logit vectors, so that the student model can directly learn the logit of the teacher model.
FINE Samples for Learning with Noisy Labels
Our framework, coined as filtering noisy instances via their eigenvectors (FINE), provides a robust detector with derivative-free simple methods having theoretical guarantees.
Ranked #2 on
Image Classification
on WebVision
Understanding Knowledge Distillation
To verify this conjecture, we test an extreme logit learning model, where the KD is implemented with Mean Squared Error (MSE) between the student's logit and the teacher's logit.
Adaptive Local Bayesian Optimization Over Multiple Discrete Variables
In the machine learning algorithms, the choice of the hyperparameter is often an art more than a science, requiring labor-intensive search with expert experience.
Accurate and Fast Federated Learning via Combinatorial Multi-Armed Bandits
Federated learning has emerged as an innovative paradigm of collaborative machine learning.
Efficient Model for Image Classification With Regularization Tricks
Our final score is 0. 0054, which represents 370x improvements over the baseline for the CIFAR100 dataset.
