Search Results for author:
Found 15 papers, 8 papers with code
Exploring connections of spectral analysis and transfer learning in medical imaging
In this paper, we use spectral analysis to investigate transfer learning and study model sensitivity to frequency shortcuts in medical imaging.
Source Matters: Source Dataset Impact on Model Robustness in Medical Imaging
Transfer learning has become an essential part of medical imaging classification algorithms, often leveraging ImageNet weights.
Cross-domain Denoising for Low-dose Multi-frame Spiral Computed Tomography
Computed tomography (CT) has been used worldwide as a non-invasive test to assist in diagnosis.
Coordinating Distributed Example Orders for Provably Accelerated Training
Recent research on online Gradient Balancing (GraB) has revealed that there exist permutation-based example orderings for SGD that are guaranteed to outperform random reshuffling (RR).
STEP: Learning N:M Structured Sparsity Masks from Scratch with Precondition
Recent innovations on hardware (e. g. Nvidia A100) have motivated learning N:M structured sparsity masks from scratch for fast model inference.
RZSR: Reference-based Zero-Shot Super-Resolution with Depth Guided Self-Exemplars
To advance ZSSR, we obtain reference image patches with rich textures and high-frequency details which are also extracted only from the input image using cross-scale matching.
GraB: Finding Provably Better Data Permutations than Random Reshuffling
To reduce the memory overhead, we leverage discrepancy minimization theory to propose an online Gradient Balancing algorithm (GraB) that enjoys the same rate as herding, while reducing the memory usage from $O(nd)$ to just $O(d)$ and computation from $O(n^2)$ to $O(n)$, where $d$ denotes the model dimension.
Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam
1-bit gradient compression and local steps are two representative techniques that enable drastic communication reduction in distributed SGD.
A General Analysis of Example-Selection for Stochastic Gradient Descent
In this paper, we develop a broad condition on the sequence of examples used by SGD that is sufficient to prove tight convergence rates in both strongly convex and non-convex settings.
Progressive Joint Low-light Enhancement and Noise Removal for Raw Images
Low-light imaging on mobile devices is typically challenging due to insufficient incident light coming through the relatively small aperture, resulting in a low signal-to-noise ratio.
Variance Reduced Training with Stratified Sampling for Forecasting Models
In large-scale time series forecasting, one often encounters the situation where the temporal patterns of time series, while drifting over time, differ from one another in the same dataset.
Hyperparameter Optimization Is Deceiving Us, and How to Stop It
Recent empirical work shows that inconsistent results based on choice of hyperparameter optimization (HPO) configuration are a widespread problem in ML research.
Optimal Complexity in Decentralized Training
Decentralization is a promising method of scaling up parallel machine learning systems.
MixML: A Unified Analysis of Weakly Consistent Parallel Learning
Parallelism is a ubiquitous method for accelerating machine learning algorithms.
Moniqua: Modulo Quantized Communication in Decentralized SGD
Running Stochastic Gradient Descent (SGD) in a decentralized fashion has shown promising results.
