Search Results for author:
Found 9 papers, 3 papers with code
AdaContour: Adaptive Contour Descriptor with Hierarchical Representation
Existing angle-based contour descriptors suffer from lossy representation for non-starconvex shapes.
The Efficiency Spectrum of Large Language Models: An Algorithmic Survey
The rapid growth of Large Language Models (LLMs) has been a driving force in transforming various domains, reshaping the artificial general intelligence landscape.
DREAM: Diffusion Rectification and Estimation-Adaptive Models
We present DREAM, a novel training framework representing Diffusion Rectification and Estimation Adaptive Models, requiring minimal code changes (just three lines) yet significantly enhancing the alignment of training with sampling in diffusion models.
Generalized Neural Collapse for a Large Number of Classes
However, most of the existing empirical and theoretical studies in neural collapse focus on the case that the number of classes is small relative to the dimension of the feature space.
Principled and Efficient Transfer Learning of Deep Models via Neural Collapse
As model size continues to grow and access to labeled training data remains limited, transfer learning has become a popular approach in many scientific and engineering fields.
Are All Losses Created Equal: A Neural Collapse Perspective
We extend such results and show through global solution and landscape analyses that a broad family of loss functions including commonly used label smoothing (LS) and focal loss (FL) exhibits Neural Collapse.
A Validation Approach to Over-parameterized Matrix and Image Recovery
In this paper, we study the problem of recovering a low-rank matrix from a number of noisy random linear measurements.
On the Optimization Landscape of Neural Collapse under MSE Loss: Global Optimality with Unconstrained Features
When training deep neural networks for classification tasks, an intriguing empirical phenomenon has been widely observed in the last-layer classifiers and features, where (i) the class means and the last-layer classifiers all collapse to the vertices of a Simplex Equiangular Tight Frame (ETF) up to scaling, and (ii) cross-example within-class variability of last-layer activations collapses to zero.
A Geometric Analysis of Neural Collapse with Unconstrained Features
In contrast to existing landscape analysis for deep neural networks which is often disconnected from practice, our analysis of the simplified model not only does it explain what kind of features are learned in the last layer, but it also shows why they can be efficiently optimized in the simplified settings, matching the empirical observations in practical deep network architectures.
