Search Results for author:
Found 7 papers, 2 papers with code
Incremental Learning via Rate Reduction
Current deep learning architectures suffer from catastrophic forgetting, a failure to retain knowledge of previously learned classes when incrementally trained on new classes.
Assessing Generalization of SGD via Disagreement
We empirically show that the test error of deep networks can be estimated by simply training the same architecture on the same training set but with a different run of Stochastic Gradient Descent (SGD), and measuring the disagreement rate between the two networks on unlabeled test data.
Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learning
In both OWSP and OWMP settings, we demonstrate that adding {\em intermediate rewards} to subgoals is more computationally efficient than only rewarding the agent once it completes the goal of reaching a terminal state.
Efficient Maximal Coding Rate Reduction by Variational Forms
The principle of Maximal Coding Rate Reduction (MCR$^2$) has recently been proposed as a training objective for learning discriminative low-dimensional structures intrinsic to high-dimensional data to allow for more robust training than standard approaches, such as cross-entropy minimization.
Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift
In this paper, we show a similar but surprising phenomenon also holds for the agreement between pairs of neural network classifiers: whenever accuracy-on-the-line holds, we observe that the OOD agreement between the predictions of any two pairs of neural networks (with potentially different architectures) also observes a strong linear correlation with their ID agreement.
On the Joint Interaction of Models, Data, and Features
Thus, we believe this work provides valuable new insight into our understanding of feature learning.
Predicting the Performance of Foundation Models via Agreement-on-the-Line
Recently, it was shown that ensembles of neural networks observe the phenomena ``agreement-on-the-line'', which can be leveraged to reliably predict OOD performance without labels.
