Search Results for author:
Found 8 papers, 3 papers with code
Provable Tempered Overfitting of Minimal Nets and Typical Nets
We study the overfitting behavior of fully connected deep Neural Networks (NNs) with binary weights fitted to perfectly classify a noisy training set.
The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting -- An Analytical Model
Previous works have analyzed separately how forgetting is affected by either task similarity or overparameterization.
Continual Learning in Linear Classification on Separable Data
We analyze continual learning on a sequence of separable linear classification tasks with binary labels.
The Role of Codeword-to-Class Assignments in Error-Correcting Codes: An Empirical Study
Error-correcting codes (ECC) are used to reduce multiclass classification tasks to multiple binary classification subproblems.
How catastrophic can catastrophic forgetting be in linear regression?
In specific settings, we highlight differences between forgetting and convergence to the offline solution as studied in those areas.
Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
While mainstream deep learning methods train the neural networks weights while keeping the network architecture fixed, the emerging neural architecture search (NAS) techniques make the latter also amenable to training.
How do infinite width bounded norm networks look in function space?
We consider the question of what functions can be captured by ReLU networks with an unbounded number of units (infinite width), but where the overall network Euclidean norm (sum of squares of all weights in the system, except for an unregularized bias term for each unit) is bounded; or equivalently what is the minimal norm required to approximate a given function.
Efficient Loss-Based Decoding on Graphs For Extreme Classification
We build on a recent extreme classification framework with logarithmic time and space, and on a general approach for error correcting output coding (ECOC) with loss-based decoding, and introduce a flexible and efficient approach accompanied by theoretical bounds.
