Practical Knowledge Distillation: Using DNNs to Beat DNNs

no code implementations • 23 Feb 2023 • Chung-Wei Lee, Pavlos Athanasios Apostolopulos, Igor L. Markov

While gradient boosting is known to outperform DNNs on tabular data, we close the gap for datasets with 100K+ rows and give DNNs an advantage on small data sets.

Denoising Knowledge Distillation