A Continual Development Methodology for Large-scale Multitask Dynamic ML Systems
The traditional Machine Learning (ML) methodology requires to fragment the development and experimental process into disconnected iterations whose feedback is used to guide design or tuning choices. This methodology has multiple efficiency and scalability disadvantages, such as leading to spend significant resources into the creation of multiple trial models that do not contribute to the final solution.The presented work is based on the intuition that defining ML models as modular and extensible artefacts allows to introduce a novel ML development methodology enabling the integration of multiple design and evaluation iterations into the continuous enrichment of a single unbounded intelligent system. We define a novel method for the generation of dynamic multitask ML models as a sequence of extensions and generalizations. We first analyze the capabilities of the proposed method by using the standard ML empirical evaluation methodology. Finally, we propose a novel continuous development methodology that allows to dynamically extend a pre-existing multitask large-scale ML system while analyzing the properties of the proposed method extensions. This results in the generation of an ML model capable of jointly solving 124 image classification tasks achieving state of the art quality with improved size and compute cost.
PDF AbstractCode
Datasets
CIFAR-10
ImageNet
MNIST
CIFAR-100
SVHN
Fashion-MNIST
STL-10
Caltech-101
DTD
Food-101
iNaturalist
EuroSAT
EMNIST
ImageNet-A
Stanford Online Products
ImageNet-LT
ImageNet-Sketch
PlantVillage
Stanford Dogs
Oxford-IIIT Pet Dataset
CARS196
Places365
UC Merced Land Use Dataset
Imagenette
Cats and Dogs
Malaria Dataset
