2 code implementations • ECCV 2020 • Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, Eduardo Valle
Lifelong learning has attracted much attention, but existing works still struggle to fight catastrophic forgetting and accumulate knowledge over long stretches of incremental learning.
1 code implementation • 11 Feb 2021 • Arthur Douillard, Timothée Lesort
Those drifts might cause interferences in the trained model and knowledge learned on previous states of the data distribution might be forgotten.
1 code implementation • 24 Jun 2020 • Arthur Douillard, Eduardo Valle, Charles Ollion, Thomas Robert, Matthieu Cord
Continual learning aims to learn tasks sequentially, with (often severe) constraints on the storage of old learning samples, without suffering from catastrophic forgetting.
1 code implementation • CVPR 2021 • Arthur Douillard, Yifu Chen, Arnaud Dapogny, Matthieu Cord
classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes.
Ranked #1 on Domain 11-5 on Cityscapes val
Class Incremental Learning Continual Semantic Segmentation +16
1 code implementation • 29 Jun 2021 • Arthur Douillard, Yifu Chen, Arnaud Dapogny, Matthieu Cord
classes predicted by the old model to deal with background shift and avoid catastrophic forgetting of the old classes.
Ranked #6 on Overlapped 15-1 on PASCAL VOC 2012
Class Incremental Learning Continual Semantic Segmentation +5
1 code implementation • CVPR 2022 • Arthur Douillard, Alexandre Ramé, Guillaume Couairon, Matthieu Cord
Our strategy scales to a large number of tasks while having negligible memory and time overheads due to strict control of the parameters expansion.
Ranked #2 on Incremental Learning on ImageNet - 10 steps
1 code implementation • 15 Nov 2022 • Jorg Bornschein, Alexandre Galashov, Ross Hemsley, Amal Rannen-Triki, Yutian Chen, Arslan Chaudhry, Xu Owen He, Arthur Douillard, Massimo Caccia, Qixuang Feng, Jiajun Shen, Sylvestre-Alvise Rebuffi, Kitty Stacpoole, Diego de Las Casas, Will Hawkins, Angeliki Lazaridou, Yee Whye Teh, Andrei A. Rusu, Razvan Pascanu, Marc'Aurelio Ranzato
A shared goal of several machine learning communities like continual learning, meta-learning and transfer learning, is to design algorithms and models that efficiently and robustly adapt to unseen tasks.
1 code implementation • 30 Apr 2022 • Oleksiy Ostapenko, Timothee Lesort, Pau Rodríguez, Md Rifat Arefin, Arthur Douillard, Irina Rish, Laurent Charlin
Motivated by this, we study the efficacy of pre-trained vision models as a foundation for downstream continual learning (CL) scenarios.
1 code implementation • 17 Jan 2024 • Bo Liu, Rachita Chhaparia, Arthur Douillard, Satyen Kale, Andrei A. Rusu, Jiajun Shen, Arthur Szlam, Marc'Aurelio Ranzato
Local stochastic gradient descent (Local-SGD), also referred to as federated averaging, is an approach to distributed optimization where each device performs more than one SGD update per communication.
1 code implementation • 25 Apr 2022 • Antoine Saporta, Arthur Douillard, Tuan-Hung Vu, Patrick Pérez, Matthieu Cord
Unsupervised Domain Adaptation (UDA) is a transfer learning task which aims at training on an unlabeled target domain by leveraging a labeled source domain.
no code implementations • 6 Oct 2020 • Alexandre Rame, Arthur Douillard, Charles Ollion
That's why in addition to a first color classifier, we include a second regression stage for refinement in our newly proposed architecture.
no code implementations • CVPR 2023 • Fabio Cermelli, Matthieu Cord, Arthur Douillard
%a In this paper, we present the first continual learning model capable of operating on both semantic and panoptic segmentation.
Ranked #2 on Continual Semantic Segmentation on ADE20K
no code implementations • 25 Apr 2023 • Massimo Caccia, Alexandre Galashov, Arthur Douillard, Amal Rannen-Triki, Dushyant Rao, Michela Paganini, Laurent Charlin, Marc'Aurelio Ranzato, Razvan Pascanu
The field of transfer learning is undergoing a significant shift with the introduction of large pretrained models which have demonstrated strong adaptability to a variety of downstream tasks.
no code implementations • 14 Nov 2023 • Arthur Douillard, Qixuan Feng, Andrei A. Rusu, Rachita Chhaparia, Yani Donchev, Adhiguna Kuncoro, Marc'Aurelio Ranzato, Arthur Szlam, Jiajun Shen
In this work, we propose a distributed optimization algorithm, Distributed Low-Communication (DiLoCo), that enables training of language models on islands of devices that are poorly connected.
no code implementations • 15 Mar 2024 • Arthur Douillard, Qixuan Feng, Andrei A. Rusu, Adhiguna Kuncoro, Yani Donchev, Rachita Chhaparia, Ionel Gog, Marc'Aurelio Ranzato, Jiajun Shen, Arthur Szlam
Progress in machine learning (ML) has been fueled by scaling neural network models.