Search Results for author:
Found 28 papers, 7 papers with code
Composable Function-preserving Expansions for Transformer Architectures
Training state-of-the-art neural networks requires a high cost in terms of compute and time.
Multipath agents for modular multitask ML systems
Diverse agents can compete to produce the best performing model for a task by reusing the modules introduced to the system by competing agents.
A Multiagent Framework for the Asynchronous and Collaborative Extension of Multitask ML Systems
We believe that this novel methodology for ML development can be demonstrated through a modularized representation of ML models and the definition of novel abstractions allowing to implement and execute diverse methods for the asynchronous use and extension of modular intelligent systems.
A Continual Development Methodology for Large-scale Multitask Dynamic ML Systems
This methodology has multiple efficiency and scalability disadvantages, such as leading to spend significant resources into the creation of multiple trial models that do not contribute to the final solution. The presented work is based on the intuition that defining ML models as modular and extensible artefacts allows to introduce a novel ML development methodology enabling the integration of multiple design and evaluation iterations into the continuous enrichment of a single unbounded intelligent system.
Ranked #1 on
Image Classification
on cats_vs_dogs
An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems
Multitask learning assumes that models capable of learning from multiple tasks can achieve better quality and efficiency via knowledge transfer, a key feature of human learning.
Ranked #1 on
Image Classification
on KMNIST
muNet: Evolving Pretrained Deep Neural Networks into Scalable Auto-tuning Multitask Systems
We propose a method that uses the layers of a pretrained deep neural network as building blocks to construct an ML system that can jointly solve an arbitrary number of tasks.
Scaling Up Models and Data with $\texttt{t5x}$ and $\texttt{seqio}$
Recent neural network-based language models have benefited greatly from scaling up the size of training datasets and the number of parameters in the models themselves.
Routing Networks with Co-training for Continual Learning
We propose the use of sparse routing networks for continual learning.
Ranking architectures using meta-learning
However, it requires a large amount of computing resources and in order to alleviate this, a performance prediction network has been recently proposed that enables efficient architecture search by forecasting the performance of candidate architectures, instead of relying on actual model training.
Flexible Multi-task Networks by Learning Parameter Allocation
The binary allocation variables are learned jointly with the model parameters by standard back-propagation thanks to the Gumbel-Softmax reparametrization method.
Ranked #1 on
Multi-Task Learning
on OMNIGLOT
Evo-NAS: Evolutionary-Neural Hybrid Agent for Architecture Search
We show that the Evo-NAS agent outperforms both neural and evolutionary agents when applied to architecture search for a suite of text and image classification benchmarks.
Gumbel-Matrix Routing for Flexible Multi-task Learning
We propose the Gumbel-Matrix routing, a novel multi-task routing method based on the Gumbel-Softmax, that is designed to learn fine-grained parameter sharing.
Temporal Coding in Spiking Neural Networks with Alpha Synaptic Function: Learning with Backpropagation
The timing of individual neuronal spikes is essential for biological brains to make fast responses to sensory stimuli.
Transfer NAS: Knowledge Transfer between Search Spaces with Transformer Agents
Recent advances in Neural Architecture Search (NAS) have produced state-of-the-art architectures on several tasks.
Fast Task-Aware Architecture Inference
At the core of our framework lies a deep value network that can predict the performance of input architectures on a task by utilizing task meta-features and the previous model training experiments performed on related tasks.
Parameter-Efficient Transfer Learning for NLP
On GLUE, we attain within 0. 4% of the performance of full fine-tuning, adding only 3. 6% parameters per task.
Ranked #4 on
Image Classification
on OmniBenchmark
(using extra training data)
Neural Architecture Search Over a Graph Search Space
However, the success of NAS depends on the definition of the search space.
Evolutionary-Neural Hybrid Agents for Architecture Search
We show that the Evo-NAS agent outperforms both neural and evolutionary agents when applied to architecture search for a suite of text and image classification benchmarks.
Transfer Learning with Neural AutoML
We extend RL-based architecture search methods to support parallel training on multiple tasks and then transfer the search strategy to new tasks.
Analyzing Language Learned by an Active Question Answering Agent
We analyze the language learned by an agent trained with reinforcement learning as a component of the ActiveQA system [Buck et al., 2017].
Transfer Learning to Learn with Multitask Neural Model Search
We demonstrate that MNMS can conduct an automated architecture search for multiple tasks simultaneously while still learning well-performing, specialized models for each task.
Ask the Right Questions: Active Question Reformulation with Reinforcement Learning
The agent probes the system with, potentially many, natural language reformulations of an initial question and aggregates the returned evidence to yield the best answer.
Lemmatising Serbian as Category Tagging with Bidirectional Sequence Classification
We present a novel tool for morphological analysis of Serbian, which is a low-resource language with rich morphology.
