Search Results for author:
Found 3 papers, 2 papers with code
Neural Mask Generator: Learning to Generate Adaptive Word Maskings for Language Model Adaptation
We propose a method to automatically generate a domain- and task-adaptive maskings of the given text for self-supervised pre-training, such that we can effectively adapt the language model to a particular target task (e. g. question answering).
Episodic Memory Reader: Learning What to Remember for Question Answering from Streaming Data
We consider a novel question answering (QA) task where the machine needs to read from large streaming data (long documents or videos) without knowing when the questions will be given, which is difficult to solve with existing QA methods due to their lack of scalability.
Learning What to Remember: Long-term Episodic Memory Networks for Learning from Streaming Data
We tackle this problem by proposing a memory network fit for long-term lifelong learning scenario, which we refer to as Long-term Episodic Memory Networks (LEMN), that features a RNN-based retention agent that learns to replace less important memory entries based on the retention probability generated on each entry that is learned to identify data instances of generic importance relative to other memory entries, as well as its historical importance.
