Accurate computational approaches are needed to address this gap and to enable large-scale structural bioinformatics.
We present a method for combining multi-agent communication and traditional data-driven approaches to natural language learning, with an end goal of teaching agents to communicate with humans in natural language.
We present the Compressive Transformer, an attentive sequence model which compresses past memories for long-range sequence learning.
Ranked #2 on Language Modelling on Hutter Prize
Previous research on word embeddings has shown that sparse representations, which can be either learned on top of existing dense embeddings or obtained through model constraints during training time, have the benefit of increased interpretability properties: to some degree, each dimension can be understood by a human and associated with a recognizable feature in the data.
We consider probabilistic topic models and more recent word embedding techniques from a perspective of learning hidden semantic representations.