Deep neural networks (DNNs) have been widely used for decision making, prompting a surge of interest in interpreting how these complex models work.
While different language models are ubiquitous in NLP, it is hard to contrast their outputs and identify which contexts one can handle better than the other.
Saliency methods -- techniques to identify the importance of input features on a model's output -- are a common step in understanding neural network behavior.
Natural language generation (NLG) for storytelling is especially challenging because it requires the generated text to follow an overall theme while remaining creative and diverse to engage the reader.
In this work we study a mathematical formalization of this network motif and apply it to learning the correlational structure between words and their context in a corpus of unstructured text, a common natural language processing (NLP) task.
Large Transformer-based language models can route and reshape complex information via their multi-headed attention mechanism.
no code implementations • • Vijil Chenthamarakshan, Payel Das, Samuel C. Hoffman, Hendrik Strobelt, Inkit Padhi, Kar Wai Lim, Benjamin Hoover, Matteo Manica, Jannis Born, Teodoro Laino, Aleksandra Mojsilovic
CogMol also includes insilico screening for assessing toxicity of parent molecules and their metabolites with a multi-task toxicity classifier, synthetic feasibility with a chemical retrosynthesis predictor, and target structure binding with docking simulations.
We present exBERT, an interactive tool named after the popular BERT language model, that provides insights into the meaning of the contextual representations by matching a human-specified input to similar contexts in a large annotated dataset.