Pairing interacting protein sequences using masked language modeling

1 code implementation • 14 Aug 2023 • Umberto Lupo, Damiano Sgarbossa, Anne-Florence Bitbol

We introduce a method called DiffPALM that solves it by exploiting the ability of MSA Transformer to fill in masked amino acids in multiple sequence alignments using the surrounding context.

Masked Language Modeling Protein Language Model

Generative power of a protein language model trained on multiple sequence alignments

1 code implementation • 14 Apr 2022 • Damiano Sgarbossa, Umberto Lupo, Anne-Florence Bitbol

Moreover, for small protein families, our generation method based on MSA Transformer outperforms Potts models.

Masked Language Modeling Protein Design +1

Protein language models trained on multiple sequence alignments learn phylogenetic relationships

1 code implementation • 29 Mar 2022 • Umberto Lupo, Damiano Sgarbossa, Anne-Florence Bitbol

Simple combinations of MSA Transformer's row attentions have led to state-of-the-art unsupervised structural contact prediction.