Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy

no code implementations • 4 Dec 2023 • Danqi Liao, Chen Liu, Benjamin W. Christensen, Alexander Tong, Guillaume Huguet, Guy Wolf, Maximilian Nickel, Ian Adelstein, Smita Krishnaswamy

Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to compute reliably in high dimensions.

Sentence Embeddings using Supervised Contrastive Learning

1 code implementation • 9 Jun 2021 • Danqi Liao

Specifically our method fine-tunes pretrained BERT on SNLI data, incorporating both supervised crossentropy loss and supervised contrastive loss.

Contrastive Learning Semantic Textual Similarity +3