Search Results for author:
Found 7 papers, 5 papers with code
An Introduction to Vision-Language Modeling
Then, we present and discuss approaches to evaluate VLMs.
Modeling Caption Diversity in Contrastive Vision-Language Pretraining
Contrastive Language Pretraining (CLIP) on the other hand, works by mapping an image and its caption to a single vector -- limiting how well CLIP-like models can represent the diverse ways to describe an image.
SPARO: Selective Attention for Robust and Compositional Transformer Encodings for Vision
Using SPARO, we demonstrate improvements on downstream recognition, robustness, retrieval, and compositionality benchmarks with CLIP (up to +14% for ImageNet, +4% for SugarCrepe), and on nearest neighbors and linear probe for ImageNet with DINO (+3% each).
Language Model Alignment with Elastic Reset
We periodically reset the online model to an exponentially moving average (EMA) of itself, then reset the EMA model to the initial model.
A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation
Self-Supervised Learning (SSL) models rely on a pretext task to learn representations.
Simplicial Embeddings in Self-Supervised Learning and Downstream Classification
Simplicial Embeddings (SEM) are representations learned through self-supervised learning (SSL), wherein a representation is projected into $L$ simplices of $V$ dimensions each using a softmax operation.
Integrating Categorical Semantics into Unsupervised Domain Translation
While unsupervised domain translation (UDT) has seen a lot of success recently, we argue that mediating its translation via categorical semantic features could broaden its applicability.
