An image speaks a thousand words, but can everyone listen? On translating images for cultural relevance

1 code implementation • 1 Apr 2024 • Simran Khanuja, Sathyanarayanan Ramamoorthy, Yueqi Song, Graham Neubig

First, we build three pipelines comprising state-of-the-art generative models to do the task.

Machine Translation Translation

IMAGINATOR: Pre-Trained Image+Text Joint Embeddings using Word-Level Grounding of Images

no code implementations • 12 May 2023 • Varuna Krishna, S Suryavardan, Shreyash Mishra, Sathyanarayanan Ramamoorthy, Parth Patwa, Megha Chakraborty, Aman Chadha, Amitava Das, Amit Sheth

We also evaluate pre-trained IMAGINATOR JEs on three downstream tasks: (i) image captioning, (ii) Image2Tweet, and (iii) text-based image retrieval.

Hyperparameter Optimization Image Captioning +3

Memotion Analysis through the Lens of Joint Embedding

1 code implementation • 13 Nov 2021 • Nethra Gunti, Sathyanarayanan Ramamoorthy, Parth Patwa, Amitava Das

Joint embedding (JE) is a way to encode multi-modal data into a vector space where text remains as the grounding key and other modalities like image are to be anchored with such keys.