Improving Text-to-Image Consistency via Automatic Prompt Optimization

no code implementations • 26 Mar 2024 • Oscar Mañas, Pietro Astolfi, Melissa Hall, Candace Ross, Jack Urbanek, Adina Williams, Aishwarya Agrawal, Adriana Romero-Soriano, Michal Drozdzal

In this paper, we address these challenges and introduce a T2I optimization-by-prompting framework, OPT2I, which leverages a large language model (LLM) to improve prompt-image consistency in T2I models.

Language Modelling Large Language Model

Improving Automatic VQA Evaluation Using Large Language Models

no code implementations • 4 Oct 2023 • Oscar Mañas, Benno Krojer, Aishwarya Agrawal

Thus, there is a need to develop more robust automatic VQA metrics that serve as a proxy for human judgment.

In-Context Learning Question Answering +1

MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting

1 code implementation • 13 Oct 2022 • Oscar Mañas, Pau Rodriguez, Saba Ahmadi, Aida Nematzadeh, Yash Goyal, Aishwarya Agrawal

Large pre-trained models have proved to be remarkable zero- and (prompt-based) few-shot learners in unimodal vision and language tasks.

Image Captioning Question Answering +1

Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data

3 code implementations • ICCV 2021 • Oscar Mañas, Alexandre Lacoste, Xavier Giro-i-Nieto, David Vazquez, Pau Rodriguez

Transfer learning approaches can reduce the data requirements of deep learning algorithms.

Change Detection Self-Supervised Learning +2

A Weakly Supervised Consistency-based Learning Method for COVID-19 Segmentation in CT Images

3 code implementations • 4 Jul 2020 • Issam Laradji, Pau Rodriguez, Oscar Mañas, Keegan Lensink, Marco Law, Lironne Kurzman, William Parker, David Vazquez, Derek Nowrouzezahrai

Thus, we propose a consistency-based (CB) loss function that encourages the output predictions to be consistent with spatial transformations of the input images.