MAIRA-1: A specialised large multimodal model for radiology report generation

no code implementations • 22 Nov 2023 • Stephanie L. Hyland, Shruthi Bannur, Kenza Bouzid, Daniel C. Castro, Mercy Ranjit, Anton Schwaighofer, Fernando Pérez-García, Valentina Salvatelli, Shaury Srivastav, Anja Thieme, Noel Codella, Matthew P. Lungren, Maria Teodora Wetscherek, Ozan Oktay, Javier Alvarez-Valle

We present a radiology-specific multimodal model for the task for generating radiological reports from chest X-rays (CXRs).

Data Augmentation Language Modelling +1

Breaking Language Barriers with a LEAP: Learning Strategies for Polyglot LLMs

no code implementations • 28 May 2023 • Akshay Nambi, Vaibhav Balloli, Mercy Ranjit, Tanuja Ganu, Kabir Ahuja, Sunayana Sitaram, Kalika Bali

Our results show substantial advancements in multilingual understanding and generation across a diverse range of languages.

Question Answering Retrieval

Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models

no code implementations • 5 May 2023 • Mercy Ranjit, Gopinath Ganapathy, Ranjit Manuel, Tanuja Ganu

We propose Retrieval Augmented Generation (RAG) as an approach for automated radiology report writing that leverages multimodally aligned embeddings from a contrastively pretrained vision language model for retrieval of relevant candidate radiology text for an input radiology image and a general domain generative model like OpenAI text-davinci-003, gpt-3. 5-turbo and gpt-4 for report generation using the relevant radiology text retrieved.

Instruction Following Language Modelling +1