PropTest: Automatic Property Testing for Improved Visual Programming

no code implementations • 25 Mar 2024 • Jaywon Koo, Ziyan Yang, Paola Cascante-Bonilla, Baishakhi Ray, Vicente Ordonez

We propose PropTest, a general strategy that improves visual programming by further using an LLM to generate code that tests for visual properties in an initial round of proposed solutions.

Question Answering Referring Expression +3

Multimodal Multi-loss Fusion Network for Sentiment Analysis

1 code implementation • 1 Aug 2023 • Zehui Wu, Ziwei Gong, Jaywon Koo, Julia Hirschberg

This paper investigates the optimal selection and fusion of feature encoders across multiple modalities and combines these in one neural network to improve sentiment detection.

feature selection Multimodal Sentiment Analysis

Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities

no code implementations • 14 Jun 2022 • Hammad A. Ayyubi, Christopher Thomas, Lovish Chum, Rahul Lokesh, Long Chen, Yulei Niu, Xudong Lin, Xuande Feng, Jaywon Koo, Sounak Ray, Shih-Fu Chang

To support research on this task, we introduce the Multimodal Hierarchical Events (MultiHiEve) dataset.